Sunday, November 8, 2015

Hadoop Configuration on Windows

Command for hadoop configuration
Install Cgwin .
Install openssl and openssh
Start the sshd service
net start sshd

Create the appropriate SSH keys for your user account

After successfull installation 
Make changes in following files
hadoop-env.sh
export JAVA_HOME="C:\hadoop\jdk1.7.0_17"
core-site.xml
mapred -site.xml
hdfs -site.xml

Format the NameNode
bin/hadoop namenode -format
Start data node 
bin/hadoop/datanode 
Start name node 
bin/hadoop/namenode
Start Secondary Name Node
bin/hadoop secondarynamenode

Start Job Tracker
bin/hadoop jobtracker
Start Task Tracker 
bin/hadoop tasktracker
bin/hadoop jobtracker Browse for name node
http://localhost:50070/
For Job Tracker
http://localhost:50030/jobtracker.jsp

Load data on data node
cd hadoop-0.19.1
bin/hadoop fs -mkdir In
bin/hadoop fs -put *.txt In

When you browse
http://localhost:50070/dfshealth.jsp
Browse the filesystem you will see newly uploaded data



No comments:

Post a Comment