Hadoop CDH4伪分布式安装

[kevin@localhost ~]$ sudo ./jdk-6u34-linux-x64-rpm.bin

[kevin@localhost ~]$ sudo cp jdk-6u34-linux-amd64.rpm /usr/local
[kevin@localhost ~]$ cd /usr/local/
[kevin@localhost local]$ sudo chmod +x jdk-6u34-linux-amd64.rpm
[kevin@localhost local]$ sudo rpm -ivh jdk-6u34-linux-amd64.rpm

[kevin@localhost jdk1.6.0_34]$ sudo vi /etc/profile

添加JDK的安装目录，具体添加内容如下：

export JAVA_HOME=/usr/java/jdk1.6.0_34

export PATH=$PATH:$JAVA_HOME/bin

export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

Hadoop CHD4是基本Apache Hadoop 2.0，Map/Reduce框架使用Yarn。可以通用yum来安装（sudo yum install hadoop-conf-pseudo）。为了了解Hadoop的依赖关系，这里介绍使用rpm来安装，手动解决相应的依赖关系。具体的rpm包下载地址( Hadoop CDH4 RPM)

[kevin@localhost ~]$ chmod +x hadoop-conf-pseudo-2.0.0+545-1.cdh4.1.1.p0.5.el5.x86_64.rpm
[kevin@localhost ~]$ sudo rpm -ivh hadoop-conf-pseudo-2.0.0+545-1.cdh4.1.1.p0.5.el5.x86_64.rpm
[sudo] password for kevin:
warning: hadoop-conf-pseudo-2.0.0+545-1.cdh4.1.1.p0.5.el5.x86_64.rpm: Header V4 DSA signature: NOKEY, key ID e8f86acd
error: Failed dependencies:
hadoop = 2.0.0+545-1.cdh4.1.1.p0.5.el5 is needed by hadoop-conf-pseudo-2.0.0+545-1.cdh4.1.1.p0.5.el5.x86_64
hadoop-hdfs-namenode = 2.0.0+545-1.cdh4.1.1.p0.5.el5 is needed by hadoop-conf-pseudo-2.0.0+545-1.cdh4.1.1.p0.5.el5.x86_64
hadoop-hdfs-datanode = 2.0.0+545-1.cdh4.1.1.p0.5.el5 is needed by hadoop-conf-pseudo-2.0.0+545-1.cdh4.1.1.p0.5.el5.x86_64
hadoop-hdfs-secondarynamenode = 2.0.0+545-1.cdh4.1.1.p0.5.el5 is needed by hadoop-conf-pseudo-2.0.0+545-1.cdh4.1.1.p0.5.el5.x86_64
hadoop-yarn-resourcemanager = 2.0.0+545-1.cdh4.1.1.p0.5.el5 is needed by hadoop-conf-pseudo-2.0.0+545-1.cdh4.1.1.p0.5.el5.x86_64
hadoop-yarn-nodemanager = 2.0.0+545-1.cdh4.1.1.p0.5.el5 is needed by hadoop-conf-pseudo-2.0.0+545-1.cdh4.1.1.p0.5.el5.x86_64
hadoop-mapreduce-historyserver = 2.0.0+545-1.cdh4.1.1.p0.5.el5 is needed by hadoop-conf-pseudo-2.0.0+545-1.cdh4.1.1.p0.5.el5.x86_64

报如上依赖关系错误，可以清楚的看到hadoop-conf-pseudo依赖hadoop, hadoop-hdfs-namenode, hadoop-hdfs-datanode, hadoop-hdfs-secondarynamenode, hadoop-yarn-resourcemanager, hadoop-yarn-nodemanager, hadoop-mapreduce-historyserver，由于伪分布所有的组件都运行在同一台机器上，所以需要安装上面这些依赖包。

具体的hadoop环境参数如下：

/etc/hadoop/conf （具体的配置文件）

/etc/init.d （所有相关的启动脚本）

/usr/lib （具体安装目录包括Hadoop, hdfs, yarn, map/reduce）

安装完成之后，会默认创建几个用户（zookeeper, hdfs, yarn, mapred），及相应的组（zookeeper, hadoop, hdfs, yarn, mapred）

Hadoop已经安装成功，在启动hadoop之前，需要对namenode进行format，format namenode必须使用hdfs这个用户，如下：

[kevin@localhost init.d]$ sudo -u hdfs hdfsnamenode -format

$ for service in /etc/init.d/hadoop-hdfs-*
> do
> sudo $service start
> done

可以通过sudo jps查看启来的java进程

启动正常，可以通过hadoop fs -ls / 来访问HDFS文件系统，通过http://192.168.16.129:50070访问web console的时候不能访问，查看防火墙是启来的，应该防火墙的导致，暂时先把防火墙关闭之后，重新访问正常。

创建Hadoop HDFS初始目录