storm0.92集群安装配置完整版

1、准备四台Linux操作系统的主机(本人用的是Oracle Linux 6 update 5 x86_64bit操作系统)

 

        192.168.1.15

        192.168.1.16

        192.168.1.17

        192.168.1.18

 

        以上四台主机在按装操作系统时，只选择安装base组件及Oracle Linux Server组件就Ok了。

 

2、下载jdk1.7

        http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html

 

3、安装jdk1.7  (四台主机都要执行下面的命令来安装jdk)

        [root@Master    ~  ]# cd /usr

        [root@Master usr  ]# mkdir java

        [root@Master usr  ]# cd java

        [root@Master java]# 

        复制jdk1.7的安装文件到/usr/java下面

        [root@Master java]# tar zxvf jdk-7u60-linux-x64.tar.gz

        [root@Master java]# ln -snf jdk1.7.0_60  jdk

 

4、设置环境变量(同样四台主机都要执行下面的命令来设置环境变量)

        [root@Master java]# cd

        [root@Master   ~   ]# vi .bash_profile

 

        加入以下几行，用于给root用户设置JAVA_HOME环境变量

 

        export JAVA_HOME=/usr/java/jdk
        export PATH=$JAVA_HOME/bin:$PATH
        export CLASSPATH=$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$CLASSPATH

 

        保存退出。

        

        执行以下命令使设置生效，并测试设置是否成功

        [root@Slave3 ~]# source .bash_profile
        [root@Slave3 ~]# java -version
        java version "1.7.0_60"
        Java(TM) SE Runtime Environment (build 1.7.0_60-b19)
        Java HotSpot(TM) 64-Bit Server VM (build 24.60-b09, mixed mode)

        

5、搭建zookeeper集群环境(zookeeper集群的节点数最好为奇数个)

        关于zookeeper集群的配置只在3台机器上进行

        192.168.1.16

        192.168.1.17

        192.168.1.18

 

        首先要下载zookeeper软件

        http://zookeeper.apache.org/releases.html

 

        目前最新的版本是3.4.6

        

6、复制zookeeper-3.4.6.tar.gz 安装文件到某个目录下面(本人使用的是/u01目录)

        

        然后解压 zookeeper-3.4.6.tar.gz

        

        [root@Master u01]# tar zxvf zookeeper-3.4.6.tar.gz

 

7、配置zookeeper集群

        a、设置zookeeper环境变量

                [root@Master ~]# cd

                [root@Master ~]# vi .bash_profile

            

                加入以下两条语句，设置zookeeper环境变量

                export ZOOKEEPER_HOME=/u01/zookeeper-3.4.6
                export PATH=$PATH:$ZOOKEEPER_HOME/bin

    

                使设置生效

                [root@Master ~]# source .bash_profile

 

        b、  配置zookeeper

                [root@Master ~]# cd /u01/zookeeper-3.4.6/conf

                [root@Master conf]# cp zoo_sample.cfg   zoo.cfg

                [root@Master conf]# vi zoo.cfg

                tickTime=2000
                # The number of ticks that the initial
                # synchronization phase can take
                initLimit=10
                # The number of ticks that can pass between
                # sending a request and getting an acknowledgement
                syncLimit=5
                # the directory where the snapshot is stored.
                # do not use /tmp for storage, /tmp here is just
                # example sakes.
                dataDir=/storm/zookeeper/data
                dataLogDir=/storm/zookeeper/dataLog
                # the port at which the clients will connect
                clientPort=2181
                server.1=192.168.1.16:2888:3888
                server.2=192.168.1.17:2888:3888
                server.3=192.168.1.18:2888:3888
                # the maximum number of client connections.
                # increase this if you need to handle more clients
                #maxClientCnxns=60
                #
                # Be sure to read the maintenance section of the
                # administrator guide before turning on autopurge.
                #
                # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
                #
                # The number of snapshots to retain in dataDir
                #autopurge.snapRetainCount=3
                # Purge task interval in hours
                # Set to "0" to disable auto purge feature
                #autopurge.purgeInterval=1

 

                保存退出。

                 

                创建配置文件中用到的目录(也就是zookeeper的数据目录和日志目录)

                [root@Slave1 conf]# mkdir -p /storm/zookeeper/data

                [root@Slave1 conf]# mkdir -p /storm/zookeeper/dataLog

 

        c、在/storm/zookeeper/data目录下新建一个文件，命名为myid

                myid文件中只包含一个数字，用于标识当前服务器在zookeeper集群中的id号，根据

                server.1=192.168.1.16:2888:3888
                server.2=192.168.1.17:2888:3888
                server.3=192.168.1.18:2888:3888
                中的IP地址对应的server.后面的数字来确定当前主机的id号

 

                如：上面的配置说明，192.168.1.16这台主机的myid中的数字应该为1

                                                        192.168.1.17这台主机的myid中的数字应该为2

                                                        192.168.1.18这台主机的myid中的数字应该为3

 

        d、在192.168.1.16  \   192.168.1.17   \  192.168.1.18三台主机上启动zookeeper

            [root@Slave1 data]# zkServer.sh start            

            然后分别在三台主机上使用zkServer.sh status命令来查看当前的系统集群状态

            [root@Slave1 data]# zkServer.sh status
            JMX enabled by default
            Using config: /u01/zookeeper-3.4.6/bin/../conf/zoo.cfg
            Mode: follower

 

            [root@Slave2 data]# zkServer.sh status
            JMX enabled by default
            Using config: /u01/zookeeper-3.4.6/bin/../conf/zoo.cfg
            Mode: leader

 

            [root@Slave3 data]# zkServer.sh status
            JMX enabled by default
            Using config: /u01/zookeeper-3.4.6/bin/../conf/zoo.cfg
            Mode: follower

 

            从上面的红色部分可以看出，zookeeper中有两种角色，一种是leader，一种是follower，至此，zookeeper集群就配置完成了。

 

 8、安装Python2.7.7 (storm要用到Python语言)    (四台主机上都要安装)

        

            a、下载Python2.7.7   https://www.python.org/download/releases/2.7.7/

            b、复制Python-2.7.7.tgz到/u01下面

            c、解压Python   [root@Master u01]# tar zxvf Python-2.7.7.tgz

            d、进入/u01/Python-2.7.7目录中，执行以下命令安装Python2.7.7

                    [root@Master Python-2.7.7]# ./configure

                    [root@Master Python-2.7.7]# make

                    [root@Master Python-2.7.7]# make install

             e、修改/etc/ld.so.conf文件，在该文件中加入/usr/local/lib，加完后在内容如下：

                    include ld.so.conf.d/*.conf /usr/local/lib

             f、 执行以下命令使上一步的修改生效

                    [root@Master lib]# sudo ldconfig

              至此，Python2.7.7就安装配置完成了。

 

 

 9、安装strom依赖库(ZeroMQ，JZMQ)

            a、准备工作 

                   首先要安装几个系统包，因为在安装ZeroMQ和JZMQ 时，这两个软件要用到

 

                    [root@Master lib]# yum install uuid -y

                    [root@Master lib]# yum install e2fsprogs -y

                    [root@Master lib]# yum install libuuid*

                    [root@Slave3 lib]# yum install libtool -y

                    [root@Slave3 lib]# yum install *c++* -y

                    [root@Slave3 lib]# yum install git -y

            

                    然后修改/etc/hosts文件，修改后的内容如下：

                    127.0.0.1         localhost localhost.localdomain
                    ::1               localhost localhost.localdomain
                    192.168.1.15      Master
                    192.168.1.16      Slave1
                    192.168.1.17      Slave2
                    192.168.1.18      Slave3 

             

            b、安装ZeroMQ

                    下载ZeroMQ软件  https://github.com/zeromq/zeromq2-x/releases

                    

                    复制下载的安装文件zeromq2-x-2.2.0.tar.gz到/u01下面，然后解压缩

                    [root@Master u01]# tar zxvf zeromq2-x-2.2.0.tar.gz

 

                    然后进入到/u01/zeromq2-x-2.2.0目录中，执行以下几条命令，安装zeromq软件

                     [root@Master zeromq2-x-2.2.0]# ./autogen.sh

                     [root@Master zeromq2-x-2.2.0]# ./configure

                     [root@Master zeromq2-x-2.2.0]# make

                     [root@Master zeromq2-x-2.2.0]# make install

                     [root@Master zeromq2-x-2.2.0]# sudo ldconfig

 

                    至此，ZeroMQ就安装完成了

 

            c、安装JZMQ

                    下载JZMQ软件 (JZMQ软件要使用git工具下载源码进行编译，所以主机要求能连上网)

                     [root@Master u01]# git clone git://github.com/nathanmarz/jzmq.git

                     然后使用下面的命令进行安装

                     [root@Master jzmq]# ./autogen.sh

                     [root@Master jzmq]# ./configure

                     [root@Master jzmq]# make

                     [root@Master jzmq]# make install

                     [root@Master lib]# sudo ldconfig

                    至此，JZMQ也安装完成了

 

10、安装storm

                    下载storm   http://storm.incubator.apache.org/downloads.html

                    

            a、复制apache-storm-0.9.2-incubating.tar.gz 到/u01下面

                    [root@Master u01]# tar zxvf  apache-storm-0.9.2-incubating.tar.gz

                    [root@Master u01]# mv apache-storm-0.9.2-incubating   storm-0.9.2

 

                    修改环境变量

                    [root@Master ~]# vi .bash_profile

 

                    加入下面两句

                    export STORM_HOME=/u01/storm-0.9.2
                    export PATH=$PATH:$STORM_HOME/bin

                     

                    执行下面的命令使配置生效

                    [root@Master ~]# source .bash_profile

 

            b、 配置storm

                    进入到/u01/storm-0.9.2/conf下面

                    编辑storm.yaml配置文件

                

                    加入以下内容：

                    storm.zookeeper.servers:
                        - "192.168.1.16"
                        - "192.168.1.17"
                        - "192.168.1.18"
 
                    nimbus.host: "192.168.1.15"

                    supervisor.slots.ports:
                        - 6700
                        - 6701
                        - 6702
                        - 6703
                        - 6704

 

                    至此，storm整个就配置完成了。

 

 11、启动集群

                    a、启动zookeeper集群

                    b、在192.168.1.15上执行  storm  nimbus &   命令启动nimbus服务  (相当于Hadoop中的NameNode节点)

                    c、在192.168.1.15上执行 storm  ui &   命令启动ui图形管理器

                    d、在其他三个节点上，执行 storm supervisor & 命令启动supervisor服务

 

12、登陆http://192.168.1.15:8080/查看UI

                     

                    

             

本文出自 “涛哥的回忆” 博客，谢绝转载！

转自：http://562512305.blog.51cto.com/6496478/1432494