storm0.9-single

参考

http://shiyanjun.cn/archives/241.html

用新版包自己做了一遍

(1)安装zeromq4&jzmq

[安装依赖]
yum install uuid*
yum install libtool
yum install libuuid
yum install libuuid-devel

[安装zeromq]
cd zeromq-4.0.4
./configure (不要使用--prefix指定路径，我试着在这里指定了一下结果后面jzmq就找不到zmq.h)
make
make install

[安装jzmq]
git clone git://github.com/nathanmarz/jzmq.git
cd jzmq/
./autogen.sh
./configure
make
make install

(2)安装zookeeper

vi ~/.bashrc

#ZooKeeper
export ZOOKEEPER_HOME=/usr/local/zookeeper-3.4.6
export PATH=$PATH:$ZOOKEEPER_HOME/bin

source ~/.bashrc

(3)安装storm

[环境变量]

vi ~/.bashrc

#Storm
export STORM_HOME=/home/hadoop/soft/apache-storm-0.9.1-incubating
export PATH=$PATH:$STORM_HOME/bin

source ~/.bashrc

[配置]

vi ./conf/storm.yaml (注意yaml格式的要求,空格tab,:和-前后都有空格)

storm.zookeeper.servers:
        - "localhost"

storm.zookeeper.port : 2181
nimbus.host : "localhost"
storm.local.dir : "/home/hadoop/soft/apache-storm-0.9.1-incubating/tmp"
supervisor.slots.ports:
　　- 6700

　　- 6701

　　- 6702

(4)运行示例

[下载示例源码编译打包]

git clone https://github.com/nathanmarz/storm-starter.git
cd storm-starter/
mvn -f m2-pom.xml package

[启动服务]

启动zookeeper

./zkServer.sh start

启动storm

storm help 可以看到有很多的命令
Commands:
    activate
    classpath
    deactivate
    dev-zookeeper
    drpc
    help
    jar
    kill
    list
    localconfvalue
    logviewer
    nimbus
    rebalance
    remoteconfvalue
    repl
    shell
    supervisor
    ui
    version

Help:
    help
    help <command>

storm nimbus (启动积雨云)


storm supervisor  (启动工作节点)

storm ui  (启动web界面)

[hadoop@localhost logs]$ netstat -an|grep 6700
[hadoop@localhost logs]$ netstat -an|grep 6701
[hadoop@localhost logs]$ netstat -an|grep 6702

此时三个工作端口均未启动

web界面查看

http://localhost:8080/

[不带Topology名提交Topology]

storm jar ../storm-starter/target/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar storm.starter.WordCountTopology

这中情况不会提交到nimbus去运行

[提交Topology]

storm jar ../storm-starter/target/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar storm.starter.WordCountTopology myFirstStormApp

 [查看Topology]

storm list

Topology_name        Status     Num_tasks  Num_workers  Uptime_secs
-------------------------------------------------------------------
myFirstStormApp      ACTIVE     28         3            26063

[提交同名的Topology]

Exception in thread "main" java.lang.RuntimeException: Topology with name `myFirstStormApp` already exists on cluster
    at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:89)
    at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:58)
    at storm.starter.WordCountTopology.main(WordCountTopology.java:77)

[杀死Topology]

Topology_name        Status     Num_tasks  Num_workers  Uptime_secs
-------------------------------------------------------------------
myFirstStormApp      KILLED     0          0            26126

过一段时间后

No topologies running.

(5)日志查看

tail -100f logs/nimbus.log

tail -100f logs/supervisor.log

tail -100f logs/worker-6700.log

tail -100f logs/worker-6701.log

tail -100f logs/worker-6702.log