生活随笔
收集整理的這篇文章主要介紹了
mac10.9下eclipse的storm开发环境搭建
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
--------------------------------------- 博文作者:迦壹 博客地址:http://idoall.org/home.php?mod=space&uid=1&do=blog&id=546 轉載聲明:可以轉載, 但必須以超鏈接形式標明文章原始出處和作者信息及版權聲明,謝謝合作! ---------------------------------------
? 目錄: 一、什么是STORM? 二、STORM的官方網站在哪里? 三、在哪里下載?需要哪些組件的支持? 四、如何安裝? 五、FAQ ? 一、什么是STORM? 在上一篇文章《mac10.9下eclipse的storm開發環境搭建》中,已經有詳細的介紹。
二、STORM的官方網站在哪里? http://storm.incubator.apache.org/
三、在哪里下載?需要哪些組件的支持? Storm0.9.2在下面的地址可以下載: http://www.apache.org/dyn/closer.cgi/incubator/storm/apache-storm-0.9.2-incubating/apache-storm-0.9.2-incubating.tar.gz 1)、根據官方文檔描述,主要依賴于JDK和Python。 Next you need to install Storm’s dependencies on Nimbus and the workermachines. These are:
Java 6 Python 2.6.6 2)、Zookeeper集群的依賴 在之前的文章《ubuntu12.04+hadoop2.2.0+zookeeper3.4.5+hbase0.96.2+hive0.13.1分布式環境部署》中有寫安裝過程,這里就不再描述。 四、如何安裝? 1)、解壓下載到的apache-storm-0.9.2-incubating.zip 2)、編輯storm.yaml配置文件 ?
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 ### ldir storm.local.dir: "/home/hadoop/storm-0.9.2-incubating/ldir" ### zookeeper storm.zookeeper.servers: ????- "m1" ????- "m2" ????- "s1" ????- "s2" ### nimbus host nimbus.host: "m1" ### ui.* configs are for the master ui.port: 8081 # 我這里修改了storm的ui端口 ########### These MUST be filled in for a storm configuration # storm.zookeeper.servers: #???? - "server1" #???? - "server2" # # nimbus.host: "nimbus" # # # ##### These may optionally be filled in: #??? ## List of custom serializations # topology.kryo.register: #???? - org.mycompany.MyType #???? - org.mycompany.MyType2: org.mycompany.MyType2Serializer # ## List of custom kryo decorators # topology.kryo.decorators: #???? - org.mycompany.MyDecorator # ## Locations of the drpc servers # drpc.servers: #???? - "server1" #???? - "server2" ## Metrics Consumers # topology.metrics.consumer.register: #?? - class: "backtype.storm.metric.LoggingMetricsConsumer" #???? parallelism.hint: 1 #?? - class: "org.mycompany.MyMetricsConsumer" #???? parallelism.hint: 1 #???? argument: #?????? - endpoint: "metrics-collector.mycompany.org"
3)、將m1下的storm的目錄復制到其它發服務器下去 ?
1 2 3 root@m1:/home/hadoop# scp -r /home/hadoop/storm-0.9.2-incubating root@m2:/home/hadoop root@m1:/home/hadoop# scp -r /home/hadoop/storm-0.9.2-incubating root@s1:/home/hadoop root@m1:/home/hadoop# scp -r /home/hadoop/storm-0.9.2-incubating root@s2:/home/hadoop
4)、在m1上啟動Nimbus ?
1 2 3 root@m1:/home/hadoop/storm-0.9.2-incubating# /home/hadoop/storm-0.9.2-incubating/bin/storm nimbus & [1] 1171 root@m1:/home/hadoop/storm-0.9.2-incubating# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/idoall_storm.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating/conf -Xmx1024m -Dlogfile.name=nimbus.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.nimbus
5)、在s1、s2上啟動Supervisor ?
1 2 3 4 5 6 7 8 9 root@s1:/home/hadoop# /home/hadoop/storm-0.9.2-incubating/bin/storm supervisor & [1] 31158 root@s1:/home/hadoop# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating/conf -Xmx256m -Dlogfile.name=supervisor.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.supervisor ------上面的代碼是s1上的---------------------下面的代碼是s2上的------------------- root@s2:/home/hadoop# /home/hadoop/storm-0.9.2-incubating/bin/storm supervisor & [1] 32676 root@s2:/home/hadoop# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating/conf -Xmx256m -Dlogfile.name=supervisor.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.supervisor
6)、在m1上啟動Storm UI ?
1 2 3 root@m1:/home/hadoop# /home/hadoop/storm-0.9.2-incubating/bin/storm ui & [1] 1221 root@m1:/home/hadoop# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/idoall_storm.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating:/home/hadoop/storm-0.9.2-incubating/conf -Xmx768m -Dlogfile.name=ui.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.ui.core
在瀏覽器打開:http://m1:8081/index.html,可以看到如下界面: 7)、Zookeeper下查看是否有對應storm的目錄,在下面的第29行,
可以看出zookeeper已經管理storm了。 ?
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 root@m1:/home/hadoop/storm-0.9.2-incubating# /home/hadoop/zookeeper-3.4.5/bin/zkCli.sh Connecting to localhost:2181 2014-08-02 12:54:04,172 [myid:] - INFO? [main:Environment@100] - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 2014-08-02 12:54:04,178 [myid:] - INFO? [main:Environment@100] - Client environment:host.name=m1 2014-08-02 12:54:04,179 [myid:] - INFO? [main:Environment@100] - Client environment:java.version=1.7.0_65 2014-08-02 12:54:04,179 [myid:] - INFO? [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2014-08-02 12:54:04,180 [myid:] - INFO? [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-7-oracle/jre 2014-08-02 12:54:04,180 [myid:] - INFO? [main:Environment@100] - Client environment:java.class.path=/home/hadoop/zookeeper-3.4.5/bin/../build/classes:/home/hadoop/zookeeper-3.4.5/bin/../build/lib/*.jar:/home/hadoop/zookeeper-3.4.5/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/hadoop/zookeeper-3.4.5/bin/../lib/slf4j-api-1.6.1.jar:/home/hadoop/zookeeper-3.4.5/bin/../lib/netty-3.2.2.Final.jar:/home/hadoop/zookeeper-3.4.5/bin/../lib/log4j-1.2.15.jar:/home/hadoop/zookeeper-3.4.5/bin/../lib/jline-0.9.94.jar:/home/hadoop/zookeeper-3.4.5/bin/../zookeeper-3.4.5.jar:/home/hadoop/zookeeper-3.4.5/bin/../src/java/lib/*.jar:/home/hadoop/zookeeper-3.4.5/bin/../conf:/usr/lib/jvm/java-7-oracle/lib 2014-08-02 12:54:04,181 [myid:] - INFO? [main:Environment@100] - Client environment:java.library.path=:/usr/local/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2014-08-02 12:54:04,181 [myid:] - INFO? [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2014-08-02 12:54:04,182 [myid:] - INFO? [main:Environment@100] - Client environment:java.compiler=<NA> 2014-08-02 12:54:04,182 [myid:] - INFO? [main:Environment@100] - Client environment:os.name=Linux 2014-08-02 12:54:04,183 [myid:] - INFO? [main:Environment@100] - Client environment:os.arch=amd64 2014-08-02 12:54:04,183 [myid:] - INFO? [main:Environment@100] - Client environment:os.version=3.11.0-15-generic 2014-08-02 12:54:04,184 [myid:] - INFO? [main:Environment@100] - Client environment:user.name=root 2014-08-02 12:54:04,184 [myid:] - INFO? [main:Environment@100] - Client environment:user.home=/root 2014-08-02 12:54:04,185 [myid:] - INFO? [main:Environment@100] - Client environment:user.dir=/home/hadoop/storm-0.9.2-incubating 2014-08-02 12:54:04,187 [myid:] - INFO? [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@19b1ebe5 2014-08-02 12:54:04,210 [myid:] - INFO? [main-SendThread(localhost:2181):ClientCnxn$SendThread@966] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) Welcome to ZooKeeper! 2014-08-02 12:54:04,233 [myid:] - INFO? [main-SendThread(localhost:2181):ClientCnxn$SendThread@849] - Socket connection established to localhost/127.0.0.1:2181, initiating session JLine support is enabled 2014-08-02 12:54:04,260 [myid:] - INFO? [main-SendThread(localhost:2181):ClientCnxn$SendThread@1207] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x14794a964e20006, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0] ls / [storm, hadoop-ha, hbase, zookeeper] [zk: localhost:2181(CONNECTED) 1] quit Quitting... 2014-08-02 12:54:11,130 [myid:] - INFO? [main-EventThread:ClientCnxn$EventThread@509] - EventThread shut down 2014-08-02 12:54:11,131 [myid:] - INFO? [main:ZooKeeper@684] - Session: 0x14794a964e20006 closed root@m1:/home/hadoop/storm-0.9.2-incubating#
8)、storm的測試,將《mac10.9下eclipse的storm開發環境搭建》一文章中的Eclipse項目,打包導出為idoall_storm.jar,并上傳到storm的根目錄 ?
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 root@m1:/home/hadoop/storm-0.9.2-incubating# ll 總用量 140 drwxr-xr-x 11 root?? root??? 4096 Aug? 2 12:51 ./ drwxr-xr-x 39 hadoop hadoop? 4096 Aug? 1 14:38 ../ drwxr-xr-x? 2 root?? root??? 4096 Aug? 1 14:38 bin/ -rw-r--r--? 1??? 502 staff? 34239 Jun 13 08:46 CHANGELOG.md drwxr-xr-x? 2 root?? root??? 4096 Aug? 2 12:31 conf/ -rw-r--r--? 1??? 502 staff??? 538 Mar 13 11:17 DISCLAIMER drwxr-xr-x? 3??? 502 staff?? 4096 May? 6 03:13 examples/ drwxr-xr-x? 3 root?? root??? 4096 Aug? 1 14:38 external/ -rw-r--r--? 1 root?? root??? 8984 Aug? 2 11:25 idoall_storm.jar drwxr-xr-x? 3 root?? root??? 4096 Aug? 2 12:51 ldir/ drwxr-xr-x? 2 root?? root??? 4096 Aug? 1 14:38 lib/ -rw-r--r--? 1??? 502 staff? 22822 Jun 12 04:07 LICENSE drwxr-xr-x? 2 root?? root??? 4096 Aug? 1 14:38 logback/ drwxr-xr-x? 2 root?? root??? 4096 Aug? 1 15:07 logs/ -rw-r--r--? 1??? 502 staff??? 981 Jun 11 01:10 NOTICE drwxr-xr-x? 5 root?? root??? 4096 Aug? 1 14:38 public/ -rw-r--r--? 1??? 502 staff?? 7445 Jun 10 02:24 README.markdown -rw-r--r--? 1??? 502 staff???? 17 Jun 17 00:22 RELEASE -rw-r--r--? 1??? 502 staff?? 3581 May 30 00:20 SECURITY.md root@m1:/home/hadoop/storm-0.9.2-incubating# /home/hadoop/storm-0.9.2-incubating/bin/storm jar idoall_storm.jar storm.starter.WordCountTopology word_count Running: /usr/lib/jvm/java-7-oracle/bin/java -client -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/idoall_storm.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:idoall_storm.jar:/home/hadoop/storm-0.9.2-incubating/conf:/home/hadoop/storm-0.9.2-incubating/bin -Dstorm.jar=idoall_storm.jar storm.starter.WordCountTopology word_count 500? [main] INFO? backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar... 516? [main] INFO? backtype.storm.StormSubmitter - Uploading topology jar idoall_storm.jar to assigned location: /home/hadoop/storm-0.9.2-incubating/ldir/nimbus/inbox/stormjar-8a07e8d6-ac9d-4347-ac3a-321906349567.jar 533? [main] INFO? backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: /home/hadoop/storm-0.9.2-incubating/ldir/nimbus/inbox/stormjar-8a07e8d6-ac9d-4347-ac3a-321906349567.jar 534? [main] INFO? backtype.storm.StormSubmitter - Submitting topology word_count in distributed mode with conf {"topology.workers":3,"topology.debug":true} 1057 [main] INFO? backtype.storm.StormSubmitter - Finished submitting topology: word_count root@m1:/home/hadoop/storm-0.9.2-incubating#
這時再打開瀏覽器http://m1:8081/index.html,可以看到我們剛才提產的代碼已經在集群中顯示了。 五、FAQ 1)過程中發現storm ui顯示supervisor個數與實際不符的解決 解決方法:在刪除了local dir,重啟后恢復了。 參考資料:storm ui顯示supervisor個數與實際不符的解決 ---------------------------------------
強烈譴責不 尊重版權,盜用本文,不寫原文出處的網站: http://www.aboutyun.com/thread-8913-1-1.html
---------------------------------
轉載于:https://www.cnblogs.com/lion.net/p/3892843.html
總結
以上是生活随笔 為你收集整理的mac10.9下eclipse的storm开发环境搭建 的全部內容,希望文章能夠幫你解決所遇到的問題。
如果覺得生活随笔 網站內容還不錯,歡迎將生活随笔 推薦給好友。