首页 \ 问答 \ 部署Storm构建JAR(Deploying Storm build JAR)

部署Storm构建JAR(Deploying Storm build JAR)

     
 
  1.  我开发了一个Java类，它从Kafka队列中读取数据并将其打印出来 
    ZkHosts zkHosts=new ZkHosts("localhost:2181");
String topic_name="test";
String consumer_group_id="storm";
String zookeeper_root="";
SpoutConfig kafkaConfig=new SpoutConfig(zkHosts, 
        topic_name, zookeeper_root, consumer_group_id);
kafkaConfig.scheme=new SchemeAsMultiScheme(new StringScheme());
/*kafkaConfig.forceFromStart=false;
kafkaConfig.startOffsetTime =-2;*/

KafkaSpout kafkaSpout = new KafkaSpout(kafkaConfig);
TopologyBuilder builder=new TopologyBuilder();
//builder.setSpout("KafkaSpout", kafkaSpout, 1);
builder.setSpout("KafkaSpout", kafkaSpout);
builder.setBolt("PrinterBolt", new PrinterBolt()).globalGrouping("KafkaSpout");
Map<String, Object> conf = new HashMap<String, Object>();
conf.put(Config.TRANSACTIONAL_ZOOKEEPER_PORT, 2181);
conf.put(Config.TRANSACTIONAL_ZOOKEEPER_SERVERS, Arrays.asList("localhost"));
conf.put(Config.STORM_ZOOKEEPER_SESSION_TIMEOUT, 20000);
conf.put(Config.STORM_ZOOKEEPER_CONNECTION_TIMEOUT, 20000);
conf.put(Config.STORM_ZOOKEEPER_RETRY_TIMES, 3);
conf.put(Config.STORM_ZOOKEEPER_RETRY_INTERVAL, 30);
LocalCluster cluster=new LocalCluster();
try{
    cluster.submitTopology("KafkaConsumerTopology", conf, builder.createTopology());
    Thread.sleep(120000);
}catch (Exception e) {
    //throw new IllegalStateException("Couldn't initialize the topology", e);
    System.out.println(e.getMessage());
}

    2.  
 
  2.  在编码之后，我正在将Maven构建为JAR文件并将jar移动到Amazon AWS集群 
    3.  
 
  3.  然后像nohup java -cp uber-***-0.0.1-SNAPSHOT.jar com.***.&&&.kafka.App一样运行命令nohup java -cp uber-***-0.0.1-SNAPSHOT.jar com.***.&&&.kafka.App 
    4.  

 

 但是我在这里面临一个错误，有人能告诉我在部署中我做了什么错误吗？ 我在想我必须这样做： 
 

     
 
  •  我需要在strom配置文件夹中部署这个jar文件，我需要吗？ 但我确实将jar放在AWS的单独文件夹中（不在storm文件夹中） 
    •  
 
  •  如何看待系统输出 
    •  
 
  •  我是否需要在项目中包含任何yml文件？ 
    •  

 

 请查看以下异常： 
 

29537 [Thread-14-KafkaSpout] ERROR backtype.storm.util - Async loop died!
java.lang.ExceptionInInitializerError: null
    at org.apache.log4j.Logger.getLogger(Logger.java:39) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.utils.Logging$class.logger(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.network.BlockingChannel.logger$lzycompute(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.network.BlockingChannel.logger(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.utils.Logging$class.debug(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.network.BlockingChannel.debug(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.network.BlockingChannel.connect(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.consumer.SimpleConsumer.connect(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.consumer.SimpleConsumer.getOrMakeConnection(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.consumer.SimpleConsumer.getOffsetsBefore(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:77) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:67) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at storm.kafka.PartitionManager.<init>(PartitionManager.java:83) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:98) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:135) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at backtype.storm.daemon.executor$fn__3373$fn__3388$fn__3417.invoke(executor.clj:565) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at clojure.lang.AFn.run(AFn.java:24) [uber-iot-0.0.1-SNAPSHOT.jar:na]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
Caused by: java.lang.IllegalStateException: Detected both log4j-over-slf4j.jar AND slf4j-log4j12.jar on the class path, preempting StackOverflowError. See also http://www.slf4j.org/codes.html#log4jDelegationLoop for more details.
    at org.apache.log4j.Log4jLoggerFactory.<clinit>(Log4jLoggerFactory.java:49) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    ... 22 common frames omitted


     
 
  1. I developed a Java class which reads data from Kafka queue and prints it out
     
    ZkHosts zkHosts=new ZkHosts("localhost:2181");
String topic_name="test";
String consumer_group_id="storm";
String zookeeper_root="";
SpoutConfig kafkaConfig=new SpoutConfig(zkHosts, 
        topic_name, zookeeper_root, consumer_group_id);
kafkaConfig.scheme=new SchemeAsMultiScheme(new StringScheme());
/*kafkaConfig.forceFromStart=false;
kafkaConfig.startOffsetTime =-2;*/

KafkaSpout kafkaSpout = new KafkaSpout(kafkaConfig);
TopologyBuilder builder=new TopologyBuilder();
//builder.setSpout("KafkaSpout", kafkaSpout, 1);
builder.setSpout("KafkaSpout", kafkaSpout);
builder.setBolt("PrinterBolt", new PrinterBolt()).globalGrouping("KafkaSpout");
Map<String, Object> conf = new HashMap<String, Object>();
conf.put(Config.TRANSACTIONAL_ZOOKEEPER_PORT, 2181);
conf.put(Config.TRANSACTIONAL_ZOOKEEPER_SERVERS, Arrays.asList("localhost"));
conf.put(Config.STORM_ZOOKEEPER_SESSION_TIMEOUT, 20000);
conf.put(Config.STORM_ZOOKEEPER_CONNECTION_TIMEOUT, 20000);
conf.put(Config.STORM_ZOOKEEPER_RETRY_TIMES, 3);
conf.put(Config.STORM_ZOOKEEPER_RETRY_INTERVAL, 30);
LocalCluster cluster=new LocalCluster();
try{
    cluster.submitTopology("KafkaConsumerTopology", conf, builder.createTopology());
    Thread.sleep(120000);
}catch (Exception e) {
    //throw new IllegalStateException("Couldn't initialize the topology", e);
    System.out.println(e.getMessage());
}

    2.  
 
  2. after coding, I am doing Maven build as JAR file and moved the jar to Amazon AWS cluster
    3.  
 
  3. and then running command like nohup java -cp uber-***-0.0.1-SNAPSHOT.jar com.***.&&&.kafka.App
    4.  

 

But I am facing an error here, could anyone tell me what mistake I am doing in deployment?. I am thinking following I have to do:
 

     
 
  • I need to deploy this jar file in strom config folder, do I need to? but I did placed the jar in seperate folder in AWS (not in storm folder)
    •  
 
  • how to see the sys outs
    •  
 
  • do I need to include any yml files in my project?
    •  

 

Please find below exception:
 

29537 [Thread-14-KafkaSpout] ERROR backtype.storm.util - Async loop died!
java.lang.ExceptionInInitializerError: null
    at org.apache.log4j.Logger.getLogger(Logger.java:39) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.utils.Logging$class.logger(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.network.BlockingChannel.logger$lzycompute(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.network.BlockingChannel.logger(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.utils.Logging$class.debug(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.network.BlockingChannel.debug(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.network.BlockingChannel.connect(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.consumer.SimpleConsumer.connect(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.consumer.SimpleConsumer.getOrMakeConnection(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.consumer.SimpleConsumer.getOffsetsBefore(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:77) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:67) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at storm.kafka.PartitionManager.<init>(PartitionManager.java:83) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:98) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:135) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at backtype.storm.daemon.executor$fn__3373$fn__3388$fn__3417.invoke(executor.clj:565) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    at clojure.lang.AFn.run(AFn.java:24) [uber-iot-0.0.1-SNAPSHOT.jar:na]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
Caused by: java.lang.IllegalStateException: Detected both log4j-over-slf4j.jar AND slf4j-log4j12.jar on the class path, preempting StackOverflowError. See also http://www.slf4j.org/codes.html#log4jDelegationLoop for more details.
    at org.apache.log4j.Log4jLoggerFactory.<clinit>(Log4jLoggerFactory.java:49) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
    ... 22 common frames omitted


原文：https://stackoverflow.com/questions/34368293
更新时间：2023-11-14 09:11

最满意答案

 如果您的模板无效，我建议您在views.py中使用它，例如： 
 

from django.shortcuts import render, render_to_response

def homepage(request):
    template_name = 'homepage.html'
    extended_template = 'base_login.html'

    if request.user.is_authenticated():
        extended_template = 'base.html'

    return render(
        request, template_name, 
        {'extended_template': extended_template, ...}
    )

# homepage.html
{% extends extended_template %}

{% block content %}
  {% if request.user.is_authenticated %}
    Hello {{ request.user }}
  {% endif %}
{% endif %}

 

 
 
 注意：如果render功能仍然不能正常工作，请尝试使用render_to_response例如这个答案： https ： //stackoverflow.com/a/1331183/6396981 
 


If your template goes invalid, I suggest you to it at the views.py, an example:
 

from django.shortcuts import render, render_to_response

def homepage(request):
    template_name = 'homepage.html'
    extended_template = 'base_login.html'

    if request.user.is_authenticated():
        extended_template = 'base.html'

    return render(
        request, template_name, 
        {'extended_template': extended_template, ...}
    )

# homepage.html
{% extends extended_template %}

{% block content %}
  {% if request.user.is_authenticated %}
    Hello {{ request.user }}
  {% endif %}
{% endif %}

 

 
 
Note: if function of render still doesn't work well, please try with render_to_response such as this answer: https://stackoverflow.com/a/1331183/6396981

相关问答

更多

相关文章

更多
  • redhat6.4上build storm 0.9.0.1
  • zoj 2966 Build The Electric System
  • storm
  • (二) storm的基本使用
  • Twitter Storm 使用maven构建storm项目
  • 重新Build Hadoop后进入Hive客户端异常
  • 配置 生成hadoop-eclipse-plugin jar包
  • 关于开源jar包
  • Spring Boot CLI打包jar文件并运行
  • Storm资料汇总

    • 最新问答

    更多
  • 获取MVC 4使用的DisplayMode后缀(Get the DisplayMode Suffix being used by MVC 4)
  • 如何通过引用返回对象？(How is returning an object by reference possible?)
  • 矩阵如何存储在内存中？(How are matrices stored in memory?)
  • 每个请求的Java新会话？(Java New Session For Each Request?)
  • css：浮动div中重叠的标题h1(css: overlapping headlines h1 in floated divs)
  • 无论图像如何，Caffe预测同一类(Caffe predicts same class regardless of image)
  • xcode语法颜色编码解释？(xcode syntax color coding explained?)
  • 在Access 2010 Runtime中使用Office 2000校对工具(Use Office 2000 proofing tools in Access 2010 Runtime)
  • 从单独的Web主机将图像传输到服务器上(Getting images onto server from separate web host)
  • 从旧版本复制文件并保留它们（旧/新版本）(Copy a file from old revision and keep both of them (old / new revision))
  • 西安哪有PLC可控制编程的培训
  • 在Entity Framework中选择基类(Select base class in Entity Framework)
  • 在Android中出现错误“数据集和渲染器应该不为null，并且应该具有相同数量的系列”(Error “Dataset and renderer should be not null and should have the same number of series” in Android)
  • 电脑二级VF有什么用
  • Datamapper Ruby如何添加Hook方法(Datamapper Ruby How to add Hook Method)
  • 金华英语角.
  • 手机软件如何制作
  • 用于Android webview中图像保存的上下文菜单(Context Menu for Image Saving in an Android webview)
  • 注意：未定义的偏移量：PHP(Notice: Undefined offset: PHP)
  • 如何读R中的大数据集[复制](How to read large dataset in R [duplicate])
  • Unity 5 Heighmap与地形宽度/地形长度的分辨率关系？(Unity 5 Heighmap Resolution relationship to terrain width / terrain length?)
  • 如何通知PipedOutputStream线程写入最后一个字节的PipedInputStream线程？(How to notify PipedInputStream thread that PipedOutputStream thread has written last byte?)
  • python的访问器方法有哪些
  • DeviceNetworkInformation：哪个是哪个？(DeviceNetworkInformation: Which is which?)
  • 在Ruby中对组合进行排序(Sorting a combination in Ruby)
  • 网站开发的流程？
  • 使用Zend Framework 2中的JOIN sql检索数据(Retrieve data using JOIN sql in Zend Framework 2)
  • 条带格式类型格式模式编号无法正常工作(Stripes format type format pattern number not working properly)
  • 透明度错误IE11(Transparency bug IE11)
  • linux的基本操作命令。。。

    • Copyright ©2023 peixunduo.com All Rights Reserved.粤ICP备14003112号

    本站部分内容来源于互联网，仅供学习和参考使用，请莫用于商业用途。如有侵犯你的版权，请联系我们(neng862121861#163.com)，本站将尽快处理。谢谢合作！