Spring-kafka高容量加工(Spring-kafka high volume processing)

 使用spring-kafka 1.0.5，我正在使用10个分区的并发度为10的繁忙主题。 
 
 我当前的代码根据分区ID向队列添加消息，这些消息都保存在HashMap中。 
 
@KafkaListener(topics = "${kafka.topic}")
public void onMessage(ConsumerRecord consumerRecord, Acknowledgment acknowledgment) {
    //Pseudo code
    add to Hashmap<Integer, Queue<ConsumerRecord>> based on partition.
}
 
 不幸的是，这种设计需要花费两倍的处理时间来进行简单的消费。 
 
 我的要求是单独处理分区，但如何避免使用基于@KafkaListener的对分区的引用的hashmap。 
 
 有没有更有效的方法来解决这个问题？ 理想情况下，来自侦听器注释的每个线程都将管理自己的列表。 有没有办法在没有交叉引用的情况下执行此操作，例如基于分区ID的上述hashmap？ 

Using spring-kafka 1.0.5, I am consuming from a busy topic with 10 partitions with a concurrency of 10. 
 
My current code adds a message to a queue based on the partition ID which are both persisted in a HashMap. 
 
@KafkaListener(topics = "${kafka.topic}")
public void onMessage(ConsumerRecord consumerRecord, Acknowledgment acknowledgment) {
    //Pseudo code
    add to Hashmap<Integer, Queue<ConsumerRecord>> based on partition.
}
 
Unfortunately, that design is taking twice the processing time a simple consumption would take. 
 
My requirement is to process partitions separately but how can avoid having a hashmap with a reference to a partition based on the @KafkaListener. 
 
Is there a more efficient way of going about this? Ideally, each thread from the listener annotation would manage its own list. Is there a way to do that without having a cross reference such as the hashmap mentioned above based on the partition ID?

原文：https://stackoverflow.com/questions/41128122