如何在Scrapy / Twisted中使用线程，即如何在响应回调中进行阻止代码的异步调用？(How to use threading in Scrapy/Twisted, i.e. how to do async calls to blocking code in response callbacks?)

 我需要在Scrapy中运行一些多线程\多处理工作（因为我有一些使用阻塞调用的库），并在完成之后将请求发送回Scrapy引擎。 
 
 我需要这样的东西： 
 
def blocking_call(self, html):
    # ....
    # do some work in blocking call
    return Request(url)

def parse(self, response):
    return self.blocking_call(response.body)
 
 我怎么能这样做？ 我想我应该使用Twisted reactor和Deferred对象。 但是Scrapy parse回调必须只返回None或Request或BaseItem对象。 

I need run some multi-thread\multiprocessing work (because I have some library which uses blocking call) in Scrapy, and after its completion put back Request to Scrapy engine.
 
I need something like this:
 
def blocking_call(self, html):
    # ....
    # do some work in blocking call
    return Request(url)

def parse(self, response):
    return self.blocking_call(response.body)
 
How I can do that? I think I should to use Twisted reactor and Deferred object. But Scrapy parse callback must return only None or Request or BaseItem object.

原文：https://stackoverflow.com/questions/25842469