Java Executor:小任务还是大任务?(Java Executor: Small tasks or big ones?)
考虑一个可以分解成数百个小的,可独立运行的任务的大任务。 更具体地说,每个小任务是发送轻型网络请求并决定从服务器接收的答案。 这些小任务预计不会花费超过一秒的时间,并且总共需要几台服务器。
我想到了使用Executor框架实现这一点的两种方法,我想知道哪一个更好,为什么。
- 创建一些,比如5到10个任务,每个任务涉及做一堆发送和接收。
- 为每个发送和接收创建单个任务(可调用或可运行),并安排执行者运行所有这些任务(数百个)。
我很抱歉,如果我的问题表明我懒得测试这些,并亲自看看哪些更好(至少在性能方面)。 在回答这个具体案例时,我的问题有一个更为笼统的方面。 在这种情况下,当你想使用执行程序来完成所有的调度和其他工作时,创建大量小任务或将它们分组到更少数量的更大任务中会更好吗?
Consider one big task which could be broken into hundreds of small, independently-runnable tasks. To be more specific, each small task is to send a light network request and decide upon the answer received from the server. These small tasks are not expected to take longer than a second, and involve a few servers in total.
I have in mind two approaches to implement this using the Executor framework, and I want to know which one's better and why.
- Create a few, say 5 to 10 tasks each involving doing a bunch of send and receives.
- Create a single task (Callable or Runnable) for each send & receive and schedule all of them (hundreds) to be run by the executor.
I'm sorry if my question shows that I'm lazy to test these and see for myself what's better (at least performance-wise). My question, while looking after an answer to this specific case, has a more general aspect. In situations like these when you want to use an executor to do all the scheduling and other stuff, is it better to create lots of small tasks or to group those into a less number of bigger tasks?
原文:https://stackoverflow.com/questions/12717251
最满意答案
同步调度会暂停代码的执行,直到调度块完成为止。 异步调度立即返回,该块与调用代码异步执行:
dispatch_sync(somewhere, ^{ something }); // Reached later, when the block is finished. dispatch_async(somewhere, ^{ something }); // Reached immediately. The block might be waiting // to be executed, executing or already finished.
并且有两种调度队列,串行和并发。 串行的按照添加顺序严格逐个发送块。 当一个完成时,另一个开始。 这种执行只需要一个线程。 并发队列并行地同时调度块。 那里使用了更多的线程。
如果您认为合适,您可以混合使用同步/异步调度和串行/并发队列。 如果您想要使用GCD来保护对关键部分的访问,请使用单个串行队列并分配此队列上共享数据的所有操作(同步或异步,无关紧要)。 这样,共享数据总是只有一个块操作:
- (void) addFoo: (id) foo { dispatch_sync(guardingQueue, ^{ [sharedFooArray addObject:foo]; }); } - (void) removeFoo: (id) foo { dispatch_sync(guardingQueue, ^{ [sharedFooArray removeObject:foo]; }); }
现在,如果
guardingQueue
是一个串行队列,即使从不同线程同时调用addFoo:
和removeFoo:
方法,add / remove操作也不会发生冲突。Synchronous dispatch suspends the execution of your code until the dispatched block has finished. Asynchronous dispatch returns immediately, the block is executed asynchronously with regard to the calling code:
dispatch_sync(somewhere, ^{ something }); // Reached later, when the block is finished. dispatch_async(somewhere, ^{ something }); // Reached immediately. The block might be waiting // to be executed, executing or already finished.
And there are two kinds of dispatch queues, serial and concurrent. The serial ones dispatch the blocks strictly one by one in the order they are being added. When one finishes, another one starts. There is only one thread needed for this kind of execution. The concurrent queues dispatch the blocks concurrently, in parallel. There are more threads being used there.
You can mix and match sync/async dispatch and serial/concurrent queues as you see fit. If you want to use GCD to guard access to a critical section, use a single serial queue and dispatch all operations on the shared data on this queue (synchronously or asynchronously, does not matter). That way there will always be just one block operating with the shared data:
- (void) addFoo: (id) foo { dispatch_sync(guardingQueue, ^{ [sharedFooArray addObject:foo]; }); } - (void) removeFoo: (id) foo { dispatch_sync(guardingQueue, ^{ [sharedFooArray removeObject:foo]; }); }
Now if
guardingQueue
is a serial queue, the add/remove operations can never clash even if theaddFoo:
andremoveFoo:
methods are called concurrently from different threads.
相关问答
更多-
TCP/IP模型是一个________。[2023-05-19]
a -
下列中不属于面向对象的编程语言的是?[2022-05-30]
a -
定时锁定GCD?(Timed locks in GCD?)[2022-12-10]
您可以在dispatch_semaphore上使用超时的下一个作业块。 你需要等待提交作业,不仅仅是作业执行......否则GCD会激活大量的线程,这些线程会立即等待,这不是你想要的。 You could have the next job block on a dispatch_semaphore with a timeout. You'd need to wait on job submission, though, not just job execution... otherwise GCD wil ... -
使用dispatch_queue_set_specific()分配所需的任何标识符。 然后,您可以使用dispatch_get_specific()检查您的标识符。 请记住, dispatch_get_specific()很好,因为它将从当前队列开始,然后在当前队列中没有设置密钥的情况下走上目标队列。 这通常没有关系,但在某些情况下可能会有用。 Assign whatever identifier you want using dispatch_queue_set_specific(). You can t ...
-
iOS GCD与异步块同步(iOS GCD Sync with Async Block)[2021-03-21]
如果要在完成一系列异步任务时启动某个进程,但希望允许这些任务相对于彼此同时运行,则可以使用分派组(尤其是对于网络请求,可以提供更好的性能比顺序运行它们): dispatch_group_t group = dispatch_group_create(); for (int i = 0; i < array.count; i++) { dispatch_group_enter(group); [self performAsyncTaskCompletion: ^{ //Do ... -
异步回调是指当前线程继续执行语句,并将代码的执行分离到另一个线程以便稍后运行的回调。 有几种技术可以实现这一目标。 在这个例子中,我用4种不同的异步方式调用方法cacheImage:,带参数image(只是一个例子)。 // 1. NSThread [NSThread detachNewThreadSelector:@selector(cacheImage:) toTarget:self withObject:image]; // 2. performSelector... [self performSe ...
-
没有公共API来“跳转”队列,也没有从队列中删除或重新排序先前排队但尚未执行的块。 如果您只是希望块以比其他块更高的优先级执行,则可以将其提交到高优先级全局并发队列,但是通过使用特定队列提供的任何并发保证自然会因此而丢失。 除此之外,您必须自己实现此功能。 一种方法可能是在dispatch_async周围拥有自己的包装器,它将每个块包装在另一个块中,这将使“队列跳转”块在每个非队列跳转块之前有机会执行。 可以想象为“跳线”制作第二个串行队列,这些包装块可以相应地暂停和恢复。 但一般来说,一旦提交,就无法重新 ...
-
同步调度会暂停代码的执行,直到调度块完成为止。 异步调度立即返回,该块与调用代码异步执行: dispatch_sync(somewhere, ^{ something }); // Reached later, when the block is finished. dispatch_async(somewhere, ^{ something }); // Reached immediately. The block might be waiting // to be executed, executin ...
-
CoreData与GCD队列中的AFNetworking请求不兼容?(CoreData not compatible with an AFNetworking request from a GCD queue?)[2023-05-21]
嗯..处理MOC和线程的推荐方法是始终创建一个新的MOC,它是主线程MOC的子moc。 让主线程执行所有保存,但是您的GCD线程基本上可以将更改合并到主MOC。 我使用https://github.com/magicalpanda/MagicalRecord/取得了相当不错的成功,以更简单的方式实现这一目标。 Hmm .. the recommended way to deal with MOC and threads is to always make a new MOC that is a sub-mo ... -
解决方案草图: q = /* custom serial queue */ urls = /* urls array */; NSMutableArray *images = [NSMutableArray new]; for (NSURL *url in URLs) { NSURLRequest *req = [self imageRequestForURL:url]; dispatch_async(q, ^{ UIImage *image = [self imageFromRequ ...