如何使用cURL无延迟地打开多个URL(How to open multiple URLs with cURL without delay)

 很高兴听到你现在获得更好的表现！ 
 
 您的作业使用在托管代码中执行的表达式运行超过2800个非常小的文件，而不是转换为C ++，因为U-SQL中的一些更常见的C＃表达式是。 
 
 这导致以下问题： 
 
 
 
 您可以使用一定数量的AU开始工作。 然后每个AU启动一个YARN容器来执行部分工作。 这意味着容器需要干净地初始化，这需要一些时间（您可以在顶点执行视图中将其视为创建时间）。 现在这需要一些时间，如果您的顶点执行大量处理，那就不会有太大的开销。 不幸的是，在你的情况下，在一个小文件上处理非常快，因此开销很大。 
 
 
 如果顶点只执行我们将代码生成到C ++代码中的系统生成的代码，那么我们可以在没有重新初始化时间的情况下重用容器。 遗憾的是，由于潜在的工件被遗忘，我们无法重用使用托管运行时执行的一般用户代码。 因此，在这种情况下，我们需要重新初始化容器，这需要花费时间（超过2800次）。 
 
 
 现在根据您的反馈，我们正在改进我们的重新初始化逻辑（如果您不使用内联C＃表达式做任何事情，我们仍然可以重新初始化）。 此外，一旦我们可以在单个顶点内处理多个小文件而不是每个顶点一个文件，它会变得更好。 
 
 您的解决方法是增加文件的大小，并可能避免自定义代码（当然不总是可能）在太多顶点中。 

Great to hear you are getting the better performance now!
 
Your job runs on over 2800 very small files using an expression that is being executed in managed code and not translated into C++ as some of the more common C# expressions in U-SQL are. 
 
This leads to the following problem:
 
 
 
You start your job with a certain number of AUs. Each AU then starts a YARN container to execute part of your job. This means that the container needs to be initialized cleanly which takes some time (you can see it in the Vertex Execution View as creation time). Now this takes a bit of time, that is not that much overhead if your vertex does some large amount of processing. Unfortunately in your case, the processing is very quick on a small file so there is a large overhead.
 
 
If the vertex only executes system generated code that we codegen into C++ code, then we can reuse containers without the re-initialization time. Unfortunately, we cannot reuse general user-code that gets executed with the managed runtime due to potential artifacts being left behind. So in that case we need to re-initialize the containers which will take time (over 2800 times).
 
 
Now based on your feedback, we are improving our reinitialization logic (we can still reinitialize if you do not do anything fancy with inline C# expressions). Also, it will get better once we can process several small files inside a single vertex instead of one file per vertex. 
 
Workarounds for you is to increase the sizes of your files and to possibly avoid custom code (not always possible of course) to be in too many of the vertices.