寻求关于共享Netty老板/工人池的建议(Seeking advice on sharing Netty boss/worker pools)
上下文:Netty 3.6.3.Final,Java 1.7,Scala 2.9.x.
为了最小化(可能是空闲的)线程的数量,我想与不同的NIO套接字通道工厂(TCP)和一个NioDatagramChannelFactory共享NIO客户端/服务器和工作池。 我使用至少两个(或三个与Finagle堆栈)的服务器/客户端引导集,每个都有自己的NIO套接字通道工厂。 为每个boss和worker池使用一个新的缓存线程池会导致大量时间没有使用的线程负载。 粗略的目标是将所有bootstraps / channel工厂的工作者数量限制为2 * CPU核心数和老板数到CPU核心数。
我正在尝试切换到NioServer / ClientBossPool和NioWorkerPool,用于我自己的一组bootstrap。 但是根据底层ThreadPoolExecutor的配置,关闭引导程序会导致主线程在AbstractNioSelector关闭锁存器上永远等待。
class NioClientBossPoolTest { @Test def shutdown() { val corePoolSize = 1 val maxPoolSize = Integer.MAX_VALUE val keepAliveSeconds = 60 val keepAliveUnit = TimeUnit.SECONDS val blocking = true val queue: BlockingQueue[Runnable] = if(blocking) new LinkedBlockingQueue[Runnable](Integer.MAX_VALUE) else new SynchronousQueue[Runnable]() val executor = new ThreadPoolExecutor(corePoolSize, maxPoolSize, keepAliveSeconds, keepAliveUnit, queue) val clientBossPool = new NioClientBossPool(executor, 1) // B new NioServerBossPool(executor, 1) // C val workerPool = new NioWorkerPool(executor, 1) // A val channelFactory = new NioClientSocketChannelFactory(clientBossPool, workerPool) val bootstrap = new ClientBootstrap(channelFactory) // hangs waiting for shutdown latch in AbstractNioSelector (NioWorker or NioClientBoss // depending on the order of statement A, B, C) for // LinkedBlockingQueue, corePoolSize = 1 and sequence of statements A, B and C other than [B, A, C] // LinkedBlockingQueue, corePoolSize = 2 and sequence of statements A, B and C other than // [A, B, C], [B, C, A] and [C, B, A] bootstrap.shutdown() } }
我很确定执行程序服务配置必须满足一些特定要求,但哪些(核心池大小,队列类型)? bootstrap.shutdown()将永远阻塞,除非语句A,B和C的执行顺序正好是[B,A,C]。 将核心池大小增加到2个块,用于三个语句的六个组合中的三个。 核心池大小> 2或具有SynchronousQueue时,每个组合终止。
Context: Netty 3.6.3.Final, Java 1.7, Scala 2.9.x
In order to minimize the number (of possibly idling) threads, I'd like to share NIO client/server and worker pools with different NIO socket channel factories (TCP) and one NioDatagramChannelFactory. I'm using at least two (or three with the Finagle stack) sets of server/client bootstraps, each with their own NIO socket channel factories. Using a new cached thread pool for each and every boss and worker pool result in a load of threads which are not used most of the time. The rough goal would be to limit the number of workers over all bootstraps/channel factories to 2 * CPU core count and number of bosses to CPU core count.
I'm trying to switch over to NioServer/ClientBossPool and NioWorkerPool for one of my own set of bootstraps. But depending of the configuration of the underlying ThreadPoolExecutor, shutting down a bootstrap causes the main thread to wait forever on a AbstractNioSelector shutdown latch.
class NioClientBossPoolTest { @Test def shutdown() { val corePoolSize = 1 val maxPoolSize = Integer.MAX_VALUE val keepAliveSeconds = 60 val keepAliveUnit = TimeUnit.SECONDS val blocking = true val queue: BlockingQueue[Runnable] = if(blocking) new LinkedBlockingQueue[Runnable](Integer.MAX_VALUE) else new SynchronousQueue[Runnable]() val executor = new ThreadPoolExecutor(corePoolSize, maxPoolSize, keepAliveSeconds, keepAliveUnit, queue) val clientBossPool = new NioClientBossPool(executor, 1) // B new NioServerBossPool(executor, 1) // C val workerPool = new NioWorkerPool(executor, 1) // A val channelFactory = new NioClientSocketChannelFactory(clientBossPool, workerPool) val bootstrap = new ClientBootstrap(channelFactory) // hangs waiting for shutdown latch in AbstractNioSelector (NioWorker or NioClientBoss // depending on the order of statement A, B, C) for // LinkedBlockingQueue, corePoolSize = 1 and sequence of statements A, B and C other than [B, A, C] // LinkedBlockingQueue, corePoolSize = 2 and sequence of statements A, B and C other than // [A, B, C], [B, C, A] and [C, B, A] bootstrap.shutdown() } }
I'm pretty sure that the executor service configuration has to meet some specific requirements, but which (core pool size, queue type)? bootstrap.shutdown() will block forever, except when the execution order of statement A, B and C is exactly [B, A, C]. Increasing core pool size to 2 blocks for three out of six combinations of the three statements. With a core pool size > 2 or with a SynchronousQueue each combination terminates.
原文:https://stackoverflow.com/questions/15587331
最满意答案
在2个字段的水果和颜色上创建复合uniqe索引。
ALTER TABLE fruit_table ADD UNIQUE KEY (`fruit` , `colour`) ;
Create a composite uniqe index over the 2 fields fruit and colour.
ALTER TABLE fruit_table ADD UNIQUE KEY (`fruit` , `colour`) ;
相关问答
更多-
Mysql LOAD DATA INFILE - 输入数据意外截断(Mysql LOAD DATA INFILE - input data unexpectedly truncated)[2023-05-13]
如果您使用的是Windows,只需为Thilo的答案添加一些改进。 LOAD DATA INFILE '/path/to/data.csv' IGNORE INTO TABLE `condensed` FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\'' LINES TERMINATED BY '\r\n' --Windows right line terminator (id, src, hash, ihash, last_updated) 它对我 ... -
我很确定这是验证,而不是verification.MYD或其他两个。 .MYD是数据,.MYI是索引,.frm是架构。 琴弦有多长? 是十六进制? 如果是32位十六进制数字,那么你不希望BINARY(16)输出UNHEX吗? 该过程的很长一部分可能是ENABLE KEYS ,何时它将构建索引。 做SHOW PROCESSLIST; 当它正在运行时 - 如果它说“使用密钥缓冲区”,杀死它,它将需要永远。 如果说的是“通过修复构建”,那么它很好 - 它正在排序,然后有效地加载索引。 通过在启动进程之前设置myi ...
-
文件说明: 如果指定REPLACE ,则输入行将替换现有行。 换句话说,主键或唯一索引的值与现有行的值相同。 请参见第13.2.8节“REPLACE语法” 。 REPLACE不是UPDATE 。 REPLACE是SQL的MySQL扩展,如果存在,则首先删除该行,然后插入新的行。 在INSERT ,MySQL使用查询中未提供的字段的默认值。 在您的情况下,这些字段可能默认为NULL 。 无法使用LOAD DATA INFILE更新现有行。 我建议您创建一个工作表,并仅将其用于将数据加载到其中,如下所示: 在使 ...
-
你这里基本上有3个问题。 以相反的顺序 你在个人陈述中进行Python插入吗? 您可能希望使用begin transaction / commit将它们全部包围起来。 20,000次提交可能需要数小时。 您的import语句定义了6个字段,但CSV有7个字段。 这将解释双行计数:每行输入在数据库中产生2行,第2行在字段2-6中为null。 将插入不完整的行,并为缺少的列提供空值或默认值。 对于那些格式错误的行,这可能不是您想要的。 如果你的python程序即使只有一个事务也不能足够快地执行,你至少应该让py ...
-
您需要转义行终止子句的斜杠,否则您只需在查询中添加换行符。 LINES TERMINATED BY '\\n' Mysql只会得到\n ,但PHP试图解释这些,所以你必须逃避它们。 您可能还有其他问题,但您没有告诉我们会发生什么。 You need to escape the slash for the lines terminated clause, otherwise you just added a newline to your query. LINES TERMINATED BY '\\n' My ...
-
MySQL LOAD DATA INFILE ...想要读取空的excel行(MySQL LOAD DATA INFILE … wants to read excel rows that are empty)[2022-12-22]
LOAD DATA INFILE无法读取Excel电子表格。 将Excel电子表格转换为CSV文件,并对转换结果使用LOAD DATA INFILE 。 LOAD DATA INFILE cannot read Excel spreadsheets. Convert the Excel spreadsheet to a CSV file and use LOAD DATA INFILE on the result of the conversion. -
Mysql CSV加载infile(Mysql CSV load infile)[2023-04-08]
$sql = 'LOAD DATA LOCAL INFILE "../csvtemp/test.csv" INTO TABLE sample FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY """" IGNORE 1 LINES (col1, col2, col3, col4, col5, col6, col7, col8, col9)' ; ... -
从txt加载数据Infile(Load Data Infile from txt)[2022-02-08]
您无法使用Load Data infile为您动态创建列。 您必须事先创建列。 可以这样想,用第一行中给出的名称创建列是不够的,因为我们还需要关联的列类型(例如int,date,varchar等)。 You cannot use Load Data infile to dynamically create columns for you. You have to create the columns beforehand. Think of it this way, creating the columns ... -
在2个字段的水果和颜色上创建复合uniqe索引。 ALTER TABLE fruit_table ADD UNIQUE KEY (`fruit` , `colour`) ; Create a composite uniqe index over the 2 fields fruit and colour. ALTER TABLE fruit_table ADD UNIQUE KEY (`fruit` , `colour`) ;
-
MySql:加载数据infile,忽略带有null变量的行(MySql: load data infile, Ignore rows with a null variable)[2022-04-11]
这不能完成,你可以在这里看到这个问题 没有。有一个IGNORE选项。 但是它使用行号而不是内联逻辑比较。 您正在使用的IGNORE部分用于处理重复项,例如无法在此处放置子句的条件。 即使您将列类型设置为NOT NULL ,也会将默认值应用于该字段。 尝试将NULL加载到NOT NULL列会导致为列的数据类型和警告分配隐式默认值,或者在严格SQL模式下分配错误。 第11.7节“数据类型默认值”中讨论了隐式默认值。 请参阅MySQL文档LOAD DATA INFILE ,它也引用 REPLACE和IGNORE关 ...