Mysql在哪里查询优化(Mysql where between query optimization)
以下是自治系统号码数据库的格式(从该站点下载和解析!)。
range_start range_end number cc provider ----------- --------- ------ -- ------------------------------------- 16778240 16778495 56203 AU AS56203 - BIGRED-NET-AU Big Red Group 16793600 16809983 18144 AS18144
745465总行数
普通查询如下所示:
select * from table where 3232235520 BETWEEN range_start AND range_end
工作正常,但我查询大量的IP来检查他们的AS信息,最终需要花费太多的电话和时间。
Profiler快照:
我有两个索引:
- id列
- range_start和range_end列上的组合索引作为make唯一行。
问题:
- 有没有办法在单个查询中查询大量的IP?
- multiple
where (IP between range_start and range_end) OR where (IP between range_start and range_end) OR ...
有效,但我无法获取IP - >行映射或检索哪些IP的行。- 有没有建议改变数据库结构以优化查询速度并减少时间?
任何帮助将不胜感激! 谢谢!
Below is the format of the database of Autonomous System Numbers ( download and parsed from this site! ).
range_start range_end number cc provider ----------- --------- ------ -- ------------------------------------- 16778240 16778495 56203 AU AS56203 - BIGRED-NET-AU Big Red Group 16793600 16809983 18144 AS18144
745465 total rows
A Normal query looks like this:
select * from table where 3232235520 BETWEEN range_start AND range_end
Works properly but I query a huge number of IPs to check for their AS information which ends up taking too many calls and time.
Profiler Snapshot:
I've two indexes:
- id column
- a combine index on the range_start and range_end column as both the make unique row.
Questions:
- Is there a way to query a huge number of IPs in a single query?
- multiple
where (IP between range_start and range_end) OR where (IP between range_start and range_end) OR ...
works but I can't get the IP -> row mapping or which rows are retrieved for which IP.- Any suggestions to change the database structure to optimize the query speed and decrease the time?
Any help will be appreciated! Thanks!
原文:https://stackoverflow.com/questions/42513868
最满意答案
第二次查看你的代码(在我的咖啡之后,它更好),
java.io.IOException: Pipe closed
使用JSch 0.1.42java.io.IOException: Pipe closed
是由你建议的逻辑问题引起的:你的finally块在你的for循环中。 因为始终调用finally块,无论是否发生异常,客户端总是在第一个iteraton末尾的finally块中断开连接:
for(...){ try{ ... }catch(...){ ... }finally{ # this finally block is inside your for loop # finally block is always called whether an exception occur or not # on the first iteration, this is called and close your client try{ if(client!=null){ # nope, it will close the client for next iteration! client.disconnect(); }if(fsdisPath!=null){ # good to close this however fsdisPath.close(); } }catch(Exceptionex){ ex.printStackTrace(); } } }
相反,你应该有类似的东西:
try { for(...){ try{ ... }catch(...){ ... }finally{ try{ if(fsdisPath!=null){ fsdisPath.close(); } }catch(Exceptionex){ ex.printStackTrace(); } } } } finally { if(client!=null){ client.disconnect(); } }
将for循环作为代码可用性的函数也可能是个好主意;)
java.lang.NoSuchFieldError: identities
是由您的应用程序使用不同的JSch版本的另一个标准引起的,因为您根据Maven依赖关系树提供了证据。做了一个mvn依赖:tree -Dverbose在项目中(使用jsch 0.1.50)我发现该项目和我在原帖中添加了所有日志:特别是:+ - org.mule.transports:mule-transport -sftp:jar:3.4.0:编译[INFO] | + - (com.jcraft:jsch:jar:0.1.44-1:compile - 与0.1.42冲突省略)+ - (com.jcraft:jsch:jar:0.1.42:compile - 省略与0.1的冲突。 50)[INFO] - com.jcraft:jsch:jar:0.1.50:编译[INFO]
Looking at your code a second time (right after my coffee, it's better),
java.io.IOException: Pipe closed
with JSch 0.1.42 is effectively caused by a logic issue as you suggested:Your finally block is inside your for loop. As a finally block is always called whether an exception occur or not your client is always disconnected by the finally block at the end of the very first iteraton:
for(...){ try{ ... }catch(...){ ... }finally{ # this finally block is inside your for loop # finally block is always called whether an exception occur or not # on the first iteration, this is called and close your client try{ if(client!=null){ # nope, it will close the client for next iteration! client.disconnect(); }if(fsdisPath!=null){ # good to close this however fsdisPath.close(); } }catch(Exceptionex){ ex.printStackTrace(); } } }
Instead you should have something like:
try { for(...){ try{ ... }catch(...){ ... }finally{ try{ if(fsdisPath!=null){ fsdisPath.close(); } }catch(Exceptionex){ ex.printStackTrace(); } } } } finally { if(client!=null){ client.disconnect(); } }
It may also be a good idea to have your for loop as a function for code lisibility ;)
java.lang.NoSuchFieldError: identities
is caused by another par of your application using a different JSch version as you put into evidence per your Maven dependency tree.did a mvn dependency:tree -Dverbose in the project (using jsch 0.1.50) and I found that the project and I have added all the logs in the original post: Particularly this : +- org.mule.transports:mule-transport-sftp:jar:3.4.0:compile [INFO] | +- (com.jcraft:jsch:jar:0.1.44-1:compile - omitted for conflict with 0.1.42) +- (com.jcraft:jsch:jar:0.1.42:compile - omitted for conflict with 0.1.50) [INFO] - com.jcraft:jsch:jar:0.1.50:compile [INFO]
相关问答
更多-
尝试使用路径而不是流: String destPath = "filename.txt"; if (!filesToDownload.isEmpty()) { for (String fileDownloadName : filesToDownload) { success = false; sftpChannel.get(fileDownloadName , destPath); 编辑如果 ...
-
第二次查看你的代码(在我的咖啡之后,它更好), java.io.IOException: Pipe closed使用JSch 0.1.42 java.io.IOException: Pipe closed是由你建议的逻辑问题引起的: 你的finally块在你的for循环中。 因为始终调用finally块,无论是否发生异常,客户端总是在第一个iteraton末尾的finally块中断开连接: for(...){ try{ ... }catch(...){ ... ...
-
使用JSch进行SFTP时无法连接到安全通道(Not able to connect to Secured Channel when doing SFTP with JSch)[2022-03-04]
尝试在连接之前将频道转换为ChannelSft: Channel channel = session.openChannel("sftp"); ChannelSftp channelSftp = (ChannelSftp) channel; channelSftp.connect(); Try to cast your channel into ChannelSft before the connect: Channel channel = session.openChannel("sftp"); Cha ... -
SSH工作没有任何问题。 问题在于SSH配置中的SFTP模块路径。 以下链接无法访问。 # override default of no subsystems Subsystem sftp /usr/libexec/openssh/sftp-server SSH was working without any issue. Problem was with SFTP module path in SSH config. Below link was not accessible. # override ...
-
ChannelSftp sftpChannel = (ChannelSftp) channel; sftpChannel.get(fileName); Log.d(fileName, " has been downloaded"); ChannelSftp.get()的单参数版本不会将远程文件写入本地文件。 它返回一个InputStream。 您应该从InputStream中读取远程文件的内容,例如: try (FileOutputStream out = new FileOutputStream("/s ...
-
可以通过WinSCP打开SFTP连接,但不能与JSch连接(Can open SFTP connection via WinSCP, but cannot with JSch)[2023-10-28]
问题是由于网络的拓扑结构和通信方的许可。 如果您遇到同样的问题,请考虑这些问题。 The Issue was because of the topology of the network and permissions of the communicating parties. If you face the same problem, please take these issues in to consideration. -
正如VGR在上面的评论中指出的,传输工作正常,只是编辑器使用错误的编码来打开文件 As pointed out by VGR in a comment above the transfer was working fine, it was just the editor using the wrong encoding to open the file
-
使用JSch连接SFTP时如何选择网络接口(How to select network interface when connecting to SFTP with JSch)[2022-03-11]
我不认为在JSch中选择要连接的网络接口有明确的支持。 但是你可以实现一个SocketFactory接口,比如它使用带有localAddr的构造函数重载创建一个Socket实例: public Socket(InetAddress address, int port, InetAddress localAddr, int localPort) (对localPort使用0) 然后在调用connect()之前将工厂传递给Session.setSocketFactory() connect() ,你应该很好。 ... -
我使用CompletableFuture解决了异步执行传输的问题(看看runAsync和supplyAsync方法)。 该课程还提供了将多个期货与以下方法结合或链接的方法: CompletableFuture.allOf , CompletableFuture.thenCompose等... I solved the problem performing the transfer asynchronously using CompletableFuture(take a look at runAsync a ...
-
从异常消息判断,似乎客户端和SSH服务器之间没有共享密钥交换(KEX)算法。 您可以在尝试连接之前通过启用JSch登录来验证这一点: JSch.setLogger(new Logger() { @Override public boolean isEnabled(int i) { return true; } @Override public void log(int i, String string) { System.out.pri ...