Java和Linux OS随机挂起24分钟(Linux,arm,Debian,Java 7 ARM)(Java and Linux OS hangs randomly for 24 minutes (Linux, arm, Debian, Java 7 ARM))
背景/背景:
我们在CompuLab CoM上运行Java应用程序:
https://compulab.co.il/products/computer-on-modules/cm-fx6/#overview
JVM版本:Oracle Java 7 ARM 1.7.0_60
OS参考:
http://www.compulab.co.il/workspace/mediawiki/index.php5/CM-FX6_Linux
该应用程序并非易事:许多线程,访问以太网(LAN),串行接口,GPRS / UMTS调制解调器,访问Internet(ppp deamon),GPS,触摸屏,数据库(SQLite),文件系统。 换句话说,广泛使用OS资源.....
我们正在观察Java应用程序(它的所有线程)和OS基本功能随机挂起。 我会说这是一个Linux内核错误,但通过杀死Java应用程序,它恢复并正常运行。
这种状态总是需要24分钟。 之后它恢复并且表现正常。 平均发生率是每24-30小时一次。
当它发生时,外部调用的事件,如通过以太网或串行接口发送到应用程序的消息被缓冲(可能是OS),所有这些事件在恢复后立即处理。
当我事先建立与设备的SSH连接时,在它发生之后连接被阻止(所有命令在恢复后被缓冲和处理 - 24分钟)或其工作,而不是:
- 基本的OS实用程序不起作用:例如“top”
- jstack -F不起作用,只是挂起并且不产生任何输出
- 通过kill -9来杀死Java应用程序PID释放了操作系统,一切都开始正常运行
当它处于这种状态时,操作系统每次都表现不同。 其他发现:
- 基于网络的基本实用程序不起作用(SSH,FTP) - 无法与其他计算机建立与OS的新连接。
- 从我的另一台机器PING工作,直到我从设备上拔下插头以太网电缆,有时PING比停止工作
- 有时OS系统时间也会挂起(并非总是如此),24分钟后它会持续延迟24分钟。
- 在该状态下(始终发生),无法连接新的USB输入设备(鼠标,键盘)。
另一件奇怪的事:
触摸屏用于与用户交互(编译为内核模块的驱动程序)。 即使挂起也能正常工作。 Java应用程序(GUI Swing)可以处理按下按钮等事件,因此我可以在按钮单击处理程序后面运行一些代码。 似乎所有线程都被阻止,但Java Swing可以处理一些输入事件,我们的应用程序会先处理它们,直到它需要与已经被阻塞的线程或操作系统(按钮点击时运行bash脚本)或调用sleep方法进行交互。 比它还挂起来。 换句话说,Java应用程序“部分”挂起 - 仍然可以处理某些事情。
已经尝试过:
用于JVM远程调试的工具:Java Mission Control,VisualVM。 在挂起之前也建立了连接。 在线程转储,堆转储等方面,一切似乎都没问题(我可以通过电子邮件发送)。 即使连接仍然存在,我可以在工具中看到JVM的处理器使用率降至0%。
jstack -F(通过SSH):不起作用,只是挂起并且不产生任何输出
我试图在没有触摸屏驱动程序的情况下运行操作系统,但它仍然发生了。
我试着运行两个并行的Java应用程序。 其中一个非常简单 - 只需写入日志时间戳。 他们俩都挂了。
我尝试在按钮点击处理程序的同时运行System.exit(0)。 和所有线程挂起,它不起作用(挂起)
问题:
是Linux内核错误还是JVM(其ARM实现)错误?
Java(JVM)是否能够挂起并阻止基本的OS功能(FTP,SSH,系统时间,其他实用程序)?
当jstack -F等基本实用程序不起作用时,如何进一步诊断/调试此问题?
你有什么想法可能是这个问题的原因,为什么它总是在24分钟后恢复?
更新1:2014-07-10
最后,我设法再次“抓住”这个奇怪的状态。 以下是我的进一步发现。
基于nos建议我尝试通过ssh运行(在高级中建立):
*strace -f -p PID*
不幸的是,bash脚本命令也挂起(与jstack相同的行为)。
就用户限制(ulimit)和操作系统资源而言,下面我报告刚从系统上次挂起后恢复的数据 。 在该州,它已经运行了24小时 ,我可以确认这些数字在长期运行期间保持大致相同(在运行期间没有随机偷看)。 从我的角度来看,它们没问题,应用程序也没有以任何方式踩过任何资源或其他限制。
Java当前堆
使用:18 MB,免费:12 MB,总计:30 MB,最大:230 MB
Java堆
root@cm-debian:~# /usr/lib/jvm/jdk1.7.0_60/bin/jmap -heap 3242 Attaching to process ID 3242, please wait... Debugger attached successfully. Client compiler detected. JVM version is 24.60-b09 using thread-local object allocation. Mark Sweep Compact GC Heap Configuration: MinHeapFreeRatio = 40 MaxHeapFreeRatio = 70 MaxHeapSize = 249561088 (238.0MB) NewSize = 1048576 (1.0MB) MaxNewSize = 4294836224 (4095.875MB) OldSize = 4194304 (4.0MB) NewRatio = 2 SurvivorRatio = 8 PermSize = 12582912 (12.0MB) MaxPermSize = 67108864 (64.0MB) G1HeapRegionSize = 0 (0.0MB) Heap Usage: New Generation (Eden + 1 Survivor Space): capacity = 10092544 (9.625MB) used = 6772088 (6.458366394042969MB) free = 3320456 (3.1666336059570312MB) 67.09991058745942% used Eden Space: capacity = 9043968 (8.625MB) used = 6620336 (6.3136444091796875MB) free = 2423632 (2.3113555908203125MB) 73.2016743093297% used From Space: capacity = 1048576 (1.0MB) used = 151752 (0.14472198486328125MB) free = 896824 (0.8552780151367188MB) 14.472198486328125% used To Space: capacity = 1048576 (1.0MB) used = 0 (0.0MB) free = 1048576 (1.0MB) 0.0% used tenured generation: capacity = 22134784 (21.109375MB) used = 17650936 (16.83324432373047MB) free = 4483848 (4.276130676269531MB) 79.7429782915433% used Perm Generation: capacity = 19136512 (18.25MB) used = 19023016 (18.141761779785156MB) free = 113496 (0.10823822021484375MB) 99.40691386183647% used 9597 interned Strings occupying 729344 bytes.
最佳
top - 11:41:29 up 21:59, 2 users, load average: 1.51, 1.25, 1.22 Tasks: 93 total, 1 running, 92 sleeping, 0 stopped, 0 zombie Cpu(s): 9.4%us, 8.0%sy, 0.0%ni, 82.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 966780k total, 273080k used, 693700k free, 27216k buffers Swap: 0k total, 0k used, 0k free, 126352k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3242 root 20 0 398m 79m 11m S 23.6 8.4 346:16.82 java 3889 root 20 0 2804 1096 848 R 5.5 0.1 0:00.07 top 1 root 20 0 2124 688 596 S 0.0 0.1 0:02.92 init 2 root 20 0 0 0 0 S 0.0 0.0 0:00.03 kthreadd 3 root 20 0 0 0 0 S 0.0 0.0 0:14.32 ksoftirqd/0 5 root 20 0 0 0 0 S 0.0 0.0 0:00.14 kworker/u:0 6 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0 7 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 khelper
java限制
root@cm-debian:~# java -XX:+PrintFlagsFinal -version | grep -iE 'HeapSize|PermSize|ThreadStackSize' uintx AdaptivePermSizeWeight = 20 {product} intx CompilerThreadStackSize = 0 {pd product} uintx ErgoHeapSizeLimit = 0 {product} uintx HeapSizePerGCThread = 67108864 {product} uintx InitialHeapSize := 15468480 {product} uintx LargePageHeapSizeThreshold = 134217728 {product} uintx MaxHeapSize := 249561088 {product} uintx MaxPermSize = 67108864 {pd product} uintx PermSize = 12582912 {pd product} intx ThreadStackSize = 320 {pd product} intx VMThreadStackSize = 512 {pd product} java version "1.7.0_60" Java(TM) SE Runtime Environment (build 1.7.0_60-b19) Java HotSpot(TM) Client VM (build 24.60-b09, mixed mode)
过程限制
root@cm-debian:~# cat /proc/3242/limits Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes unlimited unlimited processes Max open files 8192 8192 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 16382 16382 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us
系统内存信息
root@cm-debian:~# cat /proc/meminfo MemTotal: 966780 kB MemFree: 694312 kB Buffers: 27384 kB Cached: 126364 kB SwapCached: 0 kB Active: 140748 kB Inactive: 107684 kB Active(anon): 94992 kB Inactive(anon): 2064 kB Active(file): 45756 kB Inactive(file): 105620 kB Unevictable: 0 kB Mlocked: 0 kB HighTotal: 524288 kB HighFree: 301088 kB LowTotal: 442492 kB LowFree: 393224 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 0 kB Writeback: 0 kB AnonPages: 94692 kB Mapped: 21220 kB Shmem: 2376 kB Slab: 13268 kB SReclaimable: 5284 kB SUnreclaim: 7984 kB KernelStack: 960 kB PageTables: 980 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 483388 kB Committed_AS: 137260 kB VmallocTotal: 286720 kB VmallocUsed: 2928 kB VmallocChunk: 283040 kB root@cm-debian:~# vmstat -s 966780 K total memory 272468 K used memory 140776 K active memory 107712 K inactive memory 694312 K free memory 27392 K buffer memory 126404 K swap cache 0 K total swap 0 K used swap 0 K free swap 726963 non-nice user cpu ticks 0 nice user cpu ticks 621187 system cpu ticks 6371123 idle cpu ticks 3683 IO-wait cpu ticks 324 IRQ cpu ticks 2146 softirq cpu ticks 0 stolen cpu ticks 130871 pages paged in 97520 pages paged out 0 pages swapped in 0 pages swapped out 293822206 interrupts 494034482 CPU context switches 1412595732 boot time 3916 forks
线程
root@cm-debian:~# cat /proc/sys/kernel/pid_max 32768 root@cm-debian:~# cat /proc/sys/kernel/threads-max 15102 root@cm-debian:~# cat /proc/sys/vm/max_map_count 65530 root@cm-debian:~# ls -l /proc/3242/task/ | wc -l 33 root@cm-debian:~# ps huH p 3242 | wc -l 32 root@cm-debian:~# grep -s '^Threads' /proc/[0-9]*/status | awk '{ sum += $2; } END { print sum; }' 122
打开文件/文件描述符
root@cm-debian:~# ls -l /proc/3242/fd | wc -l 81
更新2:2014-13-10
这次我在操作系统挂起时记录了所有Java线程堆栈跟踪(如前所述,触摸屏及其事件仍然有效,因此我根据UI按钮处理程序将堆栈跟踪写入日志文件)。
从我的角度来看, 所有线程都处于“正确”状态 (休眠,等待UDP数据报等)。 很明显,挂起不是由Java应用程序SW操作引起的,这将花费24分钟 。
10:49:42,293> [INFO ] THREAD stack traces: **************************************** ID: 56, name: Mpg123AudioPlayer_PASSENGER_ctrlLoop java.lang.Thread.sleep(Native Method) java.lang.Thread.sleep(Thread.java:340) java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360) epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169) epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19) epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88) java.lang.Thread.run(Thread.java:745) ID: 11, name: AWT-EventQueue-0 java.lang.Thread.getStackTrace(Thread.java:1589) epis5fcc.domain.debug.ThreadStackTracesLogger.log(ThreadStackTracesLogger.java:30) epis5fcc.ui.settings.FccRegistryScreen$7.actionPerformed(FccRegistryScreen.java:303) javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2018) javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2341) javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402) javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259) javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252) java.awt.Component.processMouseEvent(Component.java:6516) javax.swing.JComponent.processMouseEvent(JComponent.java:3320) java.awt.Component.processEvent(Component.java:6281) java.awt.Container.processEvent(Container.java:2229) java.awt.Component.dispatchEventImpl(Component.java:4872) java.awt.Container.dispatchEventImpl(Container.java:2287) java.awt.Component.dispatchEvent(Component.java:4698) java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4832) java.awt.LightweightDispatcher.processMouseEvent(Container.java:4492) java.awt.LightweightDispatcher.dispatchEvent(Container.java:4422) java.awt.Container.dispatchEventImpl(Container.java:2273) java.awt.Window.dispatchEventImpl(Window.java:2719) java.awt.Component.dispatchEvent(Component.java:4698) java.awt.EventQueue.dispatchEventImpl(EventQueue.java:735) java.awt.EventQueue.access$200(EventQueue.java:103) java.awt.EventQueue$3.run(EventQueue.java:694) java.awt.EventQueue$3.run(EventQueue.java:692) java.security.AccessController.doPrivileged(Native Method) java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76) java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:87) java.awt.EventQueue$4.run(EventQueue.java:708) java.awt.EventQueue$4.run(EventQueue.java:706) java.security.AccessController.doPrivileged(Native Method) java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76) java.awt.EventQueue.dispatchEvent(EventQueue.java:705) java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242) java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161) java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150) java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146) java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138) java.awt.EventDispatchThread.run(EventDispatchThread.java:91) ID: 34, name: Mpg123AudioPlayer_DRIVER_ctrlLoop java.lang.Thread.sleep(Native Method) java.lang.Thread.sleep(Thread.java:340) java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360) epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169) epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19) epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88) java.lang.Thread.run(Thread.java:745) ID: 26, name: IOTxUdpAccessLoop_IODispatchAccess java.lang.Thread.sleep(Native Method) jCommons.comm.io.access.IOUdpAccess.transmitLoop(IOUdpAccess.java:114) jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50) java.lang.Thread.run(Thread.java:745) ID: 29, name: MasterLoop_main java.lang.Thread.sleep(Native Method) jCommons.master.MasterLoop.ctrlLoop(MasterLoop.java:87) jCommons.master.MasterLoop.access$000(MasterLoop.java:11) jCommons.master.MasterLoop$1.run(MasterLoop.java:58) java.lang.Thread.run(Thread.java:745) ID: 27, name: IORxSerialPortAccessPollLoop_IOModemAccess java.lang.Thread.sleep(Native Method) jCommons.comm.io.access.IOSerialPortAccessPoll.reciveLoop(IOSerialPortAccessPoll.java:256) jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43) java.lang.Thread.run(Thread.java:745) ID: 31, name: UsbUpdateWatchService_ctrlLoop sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489) java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678) sun.nio.fs.AbstractWatchService.take(AbstractWatchService.java:118) jCommons.update.usb.UsbUpdateWatchService.ctrlLoop(UsbUpdateWatchService.java:107) jCommons.update.usb.UsbUpdateWatchService.access$000(UsbUpdateWatchService.java:25) jCommons.update.usb.UsbUpdateWatchService$1.run(UsbUpdateWatchService.java:75) java.lang.Thread.run(Thread.java:745) ID: 25, name: IORxUdpAccessLoop_IODispatchAccess java.net.PlainDatagramSocketImpl.receive0(Native Method) java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:145) java.net.DatagramSocket.receive(DatagramSocket.java:786) jCommons.comm.io.access.IOUdpAccess.reciveLoop(IOUdpAccess.java:175) jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43) java.lang.Thread.run(Thread.java:745) ID: 2, name: Reference Handler java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) ID: 30, name: VehicleCtrl_ctrlLoop java.lang.Thread.sleep(Native Method) epis5fcc.domain.vehicle.control.VehicleCtrl.ctrlLoop(VehicleCtrl.java:74) jCommons.comm.protocol.ProtCtrlBase$1.run(ProtCtrlBase.java:24) java.lang.Thread.run(Thread.java:745) ID: 35, name: Mpg123AudioPlayer_INNER_ctrlLoop java.lang.Thread.sleep(Native Method) java.lang.Thread.sleep(Thread.java:340) java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360) epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169) epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19) epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88) java.lang.Thread.run(Thread.java:745) ID: 21, name: IORxSerialPortAccessPollLoop_IOFccAccess java.lang.Thread.sleep(Native Method) jCommons.comm.io.access.IOSerialPortAccessPoll.reciveLoop(IOSerialPortAccessPoll.java:256) jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43) java.lang.Thread.run(Thread.java:745) ID: 7, name: FileWatchdog java.lang.Thread.sleep(Native Method) org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:104) ID: 8, name: Java2D Disposer java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) sun.java2d.Disposer.run(Disposer.java:145) java.lang.Thread.run(Thread.java:745) ID: 17, name: com.google.inject.internal.util.$Finalizer java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) com.google.inject.internal.util.$Finalizer.run(Finalizer.java:114) ID: 10, name: AWT-XAWT sun.awt.X11.XToolkit.waitForEvents(Native Method) sun.awt.X11.XToolkit.run(XToolkit.java:541) sun.awt.X11.XToolkit.run(XToolkit.java:505) java.lang.Thread.run(Thread.java:745) ID: 32, name: Thread-4 sun.nio.fs.LinuxWatchService.poll(Native Method) sun.nio.fs.LinuxWatchService.access$600(LinuxWatchService.java:47) sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:311) java.lang.Thread.run(Thread.java:745) ID: 28, name: IOTxSerialPortAccessPollLoop_IOModemAccess java.lang.Thread.sleep(Native Method) jCommons.comm.io.access.IOSerialPortAccessPoll.transmitLoop(IOSerialPortAccessPoll.java:187) jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50) java.lang.Thread.run(Thread.java:745) ID: 14, name: DestroyJavaVM ID: 22, name: IOTxSerialPortAccessPollLoop_IOFccAccess java.lang.Thread.sleep(Native Method) jCommons.comm.io.access.IOSerialPortAccessPoll.transmitLoop(IOSerialPortAccessPoll.java:187) jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50) java.lang.Thread.run(Thread.java:745) ID: 19, name: TimerQueue sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) java.util.concurrent.DelayQueue.take(DelayQueue.java:220) javax.swing.TimerQueue.run(TimerQueue.java:171) java.lang.Thread.run(Thread.java:745) ID: 12, name: AWT-Shutdown java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:296) java.lang.Thread.run(Thread.java:745) ID: 23, name: IORxUdpAccessLoop_IOCityScrnAccess java.net.PlainDatagramSocketImpl.receive0(Native Method) java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:145) java.net.DatagramSocket.receive(DatagramSocket.java:786) jCommons.comm.io.access.IOUdpAccess.reciveLoop(IOUdpAccess.java:175) jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43) java.lang.Thread.run(Thread.java:745) ID: 3, name: Finalizer java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) ID: 4, name: Signal Dispatcher ID: 52, name: pool-3-thread-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) ID: 24, name: IOTxUdpAccessLoop_IOCityScrnAccess java.lang.Thread.sleep(Native Method) jCommons.comm.io.access.IOUdpAccess.transmitLoop(IOUdpAccess.java:114) jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50) java.lang.Thread.run(Thread.java:745) ID: 36, name: RemoteUpdateCtrl_ctrlLoop java.lang.Thread.sleep(Native Method) epis5fcc.domain.update.remote.RemoteUpdateCtrl.ctrlLoop(RemoteUpdateCtrl.java:94) jCommons.comm.protocol.ProtCtrlBase$1.run(ProtCtrlBase.java:24) java.lang.Thread.run(Thread.java:745) ID: 55, name: Mpg123AudioPlayer_OUTER_ctrlLoop java.lang.Thread.sleep(Native Method) java.lang.Thread.sleep(Thread.java:340) java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360) epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169) epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19) epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88) java.lang.Thread.run(Thread.java:745)
Background/context:
We are running Java application on one of CompuLab CoM:
https://compulab.co.il/products/computer-on-modules/cm-fx6/#overview
JVM version: Oracle Java 7 ARM 1.7.0_60
OS reference:
http://www.compulab.co.il/workspace/mediawiki/index.php5/CM-FX6_Linux
The application is not trivial: lots of threads, access to Ethernet (LAN), serial interface, GPRS/UMTS modem, access to Internet (ppp deamon), GPS, touch screen, database (SQLite), file system. In other words use OS resources extensively.....
We are observing that Java application (all of its threads) and OS basic functionality randomly hangs. I would say it is a Linux kernel bug but by killing the Java application it recovers and operates normally.
This state always takes exactly 24 minutes. Afterwards it recovers and behaves normally. Average rate of occurrence is once per 24-30 hours.
When it happens, externally invoked events like messages sent to application via Ethernet or serial interface are buffered (by OS probably) and all of them are processed immediately after it recovers.
When I establish SSH connection to device in advance, after it happens the connection is either blocked (all command are buffered and processed after it recovers - 24 mintes) or its working, than:
- basic OS utilities does not work: "top" for example
- jstack -F does not work, just hangs and does not produce any output
- killing Java application by kill -9 PID released the OS and everything starts to operate normally
While it is in this state, the OS each time behaves differently. Other findings:
- Basic network based utilities does not work (SSH, FTP) – can not establish new connection to OS from another machine.
- PING from another machine does work until I unplug an plug Ethernet cable from device, sometimes PING than stops working
- Sometimes OS system time hangs as well (not always), after 24 minutes it continues delayed for 24 minutes.
- New USB input devices (mouse, keyboard) can not be connected while in that state (happens always).
Another strange thing:
A touch screen is used for interaction with a user (driver compiled as kernel module). And it works even while it is hung. Java application (GUI Swing) can handle events like pressing button so I can run some code behind button click handler. It seems like all threads are blocked but Java Swing can handle some input events and our application precesses them until it needs to interact with already blocked threads or OS (run bash script on button click) or call sleep method. Than it hangs as well. In other words, the Java application is hung ”partially” - can still handle something.
Already tried:
Tools for JVM remote debugging: Java Mission Control, VisualVM. Connection was also established before it hung. Everything seemed OK in terms of thread dump, heap dump etc. (I can send by e-mail). Even the connection remained and I could see in thees tools that processor usage dropped to 0 % for JVM.
jstack -F (via SSH): does not work, just hangs and does not produce any output
I tried to run OS without the driver for touch screen and it still happened.
I tried to run two parallel Java application. One of them was very simple – just writing to log timestamps. And both of them hung.
I tried to run System.exit(0) in terms of button click handler while app. and all threads hung and it does not worked (hung as well)
Questions:
Is it Linux kernel bug or JVM (its ARM implementation) bug?
Is Java (JVM) able to hang and block basic OS functionality (FTP, SSH, system time, other utilities)?
How can I further diagnose/debug this issue when basic utilities like jstack -F does not work?
Do you have any ideas what could be the cause of this issue and why it always recovers exactly after 24 minutes?
Update 1: 2014-07-10
Finally I manage to “catch” this weird state again. Here are my further findings.
Based on nos suggestion I tried run via ssh (established in advanced):
*strace -f -p PID*
Unfortunately the bash script command hung as well (same behavior like with jstack).
As far as the user limit (ulimit) and OS resources are concerned, bellow I report figures taken just after the system recovered from last hung. At that state it had been running for 24 hours and I can confirm that those figures remain roughly the same during long-term operation (no random peeks during operation). From my point of view, they are ok and application is not stepping over any resource or other limit in any way.
Java current heap
Used: 18 MB, Free: 12 MB, Total: 30 MB, Max: 230 MB
Java heap
root@cm-debian:~# /usr/lib/jvm/jdk1.7.0_60/bin/jmap -heap 3242 Attaching to process ID 3242, please wait... Debugger attached successfully. Client compiler detected. JVM version is 24.60-b09 using thread-local object allocation. Mark Sweep Compact GC Heap Configuration: MinHeapFreeRatio = 40 MaxHeapFreeRatio = 70 MaxHeapSize = 249561088 (238.0MB) NewSize = 1048576 (1.0MB) MaxNewSize = 4294836224 (4095.875MB) OldSize = 4194304 (4.0MB) NewRatio = 2 SurvivorRatio = 8 PermSize = 12582912 (12.0MB) MaxPermSize = 67108864 (64.0MB) G1HeapRegionSize = 0 (0.0MB) Heap Usage: New Generation (Eden + 1 Survivor Space): capacity = 10092544 (9.625MB) used = 6772088 (6.458366394042969MB) free = 3320456 (3.1666336059570312MB) 67.09991058745942% used Eden Space: capacity = 9043968 (8.625MB) used = 6620336 (6.3136444091796875MB) free = 2423632 (2.3113555908203125MB) 73.2016743093297% used From Space: capacity = 1048576 (1.0MB) used = 151752 (0.14472198486328125MB) free = 896824 (0.8552780151367188MB) 14.472198486328125% used To Space: capacity = 1048576 (1.0MB) used = 0 (0.0MB) free = 1048576 (1.0MB) 0.0% used tenured generation: capacity = 22134784 (21.109375MB) used = 17650936 (16.83324432373047MB) free = 4483848 (4.276130676269531MB) 79.7429782915433% used Perm Generation: capacity = 19136512 (18.25MB) used = 19023016 (18.141761779785156MB) free = 113496 (0.10823822021484375MB) 99.40691386183647% used 9597 interned Strings occupying 729344 bytes.
top
top - 11:41:29 up 21:59, 2 users, load average: 1.51, 1.25, 1.22 Tasks: 93 total, 1 running, 92 sleeping, 0 stopped, 0 zombie Cpu(s): 9.4%us, 8.0%sy, 0.0%ni, 82.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 966780k total, 273080k used, 693700k free, 27216k buffers Swap: 0k total, 0k used, 0k free, 126352k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3242 root 20 0 398m 79m 11m S 23.6 8.4 346:16.82 java 3889 root 20 0 2804 1096 848 R 5.5 0.1 0:00.07 top 1 root 20 0 2124 688 596 S 0.0 0.1 0:02.92 init 2 root 20 0 0 0 0 S 0.0 0.0 0:00.03 kthreadd 3 root 20 0 0 0 0 S 0.0 0.0 0:14.32 ksoftirqd/0 5 root 20 0 0 0 0 S 0.0 0.0 0:00.14 kworker/u:0 6 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0 7 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 khelper
java limits
root@cm-debian:~# java -XX:+PrintFlagsFinal -version | grep -iE 'HeapSize|PermSize|ThreadStackSize' uintx AdaptivePermSizeWeight = 20 {product} intx CompilerThreadStackSize = 0 {pd product} uintx ErgoHeapSizeLimit = 0 {product} uintx HeapSizePerGCThread = 67108864 {product} uintx InitialHeapSize := 15468480 {product} uintx LargePageHeapSizeThreshold = 134217728 {product} uintx MaxHeapSize := 249561088 {product} uintx MaxPermSize = 67108864 {pd product} uintx PermSize = 12582912 {pd product} intx ThreadStackSize = 320 {pd product} intx VMThreadStackSize = 512 {pd product} java version "1.7.0_60" Java(TM) SE Runtime Environment (build 1.7.0_60-b19) Java HotSpot(TM) Client VM (build 24.60-b09, mixed mode)
process limits
root@cm-debian:~# cat /proc/3242/limits Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes unlimited unlimited processes Max open files 8192 8192 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 16382 16382 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us
system memory info
root@cm-debian:~# cat /proc/meminfo MemTotal: 966780 kB MemFree: 694312 kB Buffers: 27384 kB Cached: 126364 kB SwapCached: 0 kB Active: 140748 kB Inactive: 107684 kB Active(anon): 94992 kB Inactive(anon): 2064 kB Active(file): 45756 kB Inactive(file): 105620 kB Unevictable: 0 kB Mlocked: 0 kB HighTotal: 524288 kB HighFree: 301088 kB LowTotal: 442492 kB LowFree: 393224 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 0 kB Writeback: 0 kB AnonPages: 94692 kB Mapped: 21220 kB Shmem: 2376 kB Slab: 13268 kB SReclaimable: 5284 kB SUnreclaim: 7984 kB KernelStack: 960 kB PageTables: 980 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 483388 kB Committed_AS: 137260 kB VmallocTotal: 286720 kB VmallocUsed: 2928 kB VmallocChunk: 283040 kB root@cm-debian:~# vmstat -s 966780 K total memory 272468 K used memory 140776 K active memory 107712 K inactive memory 694312 K free memory 27392 K buffer memory 126404 K swap cache 0 K total swap 0 K used swap 0 K free swap 726963 non-nice user cpu ticks 0 nice user cpu ticks 621187 system cpu ticks 6371123 idle cpu ticks 3683 IO-wait cpu ticks 324 IRQ cpu ticks 2146 softirq cpu ticks 0 stolen cpu ticks 130871 pages paged in 97520 pages paged out 0 pages swapped in 0 pages swapped out 293822206 interrupts 494034482 CPU context switches 1412595732 boot time 3916 forks
threads
root@cm-debian:~# cat /proc/sys/kernel/pid_max 32768 root@cm-debian:~# cat /proc/sys/kernel/threads-max 15102 root@cm-debian:~# cat /proc/sys/vm/max_map_count 65530 root@cm-debian:~# ls -l /proc/3242/task/ | wc -l 33 root@cm-debian:~# ps huH p 3242 | wc -l 32 root@cm-debian:~# grep -s '^Threads' /proc/[0-9]*/status | awk '{ sum += $2; } END { print sum; }' 122
open files / file descriptors
root@cm-debian:~# ls -l /proc/3242/fd | wc -l 81
Update 2: 2014-13-10
This time I logged all Java threads stack traces while the OS was hung (as I stated previously, the touch screen and its events still works so I wrote stack traces to log file in terms of UI button handler).
From my point of view, all threads are in “correct” state (sleeping, waiting for UDP datagram etc..) and it is obvious that the hang is not caused by a Java application SW operation which would took 24 minutes.
10:49:42,293> [INFO ] THREAD stack traces: **************************************** ID: 56, name: Mpg123AudioPlayer_PASSENGER_ctrlLoop java.lang.Thread.sleep(Native Method) java.lang.Thread.sleep(Thread.java:340) java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360) epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169) epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19) epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88) java.lang.Thread.run(Thread.java:745) ID: 11, name: AWT-EventQueue-0 java.lang.Thread.getStackTrace(Thread.java:1589) epis5fcc.domain.debug.ThreadStackTracesLogger.log(ThreadStackTracesLogger.java:30) epis5fcc.ui.settings.FccRegistryScreen$7.actionPerformed(FccRegistryScreen.java:303) javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2018) javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2341) javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402) javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259) javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252) java.awt.Component.processMouseEvent(Component.java:6516) javax.swing.JComponent.processMouseEvent(JComponent.java:3320) java.awt.Component.processEvent(Component.java:6281) java.awt.Container.processEvent(Container.java:2229) java.awt.Component.dispatchEventImpl(Component.java:4872) java.awt.Container.dispatchEventImpl(Container.java:2287) java.awt.Component.dispatchEvent(Component.java:4698) java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4832) java.awt.LightweightDispatcher.processMouseEvent(Container.java:4492) java.awt.LightweightDispatcher.dispatchEvent(Container.java:4422) java.awt.Container.dispatchEventImpl(Container.java:2273) java.awt.Window.dispatchEventImpl(Window.java:2719) java.awt.Component.dispatchEvent(Component.java:4698) java.awt.EventQueue.dispatchEventImpl(EventQueue.java:735) java.awt.EventQueue.access$200(EventQueue.java:103) java.awt.EventQueue$3.run(EventQueue.java:694) java.awt.EventQueue$3.run(EventQueue.java:692) java.security.AccessController.doPrivileged(Native Method) java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76) java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:87) java.awt.EventQueue$4.run(EventQueue.java:708) java.awt.EventQueue$4.run(EventQueue.java:706) java.security.AccessController.doPrivileged(Native Method) java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76) java.awt.EventQueue.dispatchEvent(EventQueue.java:705) java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242) java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161) java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150) java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146) java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138) java.awt.EventDispatchThread.run(EventDispatchThread.java:91) ID: 34, name: Mpg123AudioPlayer_DRIVER_ctrlLoop java.lang.Thread.sleep(Native Method) java.lang.Thread.sleep(Thread.java:340) java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360) epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169) epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19) epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88) java.lang.Thread.run(Thread.java:745) ID: 26, name: IOTxUdpAccessLoop_IODispatchAccess java.lang.Thread.sleep(Native Method) jCommons.comm.io.access.IOUdpAccess.transmitLoop(IOUdpAccess.java:114) jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50) java.lang.Thread.run(Thread.java:745) ID: 29, name: MasterLoop_main java.lang.Thread.sleep(Native Method) jCommons.master.MasterLoop.ctrlLoop(MasterLoop.java:87) jCommons.master.MasterLoop.access$000(MasterLoop.java:11) jCommons.master.MasterLoop$1.run(MasterLoop.java:58) java.lang.Thread.run(Thread.java:745) ID: 27, name: IORxSerialPortAccessPollLoop_IOModemAccess java.lang.Thread.sleep(Native Method) jCommons.comm.io.access.IOSerialPortAccessPoll.reciveLoop(IOSerialPortAccessPoll.java:256) jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43) java.lang.Thread.run(Thread.java:745) ID: 31, name: UsbUpdateWatchService_ctrlLoop sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489) java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678) sun.nio.fs.AbstractWatchService.take(AbstractWatchService.java:118) jCommons.update.usb.UsbUpdateWatchService.ctrlLoop(UsbUpdateWatchService.java:107) jCommons.update.usb.UsbUpdateWatchService.access$000(UsbUpdateWatchService.java:25) jCommons.update.usb.UsbUpdateWatchService$1.run(UsbUpdateWatchService.java:75) java.lang.Thread.run(Thread.java:745) ID: 25, name: IORxUdpAccessLoop_IODispatchAccess java.net.PlainDatagramSocketImpl.receive0(Native Method) java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:145) java.net.DatagramSocket.receive(DatagramSocket.java:786) jCommons.comm.io.access.IOUdpAccess.reciveLoop(IOUdpAccess.java:175) jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43) java.lang.Thread.run(Thread.java:745) ID: 2, name: Reference Handler java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) ID: 30, name: VehicleCtrl_ctrlLoop java.lang.Thread.sleep(Native Method) epis5fcc.domain.vehicle.control.VehicleCtrl.ctrlLoop(VehicleCtrl.java:74) jCommons.comm.protocol.ProtCtrlBase$1.run(ProtCtrlBase.java:24) java.lang.Thread.run(Thread.java:745) ID: 35, name: Mpg123AudioPlayer_INNER_ctrlLoop java.lang.Thread.sleep(Native Method) java.lang.Thread.sleep(Thread.java:340) java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360) epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169) epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19) epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88) java.lang.Thread.run(Thread.java:745) ID: 21, name: IORxSerialPortAccessPollLoop_IOFccAccess java.lang.Thread.sleep(Native Method) jCommons.comm.io.access.IOSerialPortAccessPoll.reciveLoop(IOSerialPortAccessPoll.java:256) jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43) java.lang.Thread.run(Thread.java:745) ID: 7, name: FileWatchdog java.lang.Thread.sleep(Native Method) org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:104) ID: 8, name: Java2D Disposer java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) sun.java2d.Disposer.run(Disposer.java:145) java.lang.Thread.run(Thread.java:745) ID: 17, name: com.google.inject.internal.util.$Finalizer java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) com.google.inject.internal.util.$Finalizer.run(Finalizer.java:114) ID: 10, name: AWT-XAWT sun.awt.X11.XToolkit.waitForEvents(Native Method) sun.awt.X11.XToolkit.run(XToolkit.java:541) sun.awt.X11.XToolkit.run(XToolkit.java:505) java.lang.Thread.run(Thread.java:745) ID: 32, name: Thread-4 sun.nio.fs.LinuxWatchService.poll(Native Method) sun.nio.fs.LinuxWatchService.access$600(LinuxWatchService.java:47) sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:311) java.lang.Thread.run(Thread.java:745) ID: 28, name: IOTxSerialPortAccessPollLoop_IOModemAccess java.lang.Thread.sleep(Native Method) jCommons.comm.io.access.IOSerialPortAccessPoll.transmitLoop(IOSerialPortAccessPoll.java:187) jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50) java.lang.Thread.run(Thread.java:745) ID: 14, name: DestroyJavaVM ID: 22, name: IOTxSerialPortAccessPollLoop_IOFccAccess java.lang.Thread.sleep(Native Method) jCommons.comm.io.access.IOSerialPortAccessPoll.transmitLoop(IOSerialPortAccessPoll.java:187) jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50) java.lang.Thread.run(Thread.java:745) ID: 19, name: TimerQueue sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) java.util.concurrent.DelayQueue.take(DelayQueue.java:220) javax.swing.TimerQueue.run(TimerQueue.java:171) java.lang.Thread.run(Thread.java:745) ID: 12, name: AWT-Shutdown java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:296) java.lang.Thread.run(Thread.java:745) ID: 23, name: IORxUdpAccessLoop_IOCityScrnAccess java.net.PlainDatagramSocketImpl.receive0(Native Method) java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:145) java.net.DatagramSocket.receive(DatagramSocket.java:786) jCommons.comm.io.access.IOUdpAccess.reciveLoop(IOUdpAccess.java:175) jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43) java.lang.Thread.run(Thread.java:745) ID: 3, name: Finalizer java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) ID: 4, name: Signal Dispatcher ID: 52, name: pool-3-thread-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) ID: 24, name: IOTxUdpAccessLoop_IOCityScrnAccess java.lang.Thread.sleep(Native Method) jCommons.comm.io.access.IOUdpAccess.transmitLoop(IOUdpAccess.java:114) jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50) java.lang.Thread.run(Thread.java:745) ID: 36, name: RemoteUpdateCtrl_ctrlLoop java.lang.Thread.sleep(Native Method) epis5fcc.domain.update.remote.RemoteUpdateCtrl.ctrlLoop(RemoteUpdateCtrl.java:94) jCommons.comm.protocol.ProtCtrlBase$1.run(ProtCtrlBase.java:24) java.lang.Thread.run(Thread.java:745) ID: 55, name: Mpg123AudioPlayer_OUTER_ctrlLoop java.lang.Thread.sleep(Native Method) java.lang.Thread.sleep(Thread.java:340) java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360) epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169) epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19) epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88) java.lang.Thread.run(Thread.java:745)
原文:https://stackoverflow.com/questions/26060209
最满意答案
两种选择:
1)如果您具有更改会话权限,请在运行
SELECT
语句之前更改NLS_DATE_FORMAT,如下所示:ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MON-RR HH24:MI';
2)如果您没有ALTER会话权限,则对每个日期字段应用转换,如下所示(下面的stmnt显示为START_POUR)
'TO_DATE(''' || TO_CHAR(START_POUR, 'DD-MON-RR HH24:MI') || ''', ''DD-MON-RR HH24:MI'')'
Two options:
1) If you have alter session privilege, change the NLS_DATE_FORMAT before running the
SELECT
statement as given below:ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MON-RR HH24:MI';
2) If you don't have ALTER session privilege then apply the transformation for each date field as given below (the stmnt below shows it for START_POUR)
'TO_DATE(''' || TO_CHAR(START_POUR, 'DD-MON-RR HH24:MI') || ''', ''DD-MON-RR HH24:MI'')'
相关问答
更多-
TCP/IP模型是一个________。[2023-10-02]
a -
下列中不属于面向对象的编程语言的是?[2022-05-30]
a -
从Select查询中提取完整时间戳(包括日期);(Extract a full timestamp (date included) from a Select query; Oracle)[2023-05-25]
两种选择: 1)如果您具有更改会话权限,请在运行SELECT语句之前更改NLS_DATE_FORMAT,如下所示: ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MON-RR HH24:MI'; 2)如果您没有ALTER会话权限,则对每个日期字段应用转换,如下所示(下面的stmnt显示为START_POUR) 'TO_DATE(''' || TO_CHAR(START_POUR, 'DD-MON-RR HH24:MI') || ''', ''DD-MON-RR HH24: ... -
编辑:修正了端点计算中的错误! 在任何给定的时刻,这将给出最近通过的半小时间隔: select case when sysdate - trunc(sysdate,'HH') > (30/1440) then trunc(sysdate,'HH') else trunc(sysdate,'HH')- (30/1440) end as start_point ,case when sysdate - trunc(sysdate,'HH') > (30/1440) then tru ...
-
您可以通过执行一个范围来指定整个一天,如下所示: WHERE bk_date >= TO_DATE('2012-03-18', 'YYYY-MM-DD') AND bk_date < TO_DATE('2012-03-19', 'YYYY-MM-DD') 更简单的你可以使用TRUNC: WHERE TRUNC(bk_date) = TO_DATE('2012-03-18', 'YYYY-MM-DD') 不带参数的TRUNC从DATE中删除小时,分钟和秒。 Answer provided by Nich ...
-
Oracle 11.1错误将Julian日期编号转换为DATE或TIMESTAMP(Oracle 11.1 bug converting Julian day number to DATE or TIMESTAMP)[2023-07-14]
在试图查看Oracle应该做什么时,请查看Oracle的文档 ,“朱利安日数是自公元前4712年1月1日以来的天数。” 该措辞确实意味着Julian 1将是自公元前4712年1月1日起的一天,换句话说是1月2日。然而,目前Julian日期计算的实施已经存在了很长时间,现有代码取决于行为。 (我知道如果在Oracle中实现的Julian定义要改变,我们就会被搞砸。)此时,自公元前4713年12月31日以来,最多只能是文档错误。 编辑在Call Interface Programmer's Guide中找到Ju ... -
Oracle时间戳舍入(Oracle timestamp rounding)[2023-03-16]
这听起来像你想要的 ROUND( (TRUNC( CAST(CR_DATE AS DATE), 'HH24') - DATE '1970-01-01') * 24 * 60 * 60, 0) AS creationDate trunc(some_date, 'HH24')将返回截断到最近小时的日期。 您的客户端默认情况下可能不会显示时间组件,但它仍然存在。 with x as ( select timestamp '2016-06-08 19:56:20.473' cr_date ... -
说出来会更有意义 WHERE date_rated > sysdate - interval '7' day 如果你想要过去168小时的数据。 你可能想要 WHERE date_rated > trunc(sysdate) - interval '7' day 如果你想要7天前的任何一天的数据,而不是关心它当前的时间。 It would make more sense to say WHERE date_rated > sysdate - interval '7' day if you want da ...
-
在Oracle中, DATE值仍然有时间。 CAST(Col2 as DATE)不会改变它。 您可以使用trunc()将date (或timestamp )的时间部分设置为00:00:00 ,您不应该比较字符串,而是日期: SELECT * FROM Sample1 WHERE trunc(col2) = TO_DATE('8 NOV 2016','DD MON YYYY'); 但我强烈建议不要使用这样的日期格式。 它取决于SQL 客户端程序的NLS设置,并可能因不同的语言设置而失败。 我更喜欢使用A ...
-
您的查询有两个问题 在BigQuery Legacy SQL中你不能在GROUP BY语句中使用表达式,而只是字段(顺便说一下,BigQuery Standard SQL中不存在这个限制) 当您执行GROUP BY时 - 您不能只是简单地选择所有字段,而是需要使用一些聚合函数,如COUNT或SUM等(对于那些不属于GROUP VY的字段) 所以你的查询看起来像下面这样: SELECT DATE(data_timestamp) AS dt, COUNT(1) AS cnt FROM [myproj:myda ...