首页 \ 问答 \ Java和Linux OS随机挂起24分钟(Linux,arm,Debian,Java 7 ARM)(Java and Linux OS hangs randomly for 24 minutes (Linux, arm, Debian, Java 7 ARM))

Java和Linux OS随机挂起24分钟(Linux,arm,Debian,Java 7 ARM)(Java and Linux OS hangs randomly for 24 minutes (Linux, arm, Debian, Java 7 ARM))

背景/背景:

我们在CompuLab CoM上运行Java应用程序:

https://compulab.co.il/products/computer-on-modules/cm-fx6/#overview

JVM版本:Oracle Java 7 ARM 1.7.0_60

OS参考:

http://www.compulab.co.il/workspace/mediawiki/index.php5/CM-FX6_Linux

该应用程序并非易事:许多线程,访问以太网(LAN),串行接口,GPRS / UMTS调制解调器,访问Internet(ppp deamon),GPS,触摸屏,数据库(SQLite),文件系统。 换句话说,广泛使用OS资源.....

我们正在观察Java应用程序(它的所有线程)和OS基本功能随机挂起。 我会说这是一个Linux内核错误,但通过杀死Java应用程序,它恢复并正常运行。

这种状态总是需要24分钟。 之后它恢复并且表现正常。 平均发生率是每24-30小时一次。

当它发生时,外部调用的事件,如通过以太网或串行接口发送到应用程序的消息被缓冲(可能是OS),所有这些事件在恢复后立即处理。

当我事先建立与设备的SSH连接时,在它发生之后连接被阻止(所有命令在恢复后被缓冲和处理 - 24分钟)或其工作,而不是:

  • 基本的OS实用程序不起作用:例如“top”
  • jstack -F不起作用,只是挂起并且不产生任何输出
  • 通过kill -9来杀死Java应用程序PID释放了操作系统,一切都开始正常运行

当它处于这种状态时,操作系统每次都表现不同。 其他发现:

  • 基于网络的基本实用程序不起作用(SSH,FTP) - 无法与其他计算机建立与OS的新连接。
  • 从我的另一台机器PING工作,直到我从设备上拔下插头以太网电缆,有时PING比停止工作
  • 有时OS系统时间也会挂起(并非总是如此),24分钟后它会持续延迟24分钟。
  • 在该状态下(始终发生),无法连接新的USB输入设备(鼠标,键盘)。

另一件奇怪的事:

触摸屏用于与用户交互(编译为内核模块的驱动程序)。 即使挂起也能正常工作。 Java应用程序(GUI Swing)可以处理按下按钮等事件,因此我可以在按钮单击处理程序后面运行一些代码。 似乎所有线程都被阻止,但Java Swing可以处理一些输入事件,我们的应用程序会先处理它们,直到它需要与已经被阻塞的线程或操作系统(按钮点击时运行bash脚本)或调用sleep方法进行交互。 比它还挂起来。 换句话说,Java应用程序“部分”挂起 - 仍然可以处理某些事情。

已经尝试过:

  • 用于JVM远程调试的工具:Java Mission Control,VisualVM。 在挂起之前也建立了连接。 在线程转储,堆转储等方面,一切似乎都没问题(我可以通过电子邮件发送)。 即使连接仍然存在,我可以在工具中看到JVM的处理器使用率降至0%。

  • jstack -F(通过SSH):不起作用,只是挂起并且不产生任何输出

  • 我试图在没有触摸屏驱动程序的情况下运行操作系统,但它仍然发生了。

  • 我试着运行两个并行的Java应用程序。 其中一个非常简单 - 只需写入日志时间戳。 他们俩都挂了。

  • 我尝试在按钮点击处理程序的同时运行System.exit(0)。 和所有线程挂起,它不起作用(挂起)

问题:

是Linux内核错误还是JVM(其ARM实现)错误?

Java(JVM)是​​否能够挂起并阻止基本的OS功能(FTP,SSH,系统时间,其他实用程序)?

当jstack -F等基本实用程序不起作用时,如何进一步诊断/调试此问题?

你有什么想法可能是这个问题的原因,为什么它总是在24分钟后恢复?

更新1:2014-07-10

最后,我设法再次“抓住”这个奇怪的状态。 以下是我的进一步发现。

基于nos建议我尝试通过ssh运行(在高级中建立):

*strace -f -p PID*

不幸的是,bash脚本命令也挂起(与jstack相同的行为)。

就用户限制(ulimit)和操作系统资源而言,下面我报告刚从系统上次挂起后恢复的数据 。 在该州,它已经运行了24小时 ,我可以确认这些数字在长期运行期间保持大致相同(在运行期间没有随机偷看)。 从我的角度来看,它们没问题,应用程序也没有以任何方式踩过任何资源或其他限制。

Java当前堆

使用:18 MB,免费:12 MB,总计:30 MB,最大:230 MB

Java堆

 root@cm-debian:~# /usr/lib/jvm/jdk1.7.0_60/bin/jmap -heap 3242
Attaching to process ID 3242, please wait...
Debugger attached successfully.
Client compiler detected.
JVM version is 24.60-b09

using thread-local object allocation.
Mark Sweep Compact GC

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 249561088 (238.0MB)
   NewSize          = 1048576 (1.0MB)
   MaxNewSize       = 4294836224 (4095.875MB)
   OldSize          = 4194304 (4.0MB)
   NewRatio         = 2
   SurvivorRatio    = 8
   PermSize         = 12582912 (12.0MB)
   MaxPermSize      = 67108864 (64.0MB)
   G1HeapRegionSize = 0 (0.0MB)

Heap Usage:
New Generation (Eden + 1 Survivor Space):
   capacity = 10092544 (9.625MB)
   used     = 6772088 (6.458366394042969MB)
   free     = 3320456 (3.1666336059570312MB)
   67.09991058745942% used
Eden Space:
   capacity = 9043968 (8.625MB)
   used     = 6620336 (6.3136444091796875MB)
   free     = 2423632 (2.3113555908203125MB)
   73.2016743093297% used
From Space:
   capacity = 1048576 (1.0MB)
   used     = 151752 (0.14472198486328125MB)
   free     = 896824 (0.8552780151367188MB)
   14.472198486328125% used
To Space:
   capacity = 1048576 (1.0MB)
   used     = 0 (0.0MB)
   free     = 1048576 (1.0MB)
   0.0% used
tenured generation:
   capacity = 22134784 (21.109375MB)
   used     = 17650936 (16.83324432373047MB)
   free     = 4483848 (4.276130676269531MB)
   79.7429782915433% used
Perm Generation:
   capacity = 19136512 (18.25MB)
   used     = 19023016 (18.141761779785156MB)
   free     = 113496 (0.10823822021484375MB)
   99.40691386183647% used

9597 interned Strings occupying 729344 bytes.

最佳

top - 11:41:29 up 21:59,  2 users,  load average: 1.51, 1.25, 1.22
Tasks:  93 total,   1 running,  92 sleeping,   0 stopped,   0 zombie
Cpu(s):  9.4%us,  8.0%sy,  0.0%ni, 82.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    966780k total,   273080k used,   693700k free,    27216k buffers
Swap:        0k total,        0k used,        0k free,   126352k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                             
 3242 root      20   0  398m  79m  11m S 23.6  8.4 346:16.82 java                                                                                
 3889 root      20   0  2804 1096  848 R  5.5  0.1   0:00.07 top                                                                                 
    1 root      20   0  2124  688  596 S  0.0  0.1   0:02.92 init                                                                                
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.03 kthreadd                                                                            
    3 root      20   0     0    0    0 S  0.0  0.0   0:14.32 ksoftirqd/0                                                                         
    5 root      20   0     0    0    0 S  0.0  0.0   0:00.14 kworker/u:0                                                                         
    6 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0                                                                         
    7 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 khelper 

java限制

root@cm-debian:~# java -XX:+PrintFlagsFinal -version | grep -iE 'HeapSize|PermSize|ThreadStackSize'
    uintx AdaptivePermSizeWeight                    = 20              {product}           
     intx CompilerThreadStackSize                   = 0               {pd product}        
    uintx ErgoHeapSizeLimit                         = 0               {product}           
    uintx HeapSizePerGCThread                       = 67108864        {product}           
    uintx InitialHeapSize                          := 15468480        {product}           
    uintx LargePageHeapSizeThreshold                = 134217728       {product}           
    uintx MaxHeapSize                              := 249561088       {product}           
    uintx MaxPermSize                               = 67108864        {pd product}        
    uintx PermSize                                  = 12582912        {pd product}        
     intx ThreadStackSize                           = 320             {pd product}        
     intx VMThreadStackSize                         = 512             {pd product}        
java version "1.7.0_60"
Java(TM) SE Runtime Environment (build 1.7.0_60-b19)
Java HotSpot(TM) Client VM (build 24.60-b09, mixed mode)

过程限制

root@cm-debian:~# cat /proc/3242/limits
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             unlimited            unlimited            processes 
Max open files            8192                 8192                 files     
Max locked memory         65536                65536                bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       16382                16382                signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us 

系统内存信息

root@cm-debian:~# cat /proc/meminfo
MemTotal:         966780 kB
MemFree:          694312 kB
Buffers:           27384 kB
Cached:           126364 kB
SwapCached:            0 kB
Active:           140748 kB
Inactive:         107684 kB
Active(anon):      94992 kB
Inactive(anon):     2064 kB
Active(file):      45756 kB
Inactive(file):   105620 kB
Unevictable:           0 kB
Mlocked:               0 kB
HighTotal:        524288 kB
HighFree:         301088 kB
LowTotal:         442492 kB
LowFree:          393224 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:         94692 kB
Mapped:            21220 kB
Shmem:              2376 kB
Slab:              13268 kB
SReclaimable:       5284 kB
SUnreclaim:         7984 kB
KernelStack:         960 kB
PageTables:          980 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      483388 kB
Committed_AS:     137260 kB
VmallocTotal:     286720 kB
VmallocUsed:        2928 kB
VmallocChunk:     283040 kB


root@cm-debian:~# vmstat -s
       966780 K total memory
       272468 K used memory
       140776 K active memory
       107712 K inactive memory
       694312 K free memory
        27392 K buffer memory
       126404 K swap cache
            0 K total swap
            0 K used swap
            0 K free swap
       726963 non-nice user cpu ticks
            0 nice user cpu ticks
       621187 system cpu ticks
      6371123 idle cpu ticks
         3683 IO-wait cpu ticks
          324 IRQ cpu ticks
         2146 softirq cpu ticks
            0 stolen cpu ticks
       130871 pages paged in
        97520 pages paged out
            0 pages swapped in
            0 pages swapped out
    293822206 interrupts
    494034482 CPU context switches
   1412595732 boot time
         3916 forks

线程

root@cm-debian:~# cat /proc/sys/kernel/pid_max
32768

root@cm-debian:~# cat /proc/sys/kernel/threads-max
15102

root@cm-debian:~# cat /proc/sys/vm/max_map_count
65530


root@cm-debian:~# ls -l /proc/3242/task/ | wc -l
33

root@cm-debian:~# ps huH p 3242 | wc -l
32

root@cm-debian:~# grep -s '^Threads' /proc/[0-9]*/status | awk '{ sum += $2; } END { print sum; }'
122

打开文件/文件描述符

root@cm-debian:~# ls -l /proc/3242/fd | wc -l
81

更新2:2014-13-10

这次我在操作系统挂起时记录了所有Java线程堆栈跟踪(如前所述,触摸屏及其事件仍然有效,因此我根据UI按钮处理程序将堆栈跟踪写入日志文件)。

从我的角度来看, 所有线程都处于“正确”状态 (休眠,等待UDP数据报等)。 很明显,挂起不是由Java应用程序SW操作引起的,这将花费24分钟

10:49:42,293> [INFO ] THREAD stack traces: 
****************************************

ID: 56, name: Mpg123AudioPlayer_PASSENGER_ctrlLoop
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19)
epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88)
java.lang.Thread.run(Thread.java:745)

ID: 11, name: AWT-EventQueue-0
java.lang.Thread.getStackTrace(Thread.java:1589)
epis5fcc.domain.debug.ThreadStackTracesLogger.log(ThreadStackTracesLogger.java:30)
epis5fcc.ui.settings.FccRegistryScreen$7.actionPerformed(FccRegistryScreen.java:303)
javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2018)
javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2341)
javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
java.awt.Component.processMouseEvent(Component.java:6516)
javax.swing.JComponent.processMouseEvent(JComponent.java:3320)
java.awt.Component.processEvent(Component.java:6281)
java.awt.Container.processEvent(Container.java:2229)
java.awt.Component.dispatchEventImpl(Component.java:4872)
java.awt.Container.dispatchEventImpl(Container.java:2287)
java.awt.Component.dispatchEvent(Component.java:4698)
java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4832)
java.awt.LightweightDispatcher.processMouseEvent(Container.java:4492)
java.awt.LightweightDispatcher.dispatchEvent(Container.java:4422)
java.awt.Container.dispatchEventImpl(Container.java:2273)
java.awt.Window.dispatchEventImpl(Window.java:2719)
java.awt.Component.dispatchEvent(Component.java:4698)
java.awt.EventQueue.dispatchEventImpl(EventQueue.java:735)
java.awt.EventQueue.access$200(EventQueue.java:103)
java.awt.EventQueue$3.run(EventQueue.java:694)
java.awt.EventQueue$3.run(EventQueue.java:692)
java.security.AccessController.doPrivileged(Native Method)
java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:87)
java.awt.EventQueue$4.run(EventQueue.java:708)
java.awt.EventQueue$4.run(EventQueue.java:706)
java.security.AccessController.doPrivileged(Native Method)
java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
java.awt.EventQueue.dispatchEvent(EventQueue.java:705)
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138)
java.awt.EventDispatchThread.run(EventDispatchThread.java:91)

ID: 34, name: Mpg123AudioPlayer_DRIVER_ctrlLoop
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19)
epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88)
java.lang.Thread.run(Thread.java:745)

ID: 26, name: IOTxUdpAccessLoop_IODispatchAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOUdpAccess.transmitLoop(IOUdpAccess.java:114)
jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50)
java.lang.Thread.run(Thread.java:745)

ID: 29, name: MasterLoop_main
java.lang.Thread.sleep(Native Method)
jCommons.master.MasterLoop.ctrlLoop(MasterLoop.java:87)
jCommons.master.MasterLoop.access$000(MasterLoop.java:11)
jCommons.master.MasterLoop$1.run(MasterLoop.java:58)
java.lang.Thread.run(Thread.java:745)

ID: 27, name: IORxSerialPortAccessPollLoop_IOModemAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOSerialPortAccessPoll.reciveLoop(IOSerialPortAccessPoll.java:256)
jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43)
java.lang.Thread.run(Thread.java:745)

ID: 31, name: UsbUpdateWatchService_ctrlLoop
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
sun.nio.fs.AbstractWatchService.take(AbstractWatchService.java:118)
jCommons.update.usb.UsbUpdateWatchService.ctrlLoop(UsbUpdateWatchService.java:107)
jCommons.update.usb.UsbUpdateWatchService.access$000(UsbUpdateWatchService.java:25)
jCommons.update.usb.UsbUpdateWatchService$1.run(UsbUpdateWatchService.java:75)
java.lang.Thread.run(Thread.java:745)

ID: 25, name: IORxUdpAccessLoop_IODispatchAccess
java.net.PlainDatagramSocketImpl.receive0(Native Method)
java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:145)
java.net.DatagramSocket.receive(DatagramSocket.java:786)
jCommons.comm.io.access.IOUdpAccess.reciveLoop(IOUdpAccess.java:175)
jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43)
java.lang.Thread.run(Thread.java:745)

ID: 2, name: Reference Handler
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:503)
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)

ID: 30, name: VehicleCtrl_ctrlLoop
java.lang.Thread.sleep(Native Method)
epis5fcc.domain.vehicle.control.VehicleCtrl.ctrlLoop(VehicleCtrl.java:74)
jCommons.comm.protocol.ProtCtrlBase$1.run(ProtCtrlBase.java:24)
java.lang.Thread.run(Thread.java:745)

ID: 35, name: Mpg123AudioPlayer_INNER_ctrlLoop
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19)
epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88)
java.lang.Thread.run(Thread.java:745)


ID: 21, name: IORxSerialPortAccessPollLoop_IOFccAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOSerialPortAccessPoll.reciveLoop(IOSerialPortAccessPoll.java:256)
jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43)
java.lang.Thread.run(Thread.java:745)

ID: 7, name: FileWatchdog
java.lang.Thread.sleep(Native Method)
org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:104)


ID: 8, name: Java2D Disposer
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
sun.java2d.Disposer.run(Disposer.java:145)
java.lang.Thread.run(Thread.java:745)

ID: 17, name: com.google.inject.internal.util.$Finalizer
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
com.google.inject.internal.util.$Finalizer.run(Finalizer.java:114)

ID: 10, name: AWT-XAWT
sun.awt.X11.XToolkit.waitForEvents(Native Method)
sun.awt.X11.XToolkit.run(XToolkit.java:541)
sun.awt.X11.XToolkit.run(XToolkit.java:505)
java.lang.Thread.run(Thread.java:745)

ID: 32, name: Thread-4
sun.nio.fs.LinuxWatchService.poll(Native Method)
sun.nio.fs.LinuxWatchService.access$600(LinuxWatchService.java:47)
sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:311)
java.lang.Thread.run(Thread.java:745)

ID: 28, name: IOTxSerialPortAccessPollLoop_IOModemAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOSerialPortAccessPoll.transmitLoop(IOSerialPortAccessPoll.java:187)
jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50)
java.lang.Thread.run(Thread.java:745)

ID: 14, name: DestroyJavaVM

ID: 22, name: IOTxSerialPortAccessPollLoop_IOFccAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOSerialPortAccessPoll.transmitLoop(IOSerialPortAccessPoll.java:187)
jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50)
java.lang.Thread.run(Thread.java:745)

ID: 19, name: TimerQueue
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
java.util.concurrent.DelayQueue.take(DelayQueue.java:220)
javax.swing.TimerQueue.run(TimerQueue.java:171)
java.lang.Thread.run(Thread.java:745)

ID: 12, name: AWT-Shutdown
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:503)
sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:296)
java.lang.Thread.run(Thread.java:745)

ID: 23, name: IORxUdpAccessLoop_IOCityScrnAccess
java.net.PlainDatagramSocketImpl.receive0(Native Method)
java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:145)
java.net.DatagramSocket.receive(DatagramSocket.java:786)
jCommons.comm.io.access.IOUdpAccess.reciveLoop(IOUdpAccess.java:175)
jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43)
java.lang.Thread.run(Thread.java:745)

ID: 3, name: Finalizer
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

ID: 4, name: Signal Dispatcher

ID: 52, name: pool-3-thread-1
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)

ID: 24, name: IOTxUdpAccessLoop_IOCityScrnAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOUdpAccess.transmitLoop(IOUdpAccess.java:114)
jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50)
java.lang.Thread.run(Thread.java:745)

ID: 36, name: RemoteUpdateCtrl_ctrlLoop
java.lang.Thread.sleep(Native Method)
epis5fcc.domain.update.remote.RemoteUpdateCtrl.ctrlLoop(RemoteUpdateCtrl.java:94)
jCommons.comm.protocol.ProtCtrlBase$1.run(ProtCtrlBase.java:24)
java.lang.Thread.run(Thread.java:745)

ID: 55, name: Mpg123AudioPlayer_OUTER_ctrlLoop
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19)
epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88)
java.lang.Thread.run(Thread.java:745)

Background/context:

We are running Java application on one of CompuLab CoM:

https://compulab.co.il/products/computer-on-modules/cm-fx6/#overview

JVM version: Oracle Java 7 ARM 1.7.0_60

OS reference:

http://www.compulab.co.il/workspace/mediawiki/index.php5/CM-FX6_Linux

The application is not trivial: lots of threads, access to Ethernet (LAN), serial interface, GPRS/UMTS modem, access to Internet (ppp deamon), GPS, touch screen, database (SQLite), file system. In other words use OS resources extensively.....

We are observing that Java application (all of its threads) and OS basic functionality randomly hangs. I would say it is a Linux kernel bug but by killing the Java application it recovers and operates normally.

This state always takes exactly 24 minutes. Afterwards it recovers and behaves normally. Average rate of occurrence is once per 24-30 hours.

When it happens, externally invoked events like messages sent to application via Ethernet or serial interface are buffered (by OS probably) and all of them are processed immediately after it recovers.

When I establish SSH connection to device in advance, after it happens the connection is either blocked (all command are buffered and processed after it recovers - 24 mintes) or its working, than:

  • basic OS utilities does not work: "top" for example
  • jstack -F does not work, just hangs and does not produce any output
  • killing Java application by kill -9 PID released the OS and everything starts to operate normally

While it is in this state, the OS each time behaves differently. Other findings:

  • Basic network based utilities does not work (SSH, FTP) – can not establish new connection to OS from another machine.
  • PING from another machine does work until I unplug an plug Ethernet cable from device, sometimes PING than stops working
  • Sometimes OS system time hangs as well (not always), after 24 minutes it continues delayed for 24 minutes.
  • New USB input devices (mouse, keyboard) can not be connected while in that state (happens always).

Another strange thing:

A touch screen is used for interaction with a user (driver compiled as kernel module). And it works even while it is hung. Java application (GUI Swing) can handle events like pressing button so I can run some code behind button click handler. It seems like all threads are blocked but Java Swing can handle some input events and our application precesses them until it needs to interact with already blocked threads or OS (run bash script on button click) or call sleep method. Than it hangs as well. In other words, the Java application is hung ”partially” - can still handle something.

Already tried:

  • Tools for JVM remote debugging: Java Mission Control, VisualVM. Connection was also established before it hung. Everything seemed OK in terms of thread dump, heap dump etc. (I can send by e-mail). Even the connection remained and I could see in thees tools that processor usage dropped to 0 % for JVM.

  • jstack -F (via SSH): does not work, just hangs and does not produce any output

  • I tried to run OS without the driver for touch screen and it still happened.

  • I tried to run two parallel Java application. One of them was very simple – just writing to log timestamps. And both of them hung.

  • I tried to run System.exit(0) in terms of button click handler while app. and all threads hung and it does not worked (hung as well)

Questions:

Is it Linux kernel bug or JVM (its ARM implementation) bug?

Is Java (JVM) able to hang and block basic OS functionality (FTP, SSH, system time, other utilities)?

How can I further diagnose/debug this issue when basic utilities like jstack -F does not work?

Do you have any ideas what could be the cause of this issue and why it always recovers exactly after 24 minutes?

Update 1: 2014-07-10

Finally I manage to “catch” this weird state again. Here are my further findings.

Based on nos suggestion I tried run via ssh (established in advanced):

*strace -f -p PID*

Unfortunately the bash script command hung as well (same behavior like with jstack).

As far as the user limit (ulimit) and OS resources are concerned, bellow I report figures taken just after the system recovered from last hung. At that state it had been running for 24 hours and I can confirm that those figures remain roughly the same during long-term operation (no random peeks during operation). From my point of view, they are ok and application is not stepping over any resource or other limit in any way.

Java current heap

Used: 18 MB, Free: 12 MB, Total: 30 MB, Max: 230 MB

Java heap

 root@cm-debian:~# /usr/lib/jvm/jdk1.7.0_60/bin/jmap -heap 3242
Attaching to process ID 3242, please wait...
Debugger attached successfully.
Client compiler detected.
JVM version is 24.60-b09

using thread-local object allocation.
Mark Sweep Compact GC

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 249561088 (238.0MB)
   NewSize          = 1048576 (1.0MB)
   MaxNewSize       = 4294836224 (4095.875MB)
   OldSize          = 4194304 (4.0MB)
   NewRatio         = 2
   SurvivorRatio    = 8
   PermSize         = 12582912 (12.0MB)
   MaxPermSize      = 67108864 (64.0MB)
   G1HeapRegionSize = 0 (0.0MB)

Heap Usage:
New Generation (Eden + 1 Survivor Space):
   capacity = 10092544 (9.625MB)
   used     = 6772088 (6.458366394042969MB)
   free     = 3320456 (3.1666336059570312MB)
   67.09991058745942% used
Eden Space:
   capacity = 9043968 (8.625MB)
   used     = 6620336 (6.3136444091796875MB)
   free     = 2423632 (2.3113555908203125MB)
   73.2016743093297% used
From Space:
   capacity = 1048576 (1.0MB)
   used     = 151752 (0.14472198486328125MB)
   free     = 896824 (0.8552780151367188MB)
   14.472198486328125% used
To Space:
   capacity = 1048576 (1.0MB)
   used     = 0 (0.0MB)
   free     = 1048576 (1.0MB)
   0.0% used
tenured generation:
   capacity = 22134784 (21.109375MB)
   used     = 17650936 (16.83324432373047MB)
   free     = 4483848 (4.276130676269531MB)
   79.7429782915433% used
Perm Generation:
   capacity = 19136512 (18.25MB)
   used     = 19023016 (18.141761779785156MB)
   free     = 113496 (0.10823822021484375MB)
   99.40691386183647% used

9597 interned Strings occupying 729344 bytes.

top

top - 11:41:29 up 21:59,  2 users,  load average: 1.51, 1.25, 1.22
Tasks:  93 total,   1 running,  92 sleeping,   0 stopped,   0 zombie
Cpu(s):  9.4%us,  8.0%sy,  0.0%ni, 82.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    966780k total,   273080k used,   693700k free,    27216k buffers
Swap:        0k total,        0k used,        0k free,   126352k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                             
 3242 root      20   0  398m  79m  11m S 23.6  8.4 346:16.82 java                                                                                
 3889 root      20   0  2804 1096  848 R  5.5  0.1   0:00.07 top                                                                                 
    1 root      20   0  2124  688  596 S  0.0  0.1   0:02.92 init                                                                                
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.03 kthreadd                                                                            
    3 root      20   0     0    0    0 S  0.0  0.0   0:14.32 ksoftirqd/0                                                                         
    5 root      20   0     0    0    0 S  0.0  0.0   0:00.14 kworker/u:0                                                                         
    6 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0                                                                         
    7 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 khelper 

java limits

root@cm-debian:~# java -XX:+PrintFlagsFinal -version | grep -iE 'HeapSize|PermSize|ThreadStackSize'
    uintx AdaptivePermSizeWeight                    = 20              {product}           
     intx CompilerThreadStackSize                   = 0               {pd product}        
    uintx ErgoHeapSizeLimit                         = 0               {product}           
    uintx HeapSizePerGCThread                       = 67108864        {product}           
    uintx InitialHeapSize                          := 15468480        {product}           
    uintx LargePageHeapSizeThreshold                = 134217728       {product}           
    uintx MaxHeapSize                              := 249561088       {product}           
    uintx MaxPermSize                               = 67108864        {pd product}        
    uintx PermSize                                  = 12582912        {pd product}        
     intx ThreadStackSize                           = 320             {pd product}        
     intx VMThreadStackSize                         = 512             {pd product}        
java version "1.7.0_60"
Java(TM) SE Runtime Environment (build 1.7.0_60-b19)
Java HotSpot(TM) Client VM (build 24.60-b09, mixed mode)

process limits

root@cm-debian:~# cat /proc/3242/limits
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             unlimited            unlimited            processes 
Max open files            8192                 8192                 files     
Max locked memory         65536                65536                bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       16382                16382                signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us 

system memory info

root@cm-debian:~# cat /proc/meminfo
MemTotal:         966780 kB
MemFree:          694312 kB
Buffers:           27384 kB
Cached:           126364 kB
SwapCached:            0 kB
Active:           140748 kB
Inactive:         107684 kB
Active(anon):      94992 kB
Inactive(anon):     2064 kB
Active(file):      45756 kB
Inactive(file):   105620 kB
Unevictable:           0 kB
Mlocked:               0 kB
HighTotal:        524288 kB
HighFree:         301088 kB
LowTotal:         442492 kB
LowFree:          393224 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:         94692 kB
Mapped:            21220 kB
Shmem:              2376 kB
Slab:              13268 kB
SReclaimable:       5284 kB
SUnreclaim:         7984 kB
KernelStack:         960 kB
PageTables:          980 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      483388 kB
Committed_AS:     137260 kB
VmallocTotal:     286720 kB
VmallocUsed:        2928 kB
VmallocChunk:     283040 kB


root@cm-debian:~# vmstat -s
       966780 K total memory
       272468 K used memory
       140776 K active memory
       107712 K inactive memory
       694312 K free memory
        27392 K buffer memory
       126404 K swap cache
            0 K total swap
            0 K used swap
            0 K free swap
       726963 non-nice user cpu ticks
            0 nice user cpu ticks
       621187 system cpu ticks
      6371123 idle cpu ticks
         3683 IO-wait cpu ticks
          324 IRQ cpu ticks
         2146 softirq cpu ticks
            0 stolen cpu ticks
       130871 pages paged in
        97520 pages paged out
            0 pages swapped in
            0 pages swapped out
    293822206 interrupts
    494034482 CPU context switches
   1412595732 boot time
         3916 forks

threads

root@cm-debian:~# cat /proc/sys/kernel/pid_max
32768

root@cm-debian:~# cat /proc/sys/kernel/threads-max
15102

root@cm-debian:~# cat /proc/sys/vm/max_map_count
65530


root@cm-debian:~# ls -l /proc/3242/task/ | wc -l
33

root@cm-debian:~# ps huH p 3242 | wc -l
32

root@cm-debian:~# grep -s '^Threads' /proc/[0-9]*/status | awk '{ sum += $2; } END { print sum; }'
122

open files / file descriptors

root@cm-debian:~# ls -l /proc/3242/fd | wc -l
81

Update 2: 2014-13-10

This time I logged all Java threads stack traces while the OS was hung (as I stated previously, the touch screen and its events still works so I wrote stack traces to log file in terms of UI button handler).

From my point of view, all threads are in “correct” state (sleeping, waiting for UDP datagram etc..) and it is obvious that the hang is not caused by a Java application SW operation which would took 24 minutes.

10:49:42,293> [INFO ] THREAD stack traces: 
****************************************

ID: 56, name: Mpg123AudioPlayer_PASSENGER_ctrlLoop
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19)
epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88)
java.lang.Thread.run(Thread.java:745)

ID: 11, name: AWT-EventQueue-0
java.lang.Thread.getStackTrace(Thread.java:1589)
epis5fcc.domain.debug.ThreadStackTracesLogger.log(ThreadStackTracesLogger.java:30)
epis5fcc.ui.settings.FccRegistryScreen$7.actionPerformed(FccRegistryScreen.java:303)
javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2018)
javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2341)
javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
java.awt.Component.processMouseEvent(Component.java:6516)
javax.swing.JComponent.processMouseEvent(JComponent.java:3320)
java.awt.Component.processEvent(Component.java:6281)
java.awt.Container.processEvent(Container.java:2229)
java.awt.Component.dispatchEventImpl(Component.java:4872)
java.awt.Container.dispatchEventImpl(Container.java:2287)
java.awt.Component.dispatchEvent(Component.java:4698)
java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4832)
java.awt.LightweightDispatcher.processMouseEvent(Container.java:4492)
java.awt.LightweightDispatcher.dispatchEvent(Container.java:4422)
java.awt.Container.dispatchEventImpl(Container.java:2273)
java.awt.Window.dispatchEventImpl(Window.java:2719)
java.awt.Component.dispatchEvent(Component.java:4698)
java.awt.EventQueue.dispatchEventImpl(EventQueue.java:735)
java.awt.EventQueue.access$200(EventQueue.java:103)
java.awt.EventQueue$3.run(EventQueue.java:694)
java.awt.EventQueue$3.run(EventQueue.java:692)
java.security.AccessController.doPrivileged(Native Method)
java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:87)
java.awt.EventQueue$4.run(EventQueue.java:708)
java.awt.EventQueue$4.run(EventQueue.java:706)
java.security.AccessController.doPrivileged(Native Method)
java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
java.awt.EventQueue.dispatchEvent(EventQueue.java:705)
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138)
java.awt.EventDispatchThread.run(EventDispatchThread.java:91)

ID: 34, name: Mpg123AudioPlayer_DRIVER_ctrlLoop
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19)
epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88)
java.lang.Thread.run(Thread.java:745)

ID: 26, name: IOTxUdpAccessLoop_IODispatchAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOUdpAccess.transmitLoop(IOUdpAccess.java:114)
jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50)
java.lang.Thread.run(Thread.java:745)

ID: 29, name: MasterLoop_main
java.lang.Thread.sleep(Native Method)
jCommons.master.MasterLoop.ctrlLoop(MasterLoop.java:87)
jCommons.master.MasterLoop.access$000(MasterLoop.java:11)
jCommons.master.MasterLoop$1.run(MasterLoop.java:58)
java.lang.Thread.run(Thread.java:745)

ID: 27, name: IORxSerialPortAccessPollLoop_IOModemAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOSerialPortAccessPoll.reciveLoop(IOSerialPortAccessPoll.java:256)
jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43)
java.lang.Thread.run(Thread.java:745)

ID: 31, name: UsbUpdateWatchService_ctrlLoop
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
sun.nio.fs.AbstractWatchService.take(AbstractWatchService.java:118)
jCommons.update.usb.UsbUpdateWatchService.ctrlLoop(UsbUpdateWatchService.java:107)
jCommons.update.usb.UsbUpdateWatchService.access$000(UsbUpdateWatchService.java:25)
jCommons.update.usb.UsbUpdateWatchService$1.run(UsbUpdateWatchService.java:75)
java.lang.Thread.run(Thread.java:745)

ID: 25, name: IORxUdpAccessLoop_IODispatchAccess
java.net.PlainDatagramSocketImpl.receive0(Native Method)
java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:145)
java.net.DatagramSocket.receive(DatagramSocket.java:786)
jCommons.comm.io.access.IOUdpAccess.reciveLoop(IOUdpAccess.java:175)
jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43)
java.lang.Thread.run(Thread.java:745)

ID: 2, name: Reference Handler
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:503)
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)

ID: 30, name: VehicleCtrl_ctrlLoop
java.lang.Thread.sleep(Native Method)
epis5fcc.domain.vehicle.control.VehicleCtrl.ctrlLoop(VehicleCtrl.java:74)
jCommons.comm.protocol.ProtCtrlBase$1.run(ProtCtrlBase.java:24)
java.lang.Thread.run(Thread.java:745)

ID: 35, name: Mpg123AudioPlayer_INNER_ctrlLoop
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19)
epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88)
java.lang.Thread.run(Thread.java:745)


ID: 21, name: IORxSerialPortAccessPollLoop_IOFccAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOSerialPortAccessPoll.reciveLoop(IOSerialPortAccessPoll.java:256)
jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43)
java.lang.Thread.run(Thread.java:745)

ID: 7, name: FileWatchdog
java.lang.Thread.sleep(Native Method)
org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:104)


ID: 8, name: Java2D Disposer
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
sun.java2d.Disposer.run(Disposer.java:145)
java.lang.Thread.run(Thread.java:745)

ID: 17, name: com.google.inject.internal.util.$Finalizer
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
com.google.inject.internal.util.$Finalizer.run(Finalizer.java:114)

ID: 10, name: AWT-XAWT
sun.awt.X11.XToolkit.waitForEvents(Native Method)
sun.awt.X11.XToolkit.run(XToolkit.java:541)
sun.awt.X11.XToolkit.run(XToolkit.java:505)
java.lang.Thread.run(Thread.java:745)

ID: 32, name: Thread-4
sun.nio.fs.LinuxWatchService.poll(Native Method)
sun.nio.fs.LinuxWatchService.access$600(LinuxWatchService.java:47)
sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:311)
java.lang.Thread.run(Thread.java:745)

ID: 28, name: IOTxSerialPortAccessPollLoop_IOModemAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOSerialPortAccessPoll.transmitLoop(IOSerialPortAccessPoll.java:187)
jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50)
java.lang.Thread.run(Thread.java:745)

ID: 14, name: DestroyJavaVM

ID: 22, name: IOTxSerialPortAccessPollLoop_IOFccAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOSerialPortAccessPoll.transmitLoop(IOSerialPortAccessPoll.java:187)
jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50)
java.lang.Thread.run(Thread.java:745)

ID: 19, name: TimerQueue
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
java.util.concurrent.DelayQueue.take(DelayQueue.java:220)
javax.swing.TimerQueue.run(TimerQueue.java:171)
java.lang.Thread.run(Thread.java:745)

ID: 12, name: AWT-Shutdown
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:503)
sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:296)
java.lang.Thread.run(Thread.java:745)

ID: 23, name: IORxUdpAccessLoop_IOCityScrnAccess
java.net.PlainDatagramSocketImpl.receive0(Native Method)
java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:145)
java.net.DatagramSocket.receive(DatagramSocket.java:786)
jCommons.comm.io.access.IOUdpAccess.reciveLoop(IOUdpAccess.java:175)
jCommons.comm.io.access.IOAccessBase$1.run(IOAccessBase.java:43)
java.lang.Thread.run(Thread.java:745)

ID: 3, name: Finalizer
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

ID: 4, name: Signal Dispatcher

ID: 52, name: pool-3-thread-1
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)

ID: 24, name: IOTxUdpAccessLoop_IOCityScrnAccess
java.lang.Thread.sleep(Native Method)
jCommons.comm.io.access.IOUdpAccess.transmitLoop(IOUdpAccess.java:114)
jCommons.comm.io.access.IOAccessBase$2.run(IOAccessBase.java:50)
java.lang.Thread.run(Thread.java:745)

ID: 36, name: RemoteUpdateCtrl_ctrlLoop
java.lang.Thread.sleep(Native Method)
epis5fcc.domain.update.remote.RemoteUpdateCtrl.ctrlLoop(RemoteUpdateCtrl.java:94)
jCommons.comm.protocol.ProtCtrlBase$1.run(ProtCtrlBase.java:24)
java.lang.Thread.run(Thread.java:745)

ID: 55, name: Mpg123AudioPlayer_OUTER_ctrlLoop
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:360)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.ctrlLoop(MpgAudioOutputPlayer.java:169)
epis5fcc.audio.mpg.MpgAudioOutputPlayer.access$000(MpgAudioOutputPlayer.java:19)
epis5fcc.audio.mpg.MpgAudioOutputPlayer$1.run(MpgAudioOutputPlayer.java:88)
java.lang.Thread.run(Thread.java:745)

原文:https://stackoverflow.com/questions/26060209
更新时间:2023-03-16 06:03

最满意答案

两种选择:

1)如果您具有更改会话权限,请在运行SELECT语句之前更改NLS_DATE_FORMAT,如下所示:

ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MON-RR HH24:MI';

2)如果您没有ALTER会话权限,则对每个日期字段应用转换,如下所示(下面的stmnt显示为START_POUR)

'TO_DATE(''' || TO_CHAR(START_POUR, 'DD-MON-RR HH24:MI') || ''', ''DD-MON-RR HH24:MI'')'

Two options:

1) If you have alter session privilege, change the NLS_DATE_FORMAT before running the SELECT statement as given below:

ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MON-RR HH24:MI';

2) If you don't have ALTER session privilege then apply the transformation for each date field as given below (the stmnt below shows it for START_POUR)

'TO_DATE(''' || TO_CHAR(START_POUR, 'DD-MON-RR HH24:MI') || ''', ''DD-MON-RR HH24:MI'')'

相关问答

更多
  • 两种选择: 1)如果您具有更改会话权限,请在运行SELECT语句之前更改NLS_DATE_FORMAT,如下所示: ALTER SESSION SET NLS_DATE_FORMAT = 'DD-MON-RR HH24:MI'; 2)如果您没有ALTER会话权限,则对每个日期字段应用转换,如下所示(下面的stmnt显示为START_POUR) 'TO_DATE(''' || TO_CHAR(START_POUR, 'DD-MON-RR HH24:MI') || ''', ''DD-MON-RR HH24: ...
  • 编辑:修正了端点计算中的错误! 在任何给定的时刻,这将给出最近通过的半小时间隔: select case when sysdate - trunc(sysdate,'HH') > (30/1440) then trunc(sysdate,'HH') else trunc(sysdate,'HH')- (30/1440) end as start_point ,case when sysdate - trunc(sysdate,'HH') > (30/1440) then tru ...
  • 您可以通过执行一个范围来指定整个一天,如下所示: WHERE bk_date >= TO_DATE('2012-03-18', 'YYYY-MM-DD') AND bk_date < TO_DATE('2012-03-19', 'YYYY-MM-DD') 更简单的你可以使用TRUNC: WHERE TRUNC(bk_date) = TO_DATE('2012-03-18', 'YYYY-MM-DD') 不带参数的TRUNC从DATE中删除小时,分钟和秒。 Answer provided by Nich ...
  • 在试图查看Oracle应该做什么时,请查看Oracle的文档 ,“朱利安日数是自公元前4712年1月1日以来的天数。” 该措辞确实意味着Julian 1将是自公元前4712年1月1日起的一天,换句话说是1月2日。然而,目前Julian日期计算的实施已经存在了很长时间,现有代码取决于行为。 (我知道如果在Oracle中实现的Julian定义要改变,我们就会被搞砸。)此时,自公元前4713年12月31日以来,最多只能是文档错误。 编辑在Call Interface Programmer's Guide中找到Ju ...
  • 这听起来像你想要的 ROUND( (TRUNC( CAST(CR_DATE AS DATE), 'HH24') - DATE '1970-01-01') * 24 * 60 * 60, 0) AS creationDate trunc(some_date, 'HH24')将返回截断到最近小时的日期。 您的客户端默认情况下可能不会显示时间组件,但它仍然存在。 with x as ( select timestamp '2016-06-08 19:56:20.473' cr_date ...
  • 说出来会更有意义 WHERE date_rated > sysdate - interval '7' day 如果你想要过去168小时的数据。 你可能想要 WHERE date_rated > trunc(sysdate) - interval '7' day 如果你想要7天前的任何一天的数据,而不是关心它当前的时间。 It would make more sense to say WHERE date_rated > sysdate - interval '7' day if you want da ...
  • 在Oracle中, DATE值仍然有时间。 CAST(Col2 as DATE)不会改变它。 您可以使用trunc()将date (或timestamp )的时间部分设置为00:00:00 ,您不应该比较字符串,而是日期: SELECT * FROM Sample1 WHERE trunc(col2) = TO_DATE('8 NOV 2016','DD MON YYYY'); 但我强烈建议不要使用这样的日期格式。 它取决于SQL 客户端程序的NLS设置,并可能因不同的语言设置而失败。 我更喜欢使用A ...
  • 您的查询有两个问题 在BigQuery Legacy SQL中你不能在GROUP BY语句中使用表达式,而只是字段(顺便说一下,BigQuery Standard SQL中不存在这个限制) 当您执行GROUP BY时 - 您不能只是简单地选择所有字段,而是需要使用一些聚合函数,如COUNT或SUM等(对于那些不属于GROUP VY的字段) 所以你的查询看起来像下面这样: SELECT DATE(data_timestamp) AS dt, COUNT(1) AS cnt FROM [myproj:myda ...

相关文章

更多

最新问答

更多
  • 如何使用自由职业者帐户登录我的php网站?(How can I login into my php website using freelancer account? [closed])
  • 如何打破按钮上的生命周期循环(How to break do-while loop on button)
  • C#使用EF访问MVC上的部分类的自定义属性(C# access custom attributes of a partial class on MVC with EF)
  • 如何获得facebook app的publish_stream权限?(How to get publish_stream permissions for facebook app?)
  • 如何并排放置两个元件?(How to position two elements side by side?)
  • 在MySQL和/或多列中使用多个表用于Rails应用程序(Using multiple tables in MySQL and/or multiple columns for a Rails application)
  • 如何隐藏谷歌地图上的登录按钮?(How to hide the Sign in button from Google maps?)
  • Mysql左连接旋转90°表(Mysql Left join rotate 90° table)
  • 带有ImageMagick和许多图像的GIF动画(GIF animation with ImageMagick and many images)
  • 电脑高中毕业学习去哪里培训
  • 电脑系统专业就业状况如何啊?
  • IEnumerable linq表达式(IEnumerable linq expressions)
  • 如何在Spring测试中连接依赖关系(How to wire dependencies in Spring tests)
  • Solr可以在没有Lucene的情况下运行吗?(Can Solr run without Lucene?)
  • 如何保证Task在当前线程上同步运行?(How to guarantee that a Task runs synchronously on the current thread?)
  • 在保持每列的类的同时向数据框添加行(Adding row to data frame while maintaining the class of each column)
  • 的?(The ? marks in emacs/haskell and ghc mode)
  • 一个线程可以调用SuspendThread传递自己的线程ID吗?(Can a thread call SuspendThread passing its own thread ID?)
  • 延迟socket.io响应,并“警告 - websocket连接无效”(Delayed socket.io response, and “warn - websocket connection invalid”)
  • 悬停时的图像转换(Image transition on hover)
  • IIS 7.5仅显示homecontroller(IIS 7.5 only shows homecontroller)
  • 没有JavaScript的复选框“关闭”值(Checkbox 'off' value without JavaScript)
  • java分布式框架有哪些
  • Python:填写表单并点击按钮确认[关闭](Python: fill out a form and confirm with a button click [closed])
  • PHP将文件链接到根文件目录(PHP Linking Files to Root File Directory)
  • 我如何删除ListView中的项目?(How I can remove a item in my ListView?)
  • 您是否必须为TFS(云)中的每个BUG创建一个TASK以跟踪时间?(Do you have to create a TASK for every BUG in TFS (Cloud) to track time?)
  • typoscript TMENU ATagParams小写(typoscript TMENU ATagParams lowercase)
  • 武陟会计培训类的学校哪个好点?
  • 从链接中删除文本修饰(Remove text decoration from links)