侧边栏壁纸
  • 累计撰写 15 篇文章
  • 累计创建 10 个标签
  • 累计收到 4 条评论

目 录CONTENT

文章目录

内存不足

骑摩托的程序员
2019-03-04 / 0 评论 / 0 点赞 / 1,013 阅读 / 0 字
  • 发生情况: 多个服务部署在用一服务器,几个java程序,还有redis,jar目录出现文件【hs_err_pid16493.log】!
#  
# There is insufficient memory for the Java Runtime Environment to continue.  
# Native memory allocation (mmap) failed to map 149946368 bytes for committing reserved memory.  
# Possible reasons:  
#   The system is out of physical RAM or swap space  
#   In 32 bit mode, the process size limit was hit  
# Possible solutions:  
#   Reduce memory load on the system  
#   Increase physical memory or swap space  
#   Check if swap backing store is full  
#   Use 64 bit Java on a 64 bit OS  
#   Decrease Java heap size (-Xmx/-Xms)  
#   Decrease number of Java threads  
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (os_linux.cpp:2640), pid=16493, tid=0x00007f2500bfb700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_171-b11) (build 1.8.0_171-b11)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.171-b11 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
	
……
	
Memory: 4k page, physical 8010180k(123544k free), swap 2097148k(0k free)
	
vm_info: Java HotSpot(TM) 64-Bit Server VM (25.171-b11) for linux-amd64 JRE (1.8.0_171-b11), built on Mar 28 2018 17:07:08 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)
	
time: Wed Dec 12 17:41:54 2018
elapsed time: 93 seconds (0d 0h 1m 33s)![file](/upload/2019/1/image-155003468825320190213131130891.png)

第一次见到这种东西很懵逼,百度查了下类似文件,翻译开头说内存不足引起的。

查看top,刚开始只注意到Mem 可以空间还有将近200M,就没忘内存方面想,以为是高大上的jvm问题!

后发现swap分区free为0. use100%!

然后就各种查资料,有可以查看swap分区占用程序pid的脚本,便开始解决问题!

vim swap.sh

#!/bin/bash
# Check who used swap

SUM=0
OVERALL=0
for DIR in `find /proc/ -maxdepth 1 -type d | egrep "^/proc/[0-9]"` ; do
PID=`echo $DIR | cut -d / -f 3`
PROGNAME=`ps -p $PID -o comm --no-headers`
for SWAP in `grep Swap $DIR/smaps 2>/dev/null| awk '{ print $2 }'`
do
let SUM=$SUM+$SWAP
done
echo "PID=$PID - Swap used: $SUM - ($PROGNAME )"
let OVERALL=$OVERALL+$SUM
SUM=0
	
	
done
echo "Overall swap used: $OVERALL"

运行命令

./swap.sh |sort -nr -k5 |head

file

发现有许多java程序pid并不存在但是占用swap很大,故将这些pid kill掉即可。

查看TOP

file

swap分区恢复正常

996.icu

0

评论区