参考文档
1 2 https://blog.51cto.com/seekerwolf/2357220 https://www.centos.bz/2017/08/zabbix-zbx_mem_malloc-out-of-memory/
环境
1 2 CentOS 7.5.1804 Zabbix 3.4.15
现象 zabbix web图形界面显示zabbix server is not running，进入zabbix服务器 systemctl status zabbix-server显示服务down掉
排查 1.试着重新启动zabbix-server，启动不成功 2.查看日志/var/log/zabbix/zabbix_server.log
首先发现如下错误
1 2 3 zabbix_server [7988]: cannot open log: cannot create semaphore set: [28] No space left on device zabbix_server [8008]: cannot open log: cannot create semaphore set: [28] No space left on device zabbix_server [8022]: cannot open log: cannot create semaphore set: [28] No space left on device
根据网上资料修改kernel.sem参数，默认如下
1 2 [root@zabbix zabbix]# cat /proc/sys/kernel/sem 250 32000 32 128
其中含义分别如下
1 2 3 4 250 SEMMSL max semaphores per array 信号集容纳最大信号数量 32000 SEMMNS max semaphores system wide 所有信号的最大数量 32 SEMOPM max ops per semop call 调用单个信号集中最大信号数量 128 SEMMNI max number of arrays 信号集的最大值
直接增加1倍，并使其生效
1 2 3 [root@zabbix zabbix]# echo 'kernel.sem = 500 64000 64 256' >> /etc/sysctl.conf [root@zabbix zabbix]# sysctl -p kernel.sem = 500 64000 64 256
最后杀掉zabbix占用的共享信道
1 2 3 4 5 [root@zabbix zabbix]# ipcs -s|wc -l 143 [root@zabbix zabbix]# ipcs -s | grep zabbix | awk '{print $2}' | xargs -n 1 ipcrm -s [root@zabbix zabbix]# ipcs -s|wc -l 10
3.第二次启动zabbix-server发现如下错误
1 2 3 4 5 6 8599:20200401:140301.791 using configuration file: /etc/zabbix/zabbix_server.conf 8599:20200401:140301.798 current database version (mandatory/optional): 03040000/03040007 8599:20200401:140301.799 required mandatory version: 03040000 8599:20200401:140302.063 __mem_malloc: skipped 0 asked 120 skip_min 4294967295 skip_max 0 8599:20200401:140302.063 [file:dbconfig.c,line:90] zbx_mem_malloc(): out of memory (requested 120 bytes) 8599:20200401:140302.063 [file:dbconfig.c,line:90] zbx_mem_malloc(): please increase CacheSize configuration parameter
报错显示zabbix分配内存时出现了oom，默认zabbix服务器内存是够用的，幸好有提示请增加CacheSize，查资料发现可以在配置文件里增加该值，默认是8M，增加到1024M，第三次启动zabbix-server终于能正常启动了。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 [root@zabbix zabbix]# grep CacheSize /etc/zabbix/zabbix_server.conf ### Option: VMwareCacheSize # VMwareCacheSize=8M ### Option: CacheSize CacheSize=1024M ### Option: HistoryCacheSize # HistoryCacheSize=16M ### Option: HistoryIndexCacheSize # HistoryIndexCacheSize=4M ### Option: TrendCacheSize # TrendCacheSize=4M ### Option: ValueCacheSize # ValueCacheSize=8M
