Check Point Memory consumption – Out of Memory

If you issue the top or free command, you may be spooked and lead to believe you are out of memory and you start digging about checking the processes memory usage without getting any wiser. The reason being you and Linux using different terminology with regards to free memory, which I will try to explain this in this post.
Consider the output of the following Linux-commands

Expert@chkp-fw01]# free -m

total       used       free     shared    buffers     cached

Mem:          5958       5767        191          0        286       4122

-/+ buffers/cache:       1357       4600

Swap:        13311          0      13311

Only 191MB of the physical memory are listed as free.

[Expert@chkp-fw01]# top

top – 10:11:47 up 76 days, 13:03,  1 user,  load average: 0.57, 0.62, 0.60

Tasks: 155 total,   1 running, 154 sleeping,   0 stopped,   0 zombie

Cpu(s):  2.4%us,  0.6%sy,  0.0%ni, 91.5%id,  3.5%wa,  0.1%hi,  1.9%si,  0.0%st

Mem:   6101524k total,  5905804k used,   195720k free,   293136k buffers

Swap: 13631272k total,      140k used, 13631132k free,  4221868k cached

The same is reported by top.

Check Point software claims everything is ok.


SmartView Monitor reports no abnormal situation with regards to memory (Seems to be same numbers as cpstat -f old_memory os).
And the same goes for the following CLI-command:

[Expert@chkp-fw01]# cpstat -f memory os

 

Total Virtual Memory (Bytes):  20206383104

Active Virtual Memory (Bytes): 1423405056

Total Real Memory (Bytes):     6247960576

Active Real Memory (Bytes):    1423261696

Free Real Memory (Bytes):      4824698880

Memory Swaps/Sec:              –

Memory To Disk Transfers/Sec:  –

 

Disk Cache borrows memory
What we are looking at here is the OS “claiming” you are out of memory and Check Point claims you are not.
And the reason people thinks their systems is out of memory is because any available memory is borrowed by Linux for Disk cache and moved from the free-pool to the cached-pool.
This enables applications to access frequently used files directly from RAM rather than reading from disk, improving performance accordingly.
In our example this means 4122MB of the memory is used for disk cache.

  • If an application requries more memory than available in the free-pool, it will simply take back the memory from the cached-pool.

So the numbers you should focus on are

Expert@chkp-fw01]# free -m

total       used       free     shared    buffers     cached

Mem:          5958       5767        191          0        286       4122

-/+ buffers/cache:       1357       4600

 

[Expert@chkp-fw01]# cpstat -f memory os

 

Total Virtual Memory (Bytes):  20206383104

Active Virtual Memory (Bytes): 1423405056

Total Real Memory (Bytes):     6247960576

Active Real Memory (Bytes):    1423261696

Free Real Memory (Bytes):      4824698880

Memory Swaps/Sec:              –

Memory To Disk Transfers/Sec:  –

Double check if you have experienced any memory problems
If you want to double check you have not experienced any memory exhaustion, refer to the “fw ctl pstat”-command

[Expert@chkp-fw01]# fw ctl pstat

 

Machine Capacity Summary:

Memory used: 9% (406MB out of 4349MB) – below low watermark

Concurrent Connections: 63% (31876 out of 49900) – below low watermark

Aggressive Aging is disabled

 

Hash kernel memory (hmem) statistics:

Total memory allocated: 456097792 bytes in 111352 4KB blocks using 1 pool

Initial memory allocated: 456097792 bytes (Hash memory extended by 0 bytes)

Memory allocation  limit: 3647995904 bytes using 512 pools

Total memory bytes  used: 43932104   unused: 412165688 (90.37%)   peak: 81898532

Total memory blocks used:    12513   unused:    98839 (88%)   peak:    20760

Allocations: 2600341503 alloc, 0 failed alloc, 2600027312 free

 

System kernel memory (smem) statistics:

Total memory  bytes  used: 669813704   peak: 674592492

Total memory bytes wasted:  2111250

Blocking  memory  bytes   used:   951732   peak:  1803108

Non-Blocking memory bytes used: 668861972   peak: 672789384

Allocations: 682142 alloc, 0 failed alloc, 680932 free, 0 failed free

vmalloc bytes  used: 666632524 expensive: no

 

Kernel memory (kmem) statistics:

Total memory  bytes  used: 256392060   peak: 291448252

Allocations: 2601018382 alloc, 0 failed alloc, 2600703628 free, 0 failed free

External Allocations: 34880 for packets, 114673920 for SXL

In this example you will see the failed allocations are 0, so there is no memory problem on this machine.

Failed Allocation explanations

  • hmem means: Hash Kernel memory – Typically a configuration issue and adjustment to concurrent connections may be required.
  • smem means: System Memory – OS Memory was exhausted. Memory shortage.
  • kmem means: Kernel Memory – Some application did not get memory. Memory shortage.

Hopefully this explains how you and Linux used different terminology with regards to free/used memory and enabled to verify correct operation of your systems.

 

Was this article helpful?

Related Articles

Leave A Comment?

You must be logged in to post a comment.