中年engineerの独り言 - crumbjp

LinuxとApacheの憂鬱

Analyze the Analyzer (MongoDB 2.4)

mongodb

Analyzer problem ?

MongoDB Analyzer sometimes retuns below values that is difficult to interpret.

RS:PRIMARY> db.serverStatus().mem
{
        "bits" : 64,
        "resident" : 127,
        "virtual" : 11292,
        "supported" : true,
        "mapped" : 5262,
        "mappedWithJournal" : 10524
}
RS:PRIMARY> db.serverStatus({workingSet:1}).workingSet
{
        "note" : "thisIsAnEstimate",
        "pagesInMemory" : 114700,
        "computationTimeMicros" : 33046,
        "overSeconds" : 1189
}
resident (MB)
This value stands for how much physical memory is associated with the mongod process's virtual memory.
pagesInMemory (4kb per page)
This value stands for how many mapped pages on the physical memory.

Of course, the mapped pages are in the mongod process's virtual memory.
So the value of resident is to be lager than the value of pageInMemory ( x 4096) !

But I sometimes got below:

resident
127 (MB)
pageInMemory
114700 = 448(MB)
Additional notes
$ getconf PAGESIZE
4096

What happens !!?

By mongodb source code

I found all implementation in the "src/mongo/util/processinfo_linux2.cpp".

#FF0000;font-size:x-large;">resident: ProcessInfo::getResidentSize()
The value comes from /proc//stat
#FF0000;font-size:x-large;">pageInMemory: ProcessInfo::pagesInMemory()
The value comes from mincore() system call.

But these looks like correct implementations...

By kernel source code

I finally could grasp the root of this issue.

/proc//stat
The value of "resident" is counted by around here
               :
	if (pte_present(ptent)) {
		page = vm_normal_page(vma, addr, ptent);
	} else if (is_swap_pte(ptent)) {
		swp_entry_t swpent = pte_to_swp_entry(ptent);

		if (!non_swap_entry(swpent))
			mss->swap += ptent_size;
		else if (is_migration_entry(swpent))
			page = migration_entry_to_page(swpent);
	} else if (pte_file(ptent)) { 
                // HERE !!!
		if (pte_to_pgoff(ptent) != pgoff)
			mss->nonlinear += ptent_size;
	}

	if (!page)
		return;

	if (PageAnon(page))
		mss->anonymous += ptent_size;
               :
mincore()
The value of "pageInMemory" is counted by around here
mincore() tries to check pages in spite of the pages are out of the PTE. 
　　　　　　　　　：
		if (pte_none(pte))
			mincore_unmapped_range(vma, addr, next, vec);
		else if (pte_present(pte))
			*vec = 1;
		else if (pte_file(pte)) {
                        // HERE !!
			pgoff = pte_to_pgoff(pte);
			*vec = mincore_page(vma->vm_file->f_mapping, pgoff);
		} else { /* pte is a swap entry */
			swp_entry_t entry = pte_to_swp_entry(pte);
　　　　　　　　　：

Results

We should believe the value of "pageInMemory".
This pages are surely on the physical memory.
To touch pages that is out of the resident(PTE) will cause merely minor page fault.
So this page fault will finish immediately.

Slide