Windows PerfCounters and Powershell - Memory perf data

In the last blog I spoke of CPU counters. Now, I'll talk of Memory counters.

MEMORY Counters (CIM_PhysicalMemory class, Win32_PerfFormattedData_PerfOS_Memory class, Memory Performance Information ...):

Note: I introduced the notion of samples and how to fetch them using NextValue() so I will occasionally omit $var.NextValue() going forward.

Let me note here that if you thought previously described performance classes were complicated, you are now entering the realm of black magic ;-) There is a good series of blogs on subject of Memory by Mark Russinovich worth reading although quite old.

Memory is a key resource for any machine so I will look at the most of the values available on Windows. In Resource monitor, Memory tab, you find a bar with Hardware reserved, In use, Modified, Standby and Free values. There are also Available, Cached, Total and Installed values. Let's start with the biggest number, Installed RAM.

In-depth description of Memory Counters important for my use-case:

COUNTER: cim_physicalmemory\Capacity
TYPE: Instantaneous
USAGE: (Get-Ciminstance -class "cim_physicalmemory" | Measure-Object Capacity -Sum).Sum / 1024 / 1024 #MB
MEANING: Total capacity of the physical memory, in bytes. Refers to "Installed".
GOTCHA: You will find tips to use TotalPhysicalMemory but, according to MSDN, it's been deprecated. Also, that page recommends using TotalVisualMemorySize property in the CIM_OperatingSystem class instead but this is wrong as there is no TotalVisualMemorySize property and, even if there was, we need installed memory size.
THRESHOLD:

Intermediate step; how much of the installed memory is available to OS:
COUNTER: win32_operatingsystem\TotalVisibleMemorySize
TYPE: Instantaneous
USAGE: [math]::Round((Get-CimInstance win32_operatingsystem).TotalVisibleMemorySize / 1024,2)
MEANING: Total amount of RAM available to OS. Refers to "Total".
GOTCHA:
THRESHOLD:

Subtracting TotalVisibleMemorySize from Capacity gives us HW reserved RAM, i.e. RAM taken by various HW such as video card. Check this post for details.
COUNTER: HW reserved
TYPE: Calculated
USAGE: cim_physicalmemory\Capacity (Installed) - win32_operatingsystem\TotalVisibleMemorySize (Total)
MEANING: Size of RAM not available to OS although installed on the system. Refers to "Hardware reserved".
GOTCHA: Depends on HW and BIOS settings, not something "fixable" in Windows.
THRESHOLD:

COUNTER: win32_operatingsystem\FreePhysicalMemory (Bytes), Memory\Available MBytes
TYPE: Instantaneous
USAGE:
(Get-WmiObject win32_operatingsystem).FreePhysicalMemory
$Memory_AvailMB = New-Object Diagnostics.PerformanceCounter("Memory", "Available MBytes")
(New-Object Diagnostics.PerformanceCounter("Memory", "Available MBytes")).RawValue
MEANING: Total amount of RAM available to processes. Equal to the sum of memory assigned to the standby (cached), free and zero page lists. Refers to "Available".
GOTCHA:
THRESHOLD: A consistent value of less than 20% of installed RAM. In such situations, consult additional counters, such as Win32_PerfFormattedData_PerfOS_Memory\PagesPerSec to determine if System memory is adequate for the workload.

COUNTER: In use memory
TYPE: Calculated
USAGE: win32_operatingsystem\TotalVisibleMemorySize (Total) - Memory\Available MBytes (Available)
MEANING: Amount of RAM in use by processes running on the box.
GOTCHA:
THRESHOLD:

COUNTER: Memory\Modified Page List Bytes (Win32_PerfFormattedData_PerfOS_Memory)
TYPE: Instantaneous
USAGE: $Memory_ModPLBy = New-Object System.Diagnostics.PerformanceCounter("Memory", "Modified Page List Bytes")
MEANING: The amount of RAM taken by the pages previously belonging to a working set but removed. However, the pages were modified while in use and their current contents haven’t yet been written to storage. The Page Table Entry still refers to the physical page(s) but is marked invalid and in transition. It must be written to the backing store before the physical page can be reused.
GOTCHA: No description in MSDN!?
THRESHOLD: Keep as low as possible.

COUNTER: Win32_PerfFormattedData_PerfOS_Memory\FreeAndZeroPageListBytes
TYPE: Instantaneous
USAGE: (get-wmiobject -computername localhost -Namespace root\CIMV2 -Query "Select * from Win32_PerfFormattedData_PerfOS_Memory").FreeAndZeroPageListBytes / 1024 / 1024 #MB
MEANING: The amount of physical memory, in bytes, that is assigned to the free and zero page lists thus immediately available for allocation to a process or for system use since it does not contain any data. Refers to "Free".
GOTCHA: There is a big difference between Free and Available memory. This is due to most of the pages considered available being in some sort of transition state (i.e. waiting to be written to disk) or have not yet met all of the OS requirements (i.e. page is not considered secure until it's zeroed out).
THRESHOLD: Keep as high as possible.

COUNTER: Standby
TYPE: Calculated
USAGE:
$Memory_SBCCBy = New-Object Diagnostics.PerformanceCounter("Memory", "Standby Cache Core Bytes")
$Memory_SBCNPBy = New-Object Diagnostics.PerformanceCounter("Memory", "Standby Cache Normal Priority Bytes")
$Memory_SBCRBy = New-Object Diagnostics.PerformanceCounter("Memory", "Standby Cache Reserve Bytes")
[math]::Round($Memory_SBCCBy.NextValue()/1024/1024 + $Memory_SBCNPBy.NextValue()/1024/1024+$Memory_SBCRBy.NextValue()/1024/1024,2)
MEANING: The amount of RAM in pages previously belonging to a working set but removed (or marshaled directly into the standby list). The pages weren’t modified since last written to disk. The Page Table Entry still refers to the physical pages but are marked invalid and in transition. Or, simpler explanation, memory that has been removed from a process's working set (its physical memory) en route to disk but is still available to be recalled.
GOTCHA: Please see the explanation of the factors in Win32_PerfFormattedData_PerfOS_Memory or Memory Object MSDN pages.
THRESHOLD:

COUNTER: Cached
TYPE: Calculated
USAGE:
MEANING: This number represents the sum of the system working set, standby list and modified page list. So, Memory\Cache Bytes, Memory\Modified Page List Bytes, Memory\Standby Cache Core Bytes, Memory\Standby Cache Normal Priority Bytes and Memory\Standby Cache Reserve Bytes. In this case, Memory\Cache Bytes + Memory\Modified Page List Bytes + Standby.
GOTCHA: Presented here for the sake of completeness.
THRESHOLD:

More counters of significance:

Win32_PerfFormattedData_PerfOS_Memory\CacheBytes - Number of bytes currently being used by the file system cache. The file system cache is an area of physical memory that stores recently used pages of data for applications. The operating system continually adjusts the size of the cache, making it as large as it can while still preserving the minimum required number of available bytes for processes. This property displays the last observed value only; it is not an average. See also SystemCacheResidentBytes and relatives.
Simpler explanation would be that the memory pages that the System uses are counted in two main counters, Cache Bytes and Pool Nonpaged Bytes. The Cache Bytes counter value is the amount of resident pages allocated in RAM that the Kernel threads can address without causing a Page Fault. This counter includes the Pool Paged Resident Bytes, the System Cache Resident Bytes, the System Code Resident Bytes and the System Driver Resident Bytes.

Note: If Memory\Pool Nonpaged Bytes value is 10% or more higher than its value at system startup, there is probably a leak.

Win32_PerfFormattedData_PerfOS_Memory\CacheFaultsPerSec - Number of faults which occur when a page is not found in the file system cache and must be retrieved from elsewhere in memory (a soft fault) or from disk (a hard fault). The file system cache is an area of physical memory that stores recently used pages of data for applications. Cache activity is a reliable indicator of most application I/O operations. This property counts the number of faults without regard for the number of pages faulted in each operation.

There is a whole set of Paging counters and they do require our attention since we can deduce Memory shortages on Windows by using them. Some of the key counters I will describe below. Dealing with Windows Paging you have to keep in mind that paging occurs for various operations within OS and excessive paging doesn’t automatically indicate a memory shortage. For instance, many network card drivers utilize the Pagefile (sometimes excessively) and this can be misread as a memory shortage.

Win32_PerfFormattedData_PerfOS_Memory\PagesPerSec (and relatives) - A sustained value of over 20 should be closely monitored and a System with a sustained value of over 50 is probably lacking in System Memory. Again, it is normal for this value to spike occasionally, especially if the other Memory counters do not show a lack of System Memory.

COUNTER: Pages Input per second / Page Reads per second
TYPE: Calculated
USAGE:
$Memory_PIps = New-Object Diagnostics.PerformanceCounter("Memory", "Pages Input/sec")
$Memory_PRps = New-Object Diagnostics.PerformanceCounter("Memory", "Page Reads/sec")
[math]::Round ($Memory_PIps.NextValue() / $Memory_PRps.NextValue(),2)
MEANING: The average of Memory\Pages Input/sec divided by average of Memory\Page Reads/sec gives the number of pages per disk read. This value should not generally exceed five pages per second. A value greater than five indicates that the system is spending too much time paging and requires more memory (assuming that the application has been optimized).
GOTCHA:
THRESHOLD: Sustained value of 5 or more.

Some other interesting counters I will not be covering in detail:

Memory\Page Reads/sec
Memory\Page Writes/sec
Paging File(_total)\% Usage
and so on.

In the next blog I will cover Disk counters.

In this series:
BLOG 1: PerfCounters infrastructure
BLOG 2: PerfCounters Raw vs. Formatted values
BLOG 3: PerfCounters, fetching the values
BLOG 4: PerfCounters, CPU perf data
BLOG 5: PerfCounters, Memory perf data
BLOG 6: PerfCounters, Disk/IO perf data
BLOG 7: PerfCounters, Network and Contention perf data