Exchange 2003 Memory Optimization*
*This is a copy of an article I wrote for MSD2D.com. Since that site has been redesigned and the article is no longer online, I thought it would be a good idea to keep it on this blog.
Anyone who knows Exchange has probably heard by now that Exchange is a memory eater. The store process is the main responsible for this behavior, since store.exe starts it will grab as much memory as it can possibly get. This behavior is often wrongly seen as a problem or as a memory leak, but actually it’s a normal and expected operation. Besides, Exchange can return memory to the operating system using an algorithm known as Dynamic Buffer Allocation. And yes, you can limit the maximum amount of memory that Exchange uses by reducing the ESE Buffer size.
By these days, memory is not as expensive as it used to be, so it’s easy to find Exchange servers with a couple of GB of RAM. But with all this memory, you’ll have to give a little help to make Exchange use it wisely. If you have a server with more than 1GB of RAM, there are some configuration parameters you can change in order to optimize Exchange memory usage.
I’ll describe the modifications you should do just for Exchange 2003 running on Windows 2003. There are slightly differences for Windows 2000, but I will not mention them in order to keep this article shorter (if you really want to know the differences feel free to drop me an email).
You should not make any modifications to servers that do not contain any mailboxes or public folders (front-ends or bridgeheads), neither to Exchange Server computers which are at the same time Active Directory Domain Controllers or Global Catalogs.
- First of all you should add the switches /3GB and /USERVA=3030 to boot.ini. The /3GB switch modifies the way virtual address space is created so that 3 gigabytes are available for user mode applications. By default, Windows reserves 2GB for kernel and another 2GB for user mode processes. The /USERVA switch is a more precise tuning Microsoft recommends that increases the system page table entries (PTE) by 42MB.
- Configure the HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\HeapDeCommitFreeBlockThreshold registry value to 0x00040000. The HeapDecommitFreeBlockThreshold registry key specifies the number of contiguous bytes above which the memory is decomitted rather than retained for reuse, thus avoiding virtual memory fragmentation.
- If you have a server with more than 2 GB of memory, it may help to increase the size of the Store Database Cache (aka ESE buffer). Because of virtual address space limitations, this value must not be set higher than 1200 MB. You should use Windows Performance utility to monitor the memory of the server before you change this setting. To do this, monitor the following performance object and value:
Performance object: Process
Performance counter: Virtual Bytes
If you have a server that is configured with the /3GB and the virtual bytes counter is at 2.5 GB when the server is heavily loaded, you may be able to increase your maximum buffer size by about 300 MB, for a total size of 1200 MB. But keep in mind that increasing the buffer size may adversely affect server performance, so you’ll have to be very careful with this setting.
To modify the ESE Buffer size you may use the ADSI Edit utility. Under Configuration Container expand CN=Services, CN=Microsoft Exchange, CN=OrganizationName, CN=Administrative Groups, CN=First Administrative Group, CN=Servers, CN=servername. Under CN=servername, right-click CN=InformationStore, and then click Properties. Find the msExchESEParamCacheSizeMax property and in the Edit Attribute box, type the value that you want to assign to it (make sure that you enter a value that is a multiple of 8,192). Click Set, and then click OK.
- Verify that the HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\SystemPages registry value is set to 0.
After making all of these modifications you must restart your server for these changes to take effect.
Remember that there is no point in having a dedicated Exchange 2003 server with more than 4GB of memory. Although this may constitute a surprise for some of you, Exchange Server 2003 does not support instancing, Physical Address Extension (PAE), or Address Windowing Extensions (AWE). Therefore, 4 GB of RAM is the maximum amount of memory that an Exchange Server computer can efficiently use.
If you want to know more about Exchange Server memory usage, there are some Knowledge Base articles dedicated to this issue:
“How to Optimize Memory Usage in Exchange Server 2003”
“Using the /Userva Switch on Windows Server 2003-based computer that are running Exchange Server”
“The "HeapDecommitFreeBlockThreshold" registry key”
“CPU and Memory Scalability for Exchange 2000 and Exchange 2003”
“How to troubleshoot virtual memory fragmentation in Exchange Server 2003 and Exchange 2000 Server”