I have a Linux box running 5 instances of MongoDB, each one for a different environment. The server works fine during the day, most of the time at ~90% memory usage. But recently I started seeing that one (sometimes more) of the mongod instances running there died every night when my backup script ran, courtesy of the OS’s out-of-memory Killer. Thanks to Azure I can see the pattern very clearly:

After noticing that swap wasn’t enabled for the server, enabling it, and seeing that mongod processes kept dying nightly, I discovered that MongoDB does not use swap because it uses memory-mapped files:

Nevertheless, systems running MongoDB do not need swap for routine operation. Database files are memory-mapped and should constitute most of your MongoDB memory use. Therefore, it is unlikely that mongod will ever use any swap space in normal operation. The operating system will release memory from the memory mapped files without needing swap and MongoDB can write data to the data files without needing the swap system.

So enabling swap didn’t solve my problem of dying instances, but it’s something that the server should have anyway so I left it enabled.

I kept reading MongoDB’s official documentation and ran into this:

The default WiredTiger internal cache size value assumes that there is a single mongod instance per machine. If a single machine contains multiple MongoDB instances, then you should decrease the setting to accommodate the other mongod instances.

That sounded pretty promising! I started by looking at my instances to see what the current value was. You can do that by running db.serverStatus().wiredTiger.cache in the Mongo shell, and looking for property “maximum bytes configured” in the output document.

Sure enough, the server has 16GB of RAM, and those ~7.8GB are more or less what I’d expect based on the 0.5 * (RAM-GB - 1GB) calculation in the docs. The issue is that all five instances have the same value!

So off I went and changed that setting to 3GB instead… and voilà! Stable DB server again, even with 5 separate instances of Mongo running in there.