MongoDB

In a relational database, schema design and physical layout are separate concerns: you normalise into tables, then the planner and the storage engine decide how rows are fetched and joined. In MongoDB they are the same decision. A document is stored contiguously as BSON, and whatever you put inside it is retrieved in one read. Deciding to embed an array of order lines inside an order, instead of putting them in their own collection, is not a modelling preference. It is a choice about disk layout, index shape, and how many round trips a page render costs.

That is the single idea from which most of MongoDB’s behaviour follows, including its limits.

The document model and its hard constraint

The maximum BSON document size is 16 mebibytes. This is not tuneable. It exists so that a single document cannot consume an unbounded amount of RAM on the server or bandwidth in transit. Anything larger goes through the GridFS API instead.

Every embedding decision is bounded by that number, so the useful question is not “is this data related?” but “does this relationship have a bound?”.

The named anti-pattern is the unbounded array. The manual’s example is a books collection with a reviews array: every new review pushes another subdocument into the parent. Nothing fails at first. Then documents grow, index performance degrades, and eventually inserts start rejecting at 16 MB. By that point the fix is a migration, not a config change.

Two documented remedies. The subset pattern embeds the few reviews the page actually shows and puts the rest in their own collection; it duplicates data, so it suits data that is read often and updated rarely. Referencing stores only identifiers and joins at query time:

db.books.aggregate([
  { $match: { title: "Harry Potter" } },
  { $lookup: {
      from: "reviews",
      localField: "reviews",
      foreignField: "review_id",
      as: "reviewDetails"
  } }
])

That costs latency, which is the trade: you have swapped a guaranteed-size read for a join.

Two further document rules affect design: _id is reserved, immutable, and may not be an array or regex; and BSON field order is significant in comparisons, so {a: 1, b: 1} does not equal {b: 1, a: 1}.

Replica sets

A replica set is one primary and some secondaries. Writes go to the primary, are applied there, and are then recorded in the oplog (local.oplog.rs), a capped collection of idempotent operations that secondaries copy and replay asynchronously. Replication is asynchronous by default; the guarantees you get come from write concern, not from the topology.

Members heartbeat every two seconds. If a secondary cannot reach the primary for longer than settings.electionTimeoutMillis (10 seconds by default) it calls an election. The set cannot process writes until an election completes; the documented expectation is that the median time to elect a new primary does not exceed 12 seconds with default settings. A member cannot become primary unless it has the highest optime of any visible member. A replica set may have up to 50 members but at most 7 voting members.

Oplog sizing matters because the oplog window, the time span between its oldest and newest entries, is how long a disconnected secondary has to reconnect before it must be resynced from scratch. On Unix and Windows with WiredTiger the default is 5% of free disk space, floored at 990 MB and capped at 50 GB.

Write concern

writeConcern is { w: <value>, j: <boolean>, wtimeout: <number> }. The implicit default is w: "majority", with one documented exception: if the set contains arbiters and the number of non-arbiters is not greater than the majority of voting nodes, the implicit default drops to { w: 1 }. A P-S-A set therefore silently defaults to w: 1.

Three details cause most incidents. First, j: true alone does not prevent rollback on failover: it is a durability guarantee on individual nodes, not a replication guarantee. Second, wtimeout only applies when w is greater than 1, and when it fires the write is not rolled back: you get a write concern error for a write that may still commit. Third, with no wtimeout and an unachievable write concern, the operation blocks indefinitely.

For a 3-voting-member P-S-S set, the calculated majority is 2, so w: "majority" returns once the primary and one secondary have the write.

Read concern

Causal consistency across a session requires both readConcern: "majority" and writeConcern: "majority". One without the other buys nothing.

Read preference and the staleness it buys

Read preference decides which member serves a read: primary (default), primaryPreferred, secondary, secondaryPreferred, nearest. Every mode except primary may return stale data, because secondaries replicate asynchronously. Read preference does not affect visibility either. Clients can observe writes before they are majority-acknowledged.

maxStalenessSeconds bounds the damage: the client estimates each secondary’s lag against the primary’s last write and skips secondaries beyond the limit. The minimum permitted value is 90 seconds and a smaller value raises an error, because clients sample member write times infrequently and the estimate is coarse. It cannot be combined with mode primary. Transactions containing reads must use primary.

So secondary reads are appropriate for analytics and reporting, and inappropriate for read-your-own- writes. There is no read preference that makes a secondary current.

Sharding

A sharded collection is partitioned by shard key: one indexed field, or several fields covered by a compound index. MongoDB splits the shard key value space into non-overlapping ranges, each associated with a chunk, and the balancer distributes chunks across shards.

Ranged sharding keeps close key values together, which makes range queries targetable to few shards. Its failure mode is a monotonically increasing key such as a timestamp or ObjectId: the chunk with upper bound MaxKey receives every insert, so all write load lands on one shard. Hashed sharding hashes a single field and distributes evenly, at the cost of range queries becoming broadcast operations; equality matches are still targeted. A compound hashed key lets you keep a non-hashed prefix for zone ranges while a hashed component spreads the writes.

Hashed indexes truncate floating point values to 64-bit integers before hashing, so 2.2, 2.3 and 2.9 hash identically. Do not hash floats. Shard key indexes also cannot be descending, partial, geospatial, or multikey.

Historically a bad shard key was terminal. Since MongoDB 5.0 you can reshardCollection to a new key, and since 8.0 you can reshard onto the same key with forceRedistribution: true purely to redistribute data onto new shards. It is a real escape hatch, but an expensive one:

• Writes to the collection block for a period of about two seconds.
• Each recipient shard needs ((collection_storage_size + index_size) * 2) / shard_count of free space. These requirements are not enforced by the server; under-provisioning can fill the disk.
• I/O capacity should be below 50% and CPU below 80% before starting.
• Only one collection can be resharded at a time, and createIndexes, drop, dropIndexes, renameCollection, collMod and convertToCapped are unsupported on it while it runs.
• Until it completes, your queries must include both the old and the new shard key, or updateOne, deleteOne, replaceOne and the findAndModify family error unless the filter contains a unique field such as _id.

Refining a key - appending suffix fields to the existing one - is the cheaper option when it is sufficient.

The balancer runs on the config server replica set primary and is enabled by default. A collection is considered balanced when the data difference between shards is less than three times the configured range size; at the default range size of 128 MB, two shards must differ by at least 384 MB before a migration starts. A shard participates in at most one migration at a time, so a cluster of n shards performs at most n/2 concurrent migrations.

Indexing, and the prefix rule

MongoDB supports single-field, compound, multikey, text, hashed, geospatial, TTL, partial, sparse, unique and wildcard indexes. A collection can have at most 64 indexes, and a compound index at most 32 fields.

The compound index prefix rule

An index prefix is a leading subset of the indexed fields. This is the part people get wrong: an index is not a set of fields, it is an ordered B-tree, and you can only enter it from the left.

Given:

db.inventory.createIndex({ item: 1, location: 1, stock: 1 })

the prefixes are { item: 1 } and { item: 1, location: 1 }. MongoDB can support queries on item; on item and location; and on item, location and stock. It cannot use this index for a query on location, on stock, or on location and stock together. None of those begin with item.

The interesting case is the partial one. A query on item and stock, skipping location, does use the index, but only for the item portion. The manual’s worked figure: if 10,000 documents match item: "saccharomyces cerevisiae" and only 100 of those also match stock: 60, the IXSCAN stage examines 10,000 keys and filters them down to 100. The index “works” and the query is a hundred times more expensive than it looks.

// Uses the index fully
db.inventory.find({ item: "saccharomyces cerevisiae", location: "A1", stock: 60 })

// Uses only the item prefix; examines 10000 keys to return 100
db.inventory.find({ item: "saccharomyces cerevisiae", stock: 60 })

// Cannot use the index at all - collection scan
db.inventory.find({ location: "A1", stock: 60 })

Field order is chosen with the ESR guideline: Equality fields first, then Sort fields, then Range fields. A corollary worth acting on: if you have both {a: 1, b: 1} and {a: 1}, and neither is sparse or unique, drop {a: 1}. It is redundant, and it still costs write throughput.

Other index types

TTL deletion is a single-threaded background thread, not a guarantee of punctual removal. Per index it deletes until it has removed 50,000 documents, or spent one second, then moves to the next index; the loop also yields every 60 seconds. Under load, expired documents remain visible. Since MongoDB 6.1 deletions may be batched, which shows as a BATCHED_DELETE stage in explain. Since 5.1 you can convert an existing single-field index to a TTL index with collMod rather than rebuilding it.

For full-text search MongoDB now points at Atlas Search, not $text indexes.

The aggregation pipeline

A pipeline is an ordered list of stages, each consuming a stream of documents and emitting another. Monthly revenue per region, top three regions:

db.orders.aggregate([
  { $match: {
      status: "complete",
      orderDate: { $gte: ISODate("2026-01-01"), $lt: ISODate("2026-07-01") }
  } },
  { $unwind: "$lines" },
  { $group: {
      _id: { region: "$region", month: { $dateTrunc: { date: "$orderDate", unit: "month" } } },
      revenue: { $sum: { $multiply: ["$lines.qty", "$lines.unitPrice"] } },
      orders:  { $addToSet: "$_id" }
  } },
  { $project: {
      _id: 0,
      region: "$_id.region",
      month: "$_id.month",
      revenue: 1,
      orderCount: { $size: "$orders" }
  } },
  { $sort: { revenue: -1 } },
  { $limit: 3 }
])

The $match is first deliberately. Only a $match at the start of a pipeline can use an index to limit what enters it; once documents are in the pipeline, every later stage is a full pass over the stream. The optimiser will move filters earlier where it safely can. It splits a $match into per-field predicates and pushes each one before any projection stage it does not depend on, and it moves $match before a preceding $sort. What it cannot do is move a filter that depends on a computed field. Write the pipeline so it does not have to.

Limits: at most 1000 stages; each returned document is subject to the 16 MiB BSON limit, though intermediate documents may exceed it. Since MongoDB 6.0 the allowDiskUseByDefault parameter decides what happens when a blocking stage such as $group, $sort (unsupported by an index), $bucket or $setWindowFields needs more than 100 MB of memory: spill to temporary files, or error. Per command you can override with { allowDiskUse: true | false }. A usedDisk indicator appears in the profiler and diagnostic log when a stage spilled. Treat it as a signal to add an index or a $limit after $sort, since a $sort immediately followed by $limit is coalesced by the optimiser.

Multi-document transactions

Single-document operations are atomic, which is why a well-embedded model needs transactions less often. When you do need them, MongoDB supports multi-document transactions across collections, databases and shards. Constraints, all of them practical:

Pending DDL is the sharp edge. A DDL operation on a collection with an in-progress transaction waits behind it; new transactions touching the same collection then cannot get their locks and abort after maxTransactionLockRequestTimeoutMillis, and non-transactional operations block until maxTimeMS. One long transaction plus one createIndex is enough to stall a collection.

Use the driver callback API (with_transaction, withTransaction) in preference to the core API unless you need custom error handling: it retries TransientTransactionError and UnknownTransactionCommitResult for you. Set write concern at the transaction level, never per operation.

Diagnostics

explain() is the first tool. It returns the plan as a tree of stages; COLLSCAN means no index was used, IXSCAN means one was, FETCH means documents were retrieved after the index scan, and SORT in the tree means an in-memory sort the index could not satisfy. Run it with "executionStats" to see keys examined against documents returned. A large ratio is the prefix problem above.

db.inventory.find({ item: "saccharomyces cerevisiae", stock: 60 }).explain("executionStats")

Note that explain bypasses the plan cache: it generates candidate plans and picks a winner without consulting or populating the cache, so it does not always reflect what a repeated query does.

Slow operations. The slow operation threshold is 100 ms by default and applies to the whole mongod, feeding both the diagnostic log and the profiler. It is measured on workingMillis, the time MongoDB spent working, so lock waits and flow control do not push an operation over the line. Since 8.3 there is a separate slow in-progress operation threshold, defaulting to 5000 ms.

The profiler itself is off by default (level 0); level 1 records operations exceeding slowms or matching a filter, level 2 records everything and ignores slowms. It writes to a capped system.profile collection per database and degrades performance, so prefer the diagnostic log or $queryStats where you can.

db.setProfilingLevel(1, { slowms: 20 })

db.currentOp() and the $currentOp aggregation stage show what is running now: the right tool for a stuck operation, not a slow one. TTL deletions appear here too.

Where MongoDB is the wrong choice

Pick a relational database when the data is genuinely relational and the access patterns are not known in advance. MongoDB’s performance comes from modelling around known queries; if analysts will write arbitrary joins across a dozen entities tomorrow, a normalised schema and a mature cost-based optimiser will serve you better, and $lookup is not a substitute for a real join planner.

Pick a relational database when correctness depends on constraints the database must enforce. MongoDB has no foreign keys, no cross-collection referential integrity, and no cross-collection uniqueness. Unique indexes on sharded collections are only supported when the index contains the full shard key as a prefix. Any invariant spanning documents becomes application code plus a transaction. That is possible, but you are paying transaction costs to get what a relational engine gives you by default.

Pick a relational database when multi-entity transactions are the normal path and not the exception. MongoDB’s transactions work, but the one-minute default runtime, the 5 ms lock timeout, the cache pressure and the DDL interaction all say the same thing: they are a safety net, not a foundation.

MongoDB is the right choice when documents match the aggregate you actually read and write, when those aggregates have bounded size, when the schema varies between records, and when horizontal write scaling justifies choosing a shard key carefully up front. Those conditions are common. They are not universal, and the failure mode of pretending they hold is a migration.