MongoDB

Overview

MongoDB is the most widely used document-oriented database, designed to store data as flexible JSON-like documents (BSON) rather than rigid rows and columns. It bridges the gap between the scalability of NoSQL systems and the query power of relational databases, offering rich queries, secondary indexes, aggregation pipelines, and multi-document ACID transactions (since version 4.0).

Originally developed by 10gen (now MongoDB Inc.) in 2007, MongoDB became popular by making it easy for developers to persist their application objects directly — no ORM, no schema migration, no impedance mismatch. Companies like Uber, Lyft, Adobe, eBay, and Toyota use MongoDB in production for a variety of workloads.

Key capabilities include:

• Flexible schema: Documents in the same collection can have different fields
• Rich query language: Equality, range, regex, geospatial, text search, and aggregation pipelines
• Horizontal scaling: Auto-sharding with configurable shard keys
• High availability: Replica sets with automatic failover (typically <10 seconds)
• Multi-document ACID transactions: Since 4.0 for replica sets, 4.2 for sharded clusters
• Aggregation framework: Server-side data processing pipeline (map, filter, group, sort, join)

Architecture & Core Components

Data Model

graph TD
    subgraph "MongoDB Hierarchy"
        D[Database] --> C1[Collection: users]
        D --> C2[Collection: orders]
        C1 --> Doc1["Document {<br/>_id, name, email,<br/>address: {city, zip},<br/>tags: ['admin']<br/>}"]
        C1 --> Doc2["Document {<br/>_id, name, phone,<br/>preferences: {...}<br/>}"]
        C2 --> Doc3["Document {<br/>_id, user_id, items: [...],<br/>total, status<br/>}"]
    end

• Database: Logical container for collections
• Collection: Analogous to a table, but schema-free
• Document: A BSON record (JSON superset with additional types: ObjectId, Date, Decimal128, Binary)
• Field: Key-value pair within a document (can be nested objects or arrays)
• _id field: Required primary key, auto-generated as ObjectId if not specified

Replica Sets

The unit of high availability in MongoDB:

graph TD
    Client --> P[Primary]
    P -->|"Replicates oplog"| S1[Secondary 1]
    P -->|"Replicates oplog"| S2[Secondary 2]
    S1 -->|"Heartbeat"| P
    S2 -->|"Heartbeat"| P
    S1 -->|"Heartbeat"| S2

    subgraph "Automatic Failover"
        F["Primary fails → Election → <br/>Secondary becomes new Primary<br/>(typically <10 seconds)"]
    end

• Primary: Receives all write operations. Only one primary per replica set.
• Secondaries: Replicate the primary’s oplog (operation log). Can serve read operations with configurable read preferences.
• Arbiter: Voting member without data. Used to break ties in elections (3-member minimum for elections).
• Oplog: A capped collection on the primary that records all write operations. Secondaries tail this log to replicate.

Sharded Clusters

For horizontal scaling beyond a single replica set:

graph TD
    App[Application] --> MR[mongos Router]
    MR --> S1[Shard 1<br/>Replica Set]
    MR --> S2[Shard 2<br/>Replica Set]
    MR --> S3[Shard 3<br/>Replica Set]
    MR --> CS[Config Servers<br/>Replica Set]

    CS -->|"Chunk metadata"| MR

• mongos: Query router that directs operations to the correct shard(s)
• Shard: Each shard is a replica set holding a subset of the data
• Config servers: Store metadata about which chunks live on which shards
• Shard key: Determines how data is distributed across shards (critical choice — hard to change)
• Chunks: Contiguous ranges of shard key values. MongoDB auto-splits and auto-balances chunks.

Storage Engine: WiredTiger

Since MongoDB 3.2, WiredTiger is the default storage engine:

• Document-level concurrency: Fine-grained locking (not collection-level like the old MMAPv1)
• Compression: Snappy (default) or zstd for data, prefix compression for indexes
• Checkpointing: Consistent snapshots every 60 seconds
• Write-Ahead Log (journal): Durability between checkpoints
• In-memory cache: Configurable, defaults to 50% of RAM minus 1GB
• B-tree indexes: Standard B-tree for all index types

Key Features

Aggregation Pipeline

MongoDB’s server-side data processing framework, conceptually similar to Unix pipes:

db.orders.aggregate([
  // Stage 1: Filter orders from 2024
  { $match: { createdAt: { $gte: ISODate("2024-01-01") } } },

  // Stage 2: Unwind the items array
  { $unwind: "$items" },

  // Stage 3: Group by product, calculate totals
  { $group: {
    _id: "$items.productId",
    totalRevenue: { $sum: "$items.price" },
    totalQuantity: { $sum: "$items.quantity" },
    orderCount: { $sum: 1 }
  }},

  // Stage 4: Sort by revenue descending
  { $sort: { totalRevenue: -1 } },

  // Stage 5: Top 10 products
  { $limit: 10 }
]);

Common stages: $match, $group, $project, $unwind, $lookup (left outer join), $sort, $limit, $facet (parallel pipelines), $graphLookup (recursive joins).

Indexing

Change Streams

Real-time event stream of all changes to a collection, database, or deployment:

const changeStream = db.orders.watch([
  { $match: { "fullDocument.status": "completed" } }
]);

changeStream.on("change", (event) => {
  // event.operationType: "insert", "update", "delete", "replace"
  // event.fullDocument: the changed document
  processCompletedOrder(event.fullDocument);
});

Built on the oplog, change streams provide an at-least-once delivery guarantee and are resumable from a specific point.

Use Cases

When to Use MongoDB

• Rapidly evolving schemas: Startups, prototyping, applications where the data model changes frequently
• Content management: Blog posts, product catalogs, user profiles — documents with varying fields
• Real-time analytics: Aggregation pipeline + change streams for live dashboards
• IoT and time-series data (with time-series collections, introduced in 5.0)
• Mobile/gaming backends: Flexible schema for user data, game state, events
• Catalog/inventory systems: Products with varying attributes (clothing has size/color, electronics has specs)

When NOT to Use MongoDB

• Heavy joins/relationships: If your data is highly relational with many joins, PostgreSQL is better
• Strong consistency requirements: MongoDB’s default is “majority” read concern, but single-document operations are atomic. For complex multi-document transactions, relational databases have more mature tooling.
• Simple key-value at extreme scale: DynamoDB or Redis are simpler and faster for this
• Full-text search at scale: While MongoDB has text indexes, Elasticsearch is far more capable for complex search requirements
• Analytical workloads on petabytes: Use a columnar store (ClickHouse, BigQuery) instead

Comparison with Alternatives

Performance Characteristics

Typical Benchmarks

• Single document read by _id: ~0.1-0.5ms (WiredTiger cache hit)
• Single document write: ~1-2ms (with journal, write concern majority)
• Aggregation pipeline: 10-1000ms depending on data volume and pipeline complexity
• Bulk insert throughput: 50K-200K documents/sec per shard (depends on document size, indexes)

Shard Key Selection

The shard key is the most critical decision in a sharded deployment:

Bad shard keys (common mistakes):

• Monotonically increasing keys (ObjectId, timestamp) → all writes go to one shard
• Low-cardinality keys (status, country) → uneven distribution
• Keys not used in queries → scatter-gather for every query

Production Operational Concerns

Monitoring Key Metrics

• Replication lag: rs.printSlaveReplicationInfo() — should be under 1 second
• Opcounters: serverStatus.opcounters — tracks insert/query/update/delete rates
• Cache utilization: WiredTiger cache hit ratio (should be >95%)
• Connections: Current vs max connections (default max is 65,536)
• Tickets: WiredTiger read/write tickets (concurrency throttle — 128 default)
• Chunk balance: Difference in chunk count between shards (should be minimal)

Common Failure Modes

1. Hot shard: Poor shard key choice → one shard handles majority of traffic. Fix: reshard (expensive operation).
2. Replication lag: Secondary can’t keep up → stale reads. Causes: heavy write load, undersized secondary, long-running operations.
3. WiredTiger cache pressure: Working set exceeds cache → frequent evictions → slow reads. Fix: increase cache or add shards.
4. Unsharded collection growth: Collections that weren’t sharded grow beyond a single replica set’s capacity.
5. Config server issues: Config server replica set problems → routing failures in sharded clusters.

Interview Tips

• “When would you use MongoDB vs PostgreSQL?” — MongoDB for flexible schemas, rapid prototyping, document-centric data. PostgreSQL for complex joins, strict consistency, advanced SQL features.
• “How does MongoDB achieve high availability?” — Replica sets with automatic failover via election protocol. Primary failures detected by heartbeat, election completes in <10 seconds.
• “How does sharding work in MongoDB?” — Data partitioned by shard key across replica sets. mongos routes queries. Auto-balancing moves chunks between shards.
• “What’s the CAP theorem trade-off for MongoDB?” — MongoDB is CP by default (majority write concern). With lower write concerns, it can sacrifice consistency for availability.
• “How do you handle schema evolution?” — Document databases handle it naturally — old documents coexist with new ones. Application code handles both versions (or run a background migration).