CAP Theorem Simulator

1. The Three Pillars of CAP

To understand the theorem, we must rigorously define what these three letters actually mean in the context of distributed engineering. They do not mean what they mean in casual conversation.

2. The Illusion of "Choosing Two"

The classic explanation of CAP is a Venn Diagram where you pick any two: CA, CP, or AP. This is highly misleading. In the real world of distributed computing operating across a physical network, partitions are guaranteed to happen. Networks fail. Always.

Because you cannot avoid P (Partition Tolerance), you do not get to pick "CA". A "CA" system is simply a single-node database (like a standalone Postgres instance on your laptop). The moment you put data on two machines separated by a network cable, you are forcing P into the equation.

Therefore, the CAP Theorem isn't a menu of three options. It is a decision you must make during a network failure: Do you drop Availability (cancel the request to ensure Consistency) or drop Consistency (return stale data to ensure Availability)?

3. Beyond CAP: The PACELC Theorem

A major criticism of the CAP Theorem is that it only describes what happens during a disaster (a network partition). But what happens the other 99.9% of the time when the network is perfectly fine?

In 2010, Daniel Abadi proposed the PACELC Theorem, which accurately describes the continuous trade-offs databases make. It states:

The Normal Operation Trade-off (E: L vs C)

Even if there is no partition, data takes physical time (Latency) to travel at the speed of light across a datacenter or ocean. When a client writes to Node A, Node A must send that data to Node B.

• C
  Favoring Consistency (EC) Node A blocks the client's request, waits for Node B to acknowledge receipt, and then responds "Success". This guarantees Consistency, but increases Latency. (e.g., synchronous replication).
• L
  Favoring Latency (EL) Node A responds "Success" to the client instantly, and then asynchronously replicates to Node B in the background. This provides ultra-fast Latency, but creates a brief window of Inconsistency where Node B is out of date. (e.g., asynchronous replication).

Database Classifications under PACELC:

4. Real World Chaos: Split-Brain and Consistency Models

System design is rarely binary. Engineering modern databases involves fighting physics using complex algorithms to navigate the spectrum between absolute CP and absolute AP.

The "Split-Brain" Problem

Imagine a 2-node CP system (Node A and Node B). The network cable between them is cleanly cut. Both nodes are still running and accepting client traffic, but they cannot talk to each other. Node A assumes Node B is dead. Node B assumes Node A is dead. If a client writes "X=5" to A, and another writes "X=10" to B, the system has fractured into two diverging realities. This is a Split-Brain.

To prevent this, CP systems use Quorums (see the Quorum Simulator) and consensus algorithms like Raft or Paxos. To safely accept a write, a node must communicate with a strict majority (N/2 + 1) of the cluster. In a 5-node cluster, a partition might split it into a group of 3 and a group of 2. The group of 3 has quorum and continues operating. The group of 2 realizes it lacks a majority and intentionally halts itself (sacrificing Availability) to prevent Split-Brain.

Levels of "Consistency"

If you choose AP, what exactly does your "inconsistency" look like? Database engineers have defined a gradient of consistency models:

• Strong Consistency (Linearizability): The gold standard. Read always returns the latest write. Requires synchronous coordination (CP).
• Sequential Consistency: Operations appear to take place in some total order, though not necessarily exactly correlated with physical time clocks.
• Causal Consistency: If Write B was triggered by Reading Write A, then all clients will see A before B. Causally unrelated writes can be seen in any order.
• Read-Your-Own-Writes Consistency: A client will always see its own updates immediately, even if other clients don't see them yet. Useful for UX profiles.
• Eventual Consistency: The weakest AP model. If all writes stop, eventually (in milliseconds, hours, or days), all nodes will converge to the same state. Before that, reads can return wildly different answers depending on which node you hit.

Conflict Resolution in AP Systems

If an AP system allows two disconnected nodes to accept competing writes during a partition, how does it fix the mess when the network heals?