The Cache

the problem

Databases are slow. Not because they are badly written, but because durable storage is physically slow. A network round trip to a database takes milliseconds. A lookup in local memory takes nanoseconds. That is a difference of six orders of magnitude.

A cache sits between the consumer and the source of truth, storing copies of hot data in a faster medium. When the data you need is already cached, you skip the slow path entirely. When it is not, you pay the full cost plus the overhead of checking.

The question is never whether to cache. The question is what to cache, when to invalidate, and what consistency guarantees you need.

cache-aside

The simplest pattern. The application checks the cache first. On a miss, it reads from the database, stores the result in the cache, and returns it. On a write, the application updates the database and either invalidates or updates the cache entry.

read(key):
  value = cache.get(key)
  if value is null:        // miss
    value = db.get(key)
    cache.set(key, value)
  return value             // hit or freshly cached

Cache-aside gives the application full control. It works well when read patterns are predictable and cache misses are tolerable. The downside is that the application must manage both the cache and the database, and there is always a window where the two can diverge.

write-through vs write-back

Write-through writes to the cache and the database on every write. The cache is always consistent with the database, but every write pays the full database latency. Simple and safe.

Write-back (also called write-behind) writes to the cache only, marking the entry as dirty. Dirty entries are flushed to the database later, in batches or on a timer. Writes are fast because they never wait for the database. But if the cache crashes before flushing, those writes are gone.

The tradeoff is clear: write-through trades write latency for durability. Write-back trades durability for write latency. Most systems default to write-through and only use write-back when write throughput is critical and some data loss is acceptable.

eviction

Caches have finite capacity. When the cache is full and a new entry needs space, something must go. Eviction policies decide what to remove.

LRU (Least Recently Used) evicts the entry that has not been accessed for the longest time. The assumption is that recently accessed data is likely to be accessed again. Implemented with a doubly-linked list and a hash map for O(1) lookups and evictions.

LFU (Least Frequently Used) evicts the entry with the fewest accesses. Better for workloads with stable hot sets, but more complex to implement and slower to adapt to changing access patterns.

TTL (Time To Live) expires entries after a fixed duration regardless of access patterns. Simple and predictable. Often combined with LRU or LFU.

LRU wins in practice because it is simple, fast, and adapts well to changing workloads. Redis, Memcached, and most application caches default to LRU or its variants.

cache invalidation

There are only two hard things in computer science: cache invalidation and naming things.

The core problem: when the source of truth changes, the cache must be updated or invalidated. If it is not, reads return stale data. Strategies include:

• TTL-based expiry: entries expire after a fixed time. Simple, but stale data persists until expiry.
• Event-driven invalidation: the database publishes change events, and the cache subscribes. Accurate, but adds infrastructure complexity.
• Write-through invalidation: every write updates the cache. Consistent, but couples writes to cache availability.

No strategy is perfect. TTL is easy but laggy. Event-driven is accurate but complex. Write-through is consistent but slow. Most systems combine two or more approaches.

where it shows up

• Redis: In-memory data store used as a cache layer in front of databases. Supports TTL, LRU eviction, and pub/sub for invalidation.
• Memcached: Simple, fast, distributed memory cache. No persistence, no data structures beyond key-value. Pure caching.
• CDNs (Cloudflare, CloudFront): Cache static and dynamic content at edge locations. TTL-based with cache-control headers. Reduces latency by serving content from the nearest edge.
• CPU caches (L1/L2/L3): Hardware caches that exploit spatial and temporal locality. Write-back by default (dirty cache lines flushed on eviction). The original cache.
• Browser cache: HTTP caching with Cache-Control, ETag, and Last-Modified headers. Reduces network requests for static assets.