A pricing change went out and a customer kept seeing the old price. Support cleared their browser cache. Still old. We redeployed. Still old. It turned out the price was cached in four places: the CDN, a Redis entry, an in-process memory cache on each app server, and the customer's browser. We had invalidated one of them.
The old joke about cache invalidation being one of the two hard problems is repeated so often that people treat it as a joke. It is not. It is a warning that caching converts a performance problem into a correctness problem, and the correctness problem is harder.
Here is how I think about it now, after being on the wrong end of that a few times.
Decide What Stale Means Before You Cache
The question that should come first, and almost never does: how wrong is this allowed to be, and for how long?
Answer it per piece of data, out loud, before choosing a strategy:
A blog post — hours of staleness is fine. Cache hard, invalidate on publish.
A product listing — a minute is fine. Nobody is harmed.
A stock count — seconds, maybe. Overselling has a real cost.
An account balance — zero. Do not cache it.
Permissions — zero, or you have a security bug with a TTL.
That last one is worth dwelling on. Caching authorisation decisions means a revoked user keeps their access for the length of the TTL. I have seen it, and it is the kind of thing that reads badly in an incident report.
The Patterns, and When Each Fits
Cache-aside is the default and what most people mean by caching. Check the cache; on a miss, read the database and populate it. Simple, and the cache holding only what is actually requested is a feature.
async function getProduct(id: string) {
const key = `product:${id}`;
const hit = await redis.get(key);
if (hit) return JSON.parse(hit);
const product = await db.product.findUnique({ where: { id } });
if (product) await redis.set(key, JSON.stringify(product), "EX", 300);
return product;
}
Write-through updates the cache and the database together on every write. Reads are always fresh, writes are slower, and you cache things nobody reads.
Write-behind writes to the cache and flushes to the database asynchronously. Fast writes, and you can lose data if the cache dies before the flush. I would only use this for genuinely disposable data like view counters.
For most systems, cache-aside plus explicit invalidation on write is the right combination, and it is what I reach for unless something specific rules it out.
Invalidation: Two Approaches, Pick One
TTL only. Set an expiry and accept staleness up to that window. Genuinely underrated — it is simple, it self-heals, and it cannot leak stale data forever. For most read-heavy content this is enough, and a 60-second TTL solves the problem people think requires elaborate invalidation.
Explicit invalidation. Delete the key when the underlying data changes. Correct, and it requires you to know every key affected by a write. That is where the four-places bug came from — we knew about one.
If you do explicit invalidation, two rules make it survivable. First, delete rather than update — a delete followed by a natural repopulation is easier to reason about than trying to write the new value everywhere. Second, put invalidation next to the write, not in a separate code path, so nobody can add a new writer that forgets.
// Invalidation lives with the write. One place to change.
async function updateProduct(id: string, patch: ProductPatch) {
const product = await db.product.update({ where: { id }, data: patch });
await Promise.all([
redis.del(`product:${id}`),
redis.del(`category:${product.categoryId}:products`), // the derived list too
]);
return product;
}
That second delete is the one people miss. Caching a list and then updating a member of it leaves the list stale.
Key Design Matters More Than It Looks
A few rules that have saved me repeatedly.
Include the tenant. user:profile:42 across a multi-tenant system is a data leak waiting for an ID collision. t:{tenantId}:user:42.
Include a version prefix. When the shape of the cached object changes, bumping v2: invalidates everything instantly and lets old entries expire on their own. Far better than a deploy that deserialises the old shape and throws.
Include everything that changes the result. Locale, currency, feature flag state, user role. A cached response that varies by something not in the key will serve the wrong variant to someone.
The Failure Modes Worth Knowing
Stampede. A popular key expires and five hundred concurrent requests all miss, all hit the database, and all recompute the same value. The database falls over at the exact moment traffic is highest. The fix is a short lock: the first request computes, the rest wait briefly or serve the stale value.
const lock = await redis.set(`lock:${key}`, "1", "NX", "EX", 10);
if (!lock) {
await sleep(50);
return getProduct(id); // by now someone else has populated it
}
Synchronised expiry. Warm a thousand keys at deploy with the same TTL and they all expire in the same second. Add jitter — 300 + Math.random() * 60 — so expiry spreads out.
Caching failures. An error path that caches a null or an empty array will serve that error for the whole TTL. Only cache successful results, deliberately.
Cache as a dependency. If Redis being down takes your app down, the cache became a database. Wrap reads so a cache failure falls through to the source with a logged warning.
Where to Put the Cache
Roughly in order of how much they buy you:
A CDN in front of static assets and cacheable pages. Cheapest and biggest win, and the layer most teams under-use for HTML.
HTTP caching headers — Cache-Control and ETag. Free, standard, and lets browsers and proxies avoid the request entirely. A correct ETag turns a repeat request into a 304 with no body.
Redis for shared application caching. Survives deploys, shared across instances, and the obvious default.
In-process memory for tiny, hot, rarely-changing data — feature flags, config, a currency table. Nanoseconds instead of a network hop. The catch is that every instance has its own copy and they will disagree after a write, so keep TTLs short and only use it where a few seconds of disagreement is harmless. This is the layer that caused our pricing bug.
The Thing to Do Before Caching Anything
Measure why it is slow. A query taking 800ms because it is missing an index does not need a cache — it needs the index. Caching it hides the problem, adds an invalidation surface, and leaves you with a slow query that now also has a correctness risk.
Cache when the work is genuinely expensive and unavoidable: an aggregate over millions of rows, a third-party API call, a rendered page. Not to paper over something you have not looked at yet.
And write down the layers. Our four-place bug happened because no single person knew all four existed.
