Caching

Punk::Cache is a key/value cache with TTL, declared with the cache keyword:

cache 'file', dir => '/var/cache/myapp', max_bytes => '512M';

get '/profile/:id' => sub {
    my ($c) = @_;
    my $html = $c->cache->compute("profile:" . $c->param('id'), 300, sub {
        render_expensive_thing($c);
    });
    $c->html($html);
};

compute is the method that matters: get, and on a miss run the code, store the result and return it. A cached undef is a value - an expensive lookup that legitimately finds nothing is not repeated on every request, which is the case people meet when they cache "does this user exist". get, set, delete, clear and stats exist too.

Values are bytes, uniformly, so every backend stores exactly the same things and switching backends cannot break on the one value that happened to be a reference. For a structure, pass json => 1 and it is encoded and decoded around the store.

Which backend

file is the default, and the arithmetic is the reason. An in-memory store lives in one process, so under a prefork server every worker has its own: workers => 8 with a 512M cap is four gigabytes of RSS, all caching the same things separately. The filesystem is already shared, so a file store is one copy for the whole pool, and it survives a restart. A file hit costs about six microseconds against nanoseconds for memory - a ratio that sounds decisive and is not, beside a request about to render a template or query a database.

Named stores exist because different caches want different things - a session store holds many small values that must not be evicted by a page store holding a few large ones:

cache 'file', dir => '/var/cache/app';             # the default
cache pages    => { backend => 'memory', max_bytes => '64M' };
cache sessions => { backend => 'file', dir => '/var/cache/sessions' };

$c->cache;               # the default
$c->cache('pages');      # a named one

Asking for a name that was never declared croaks - a store that silently never hits looks like a working cache with a disappointing hit rate. Any object with the five methods is a backend, so a Redis or DBI store is one module, and the conformance suite in Punk's tests takes a factory to assert yours behaves like the shipped ones.

The memory tier

cache 'file', dir        => '/var/cache/app',
              max_bytes  => '2G',
              memory     => '64M',    # per worker
              memory_ttl => 5;        # seconds

memory puts a per-worker, byte-budgeted copy in front of a shared store, so a hot key is answered without touching it. Read what it costs before reaching for it: the store stops being instantly consistent (bounded by memory_ttl and the invalidation reaching each worker), and the memory is multiplied per worker again - the arithmetic that made file the default in the first place. It is off unless asked for, and worth asking for when a small set of keys is read far more often than it is written.

A tier entry never outlives the entry it stands in for; memory_ttl is a ceiling on how long a value may be answered from the tier, and only ever shortens. A tier is only allowed in front of a shared store - memory in front of memory croaks at boot.

Invalidation across the pool

A store that is not shared between workers - the memory backend, or any shared store wearing a tier - is told when a key changes, over Hyperman's message bus. The key travels, never the value; each worker drops its copy and recomputes on demand. set invalidates as well as delete, because updating a cached thing is commoner than deleting it.

Invalidation is best effort - the bus is bounded and drops oldest under pressure - and TTL is the backstop: invalidation makes a cache fresh quickly, the TTL makes it eventually correct. Always set one. stats reports shared, pool, invalidations_sent and invalidations_received (plus memory_* counters on a tiered store), so an operator can tell a coherent pool from one where every worker is quietly serving its own stale copy.

The file store also has single-flight: when a hot key expires under load, one worker computes while the others wait for the answer. It is best effort, deliberately - a waiter that exhausts lock_wait computes anyway and a stale lock is stolen, because duplicated work is a cost while a stalled request is an outage.

What sits on the cache

The store contract is a seam, and three shipped things sit on it:

Server-side sessions - session store => ... keeps sessions in any Punk::Cache backend, with revocation, rotation and sliding expiry.

Punk::Plugin::Idempotency - Idempotency-Key on unsafe methods, replaying the stored response:

plugin 'Idempotency' => {
    scope => sub { $_[0]->current_user->{id} },
    ttl   => 86400,
};

post '/orders' => sub { ... }, { idempotent => 1 };

A client that sends POST /orders and loses the connection cannot know whether the order was created; a retry carrying the same key within the TTL replays the first response instead of creating a second order. scope is required and has no default: the stored value is a whole response - somebody's order - and if two accounts can produce the same cache key, the plugin becomes a way to read other people's responses. Be clear-eyed about the guarantee: cache-backed idempotency collapses the window between committing the work and recording the response to about three microseconds; it does not remove it. Closing it entirely needs the key written inside the same transaction as the work, which means the store is your database.

Response freshness is a different kind of caching - the client's - and lives in SEO and HTTP caching.