SEO and HTTP caching

Punk compiles every route at boot, so the application already holds a complete list of what it serves. Two things fall out of that for free: a sitemap that cannot drift from the routes, and asset URLs derived from file contents. The rest of this page is response freshness - Cache-Control, ETag and the 304s that keep unchanged bytes off the wire.

The sitemap

use Punk::Plugin::Sitemap;          # for the `sitemap` keyword

plugin 'Sitemap' => { base => 'https://example.com' };

get '/'          => sub { ... };    # in the sitemap
get '/users/:id' => sub { ... };    # out: a capture is not a URL
post '/orders'   => sub { ... };    # out: not a GET
get '/admin'     => sub { ... }, { sitemap => 0 };   # out: opted out

Punk::Plugin::Sitemap serves sitemap.xml and robots.txt from the route table. A route is listed only when it is a GET, its path holds no capture, it carries no guard, and it did not opt out. Out is the safe direction: a page missing from a sitemap is still crawled if anything links to it, while a page wrongly present is a crawler fetching a 404 or a login redirect on a schedule for as long as the site exists.

The part a hand-written file cannot do: a page behind a guard is excluded without anyone maintaining a list. under copies its guard chain into every route declared inside it, and the plugin reads it off the compiled record - so putting a section behind auth removes it from the sitemap in the same edit. sitemap => 1 opts a guarded route back in, sitemap => 0 takes any route out.

/users/:id names a shape, not a URL; the ids are the application's to supply:

sitemap users => sub {
    map { { loc => "/users/$_->{id}", lastmod => $_->{updated} } } @rows;
};

base is required and is configuration, deliberately: taking the host from the request would let Host: evil.example produce a sitemap naming that host for every page on the site - delivered to search engines, and invisible to the owner, whose own request produces a correct file.

Feeds

The other file a site owes the outside world, and the same reasoning throughout: the origin comes from configuration rather than the request, URLs are encoded before they are escaped, and the documents are built once and served from frozen bytes.

Punk::Feed serves /feed.xml and /feed.rss from entries an application supplies. It has a guide of its own.

Content-addressed assets

static 'https://fal.aihtbprolstatic-tp.hcv9jop5ns4r.cn' => 'root/static', fingerprint => 1;

<link rel="stylesheet" href="{% "/static/app.css" | asset %}">

$c->asset('/static/app.css') returns /static/app.9f3a1c2b0d4e5f60.css - the digest is the first 8 bytes of SHA-256 over the contents - and that URL serves with a year of freshness and immutable, because it cannot come to mean anything else. Change the file and the URL changes, so every deploy is its own cache bust and nothing is ever stale.

fingerprint is off unless asked for, because it changes what a path means - a URL shaped like a fingerprint stops being a 404 and starts resolving to another file. Until it is on, asset hands back the URL it was given, so templates can be written against it before the mount opts in. In development an edited file gets a new URL on the next reload; in production a digest is computed once and believed.

Freshness for plain files

static 'https://img.hcv9jop5ns4r.cn'      => 'root/img',   max_age => 3600;
static '/internal' => 'root/docs',  cache_control => 'private, no-store';

max_age puts Cache-Control: public, max-age=N on a plain URL; there is no default, so a mount that says nothing revalidates each time exactly as before. Give it only to assets you are willing to have served stale for that long - the fingerprinted URL is the answer for everything else. cache_control is a verbatim value for anything the two spellings do not cover. $c->send_file takes cache_control too.

Static files already answer conditional requests - ETag, Last-Modified, 304s and byte ranges - as does send_file.

304s for dynamic responses

A route that renders a page or returns JSON has had none of that, so an unchanged dashboard is re-queried, re-rendered, re-sent and re-parsed on every poll. Punk::Plugin::ConditionalGet plugs the gap:

plugin 'ConditionalGet';

# the validator: 304 without running the handler
get '/api/orders' => sub { ... }, {
    etag => sub { $_[0]->model('Order')->max_updated_at },
};

# or from the rendered bytes: saves the wire, not the server
get '/dashboard' => sub { ... }, { etag => 1 };

The two forms buy different things. etag => 1 hashes the rendered body: the client skips the download, the server did all the work. The coderef is a validator - something the application knows cheaply that identifies the current entity, a row's updated_at, a cache generation - and when the client sends the tag back, the response is a 304 and the handler never runs: no queries, no render, no serialisation. A validator that returns undef means "I do not know" and the request proceeds with no ETag at all - a validator that cannot answer must never be able to produce a wrong 304.

A 304 skips only the handler; it goes through the same finishing path as any response, so sessions, flash and headers all behave - flash in particular is not eaten by a request whose handler never read it.

One boundary worth restating from the CSP notes: a page that rendered a per-request nonce must never be cached, by this or anything else.