Pip, for readers who aren’t people

Every content page on this site now comes in two versions. Ask for one the way a browser does and you get HTML. Ask for it with an Accept: text/markdown header and you get the same page as plain markdown, with the navigation, styling and scripts left out. People get the site. Machines get the text.

You can try it from a terminal: curl -H "Accept: text/markdown" https://playpip.io/. Or, if you would rather not, put .md on the end of any content address and read the mirror directly.

A fair number of people now meet a website through something that read it for them. We would rather the thing doing the reading got it right. An AI tool parsing our HTML has to guess which parts are the page and which parts are the furniture; handing it the text instead removes the guessing.

It also follows from the rest of the project. Pip’s claim is that you can check it rather than take our word for it: the shuffle is seeded, the engine is a readable module, the code is all there. Being legible to machines is the same idea pointed at a different kind of reader.

Three pieces, none of them clever. A post-build script walks the finished export, converts each content page to markdown, strips the chrome, and writes the result next to the page as /privacy.md, /blog/launch-week.md and so on. The same script emits /llms.txt, an index of everything with a hand-written summary of what Pip is at the top, following the llms.txt convention. New posts are picked up from the export, so writing one requires no changes here. This post was.

In front of that sits the negotiation: a request carrying Accept: text/markdown is served the mirror, anything else is served the page. Browsers never send that header, so nothing changes for anyone using one. Both responses go out with Vary: Accept, so caches keep the two apart, and with RFC 8288 Link headers pointing at /llms.txtand at that page’s own markdown mirror. An agent that lands on the HTML by accident is told where the good version lives.

The module doing the work is 57 lines. Each of the six content routes that uses it is four more.

In March, Cloudflare announced Markdown for Agents, which does the same job at the edge and does it more thoroughly than we do. It is in beta, and it costs nothing extra, on the Pro, Business and Enterprise plans. Pip is on the free plan and intends to stay there, which ruled it out.

So the free version is the ninety-odd lines described above. If you are also on a free plan and want the same thing, the code is functions/_shared.ts and scripts/gen-llms.mjs. It is MIT licensed. Take it.

Our robots.txt now carries a Content Signal line: search=yes, ai-input=yes, ai-train=yes. Yes to being indexed, yes to being used as source material for an answer, yes to being trained on. Plenty of sites are saying no to the last two, and the reasoning is usually sound. Ours points the other way: the whole codebase is public under a licence that already permits all of it, and a model that has read Pip and can describe it accurately is a good outcome for us.

One correction while we are here. Until Friday the site was quietly turning AI crawlers away, via a managed robots.txt setting that was on by default and that we had never consciously chosen. It is off now. We had been publishing an agent-friendly index to an audience we were blocking at the door.

Only the six content pages have any of this attached: the home page, privacy, terms, credits, the blog and its posts. Everything else, the table, your profile, the stats screen, the learn pages, is the same static export it was before and never touches a server. The poker still happens entirely in your browser, which was always the point.