Built for agents
Downstream is a feed about AI agents, so it would be strange if agents couldn't read it. Every digest is published as plain text, Markdown, and JSON at stable URLs — no API key, no JavaScript, nothing rendered client-side. If you are a crawler or an agent, start at /llms.txt.
Endpoints
Static files, rebuilt with each digest.
- /llms.txttext/plain
- Start here. A map of the site in the llms.txt convention: today’s digest, every archived digest, and where the rest of the machine data lives.
- /llms-full.txttext/plain
- Every published digest inline, newest first, soft-capped at 100 KB so it stays parseable. If it truncates it says where, and points at /digests.json — whose rows link the omitted days as Markdown.
- /index.mdtext/markdown
- Today’s digest as Markdown with YAML front matter — title, date, description, keywords, canonical, generated_at, license.
- /archive/{date}.mdtext/markdown
- Any past digest, same shape. Dates are ISO (YYYY-MM-DD); /digests.json lists every one that exists.
- /feed.jsonapplication/feed+json
- JSON Feed v1.1, one entry per story, covering the 2 most recent published days. url anchors to the story on its digest page; external_url is the primary source. For anything older, walk /digests.json.
- /digests.jsonapplication/json
- The index of every digest ever published, held days included (those carry a hold_reason). One small row per day — date, item_count, generated_at, and the URLs for the page and its Markdown. It points at the content rather than carrying it: fetch a row’s markdown_url for that day’s stories.
- /rss.xmlapplication/rss+xml
- Plain RSS, for anything that already speaks it. Same 2-day window as /feed.json.
- /sitemap.xmlapplication/xml
- Every page with lastmod. Discoverable twice over: rel="sitemap" in the head of each page, and a Sitemap line in robots.txt.
- /robots.txttext/plain
- GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended and the rest are named and allowed. Nothing here is disallowed.
In the markup
Every page carries a canonical URL, a unique meta description, and a keyword list. Each digest also links a text/markdown alternate pointing at its own .md twin, so a parser can trade the HTML for clean Markdown without guessing the URL.
Structured data is Schema.org JSON-LD: WebSite and Organization site-wide, then CollectionPage wrapping an ItemList on each digest. Note where those ListItem URLs point — at the primary source of each story, not back at us. That is the whole posture: Downstream tells you what happened and hands you the original.
Terms, such as they are
What you find here is Downstream's own work — the summaries, the ranking, the corroboration counts, the topic and depth tags. Source article bodies are never republished.
So: use it, quote it, feed it to a model. When a story matters, cite and link the primary source — that is who earned the click. Crediting Downstream for the curation is welcome but optional. There is no rate limit and no auth; polite crawling is appreciated. The generated_at timestamp in the front matter and in /digests.json tells you whether you are looking at something you have already seen.
Why it is built this way
Answer engines are becoming how people find things, and most sites are illegible to them for no good reason — the fix is a handful of static files, not a platform. The reasoning, and the full recipe this site follows, is written up here: Agent: Do You Understand the Words Coming Out of My Mouth?
Something broken, or a surface you need that isn't here? hi@downstream.news