LiminalisThe lens that proved the engine wasn't about fiction.
Two lenses for creative work prove very little — they share too many assumptions. The third lens was deliberately chosen from an unrelated domain: a corpus of roughly ten thousand pieces of US state legislation, at a scale no hand-curated project reaches. It shipped without a single special case in ingestion, chunking, extraction, retrieval, assembly, the access barriers, the job model or the agent API.
Cost is a first-class part of the workflow.
At corpus scale, “upload and let it think” is a way to spend real money by accident. So ingestion is explicitly staged, previews are free, and the expensive step is a separate, deliberate action — with actual spend read back from the credit ledger by document, not estimated.
-
Stage
Upload a corpus bundle or point at a server path. Extracts and counts the manifest. Costs nothing.
-
Scan
A manifest-only report of exactly what an import would produce — around eighty milliseconds across ten thousand rows.
-
Batch
Define a batch by filter: states, sides, statuses, policy types. Preview is instant and free.
-
Ingest
Create and embed the documents. Cheap. They become searchable and citable immediately.
-
Release for analysis
The expensive half — entity, fact, summary and motif extraction. A separate action, with a durable, poll-safe estimate before you commit.
ANALYSIS 62% · RESUMES FROM DISK AFTER RESTART
An effectively unbounded corpus needs an off switch. A project-level pause gates the autonomous background workers while deliberately not gating chat or explicitly submitted jobs — the line is autonomous versus requested. And it is lossless by construction: a paused document is simply never marked as processed, so it stays pending and the whole backlog resumes on the next poll. There is no catch-up path to get wrong, because there is no catch-up path.
Two analyses that cost almost nothing to run.
Both are deliberately built out of things the system already has — embeddings and the index — rather than out of model calls. Neither needs an LLM at all.
Model legislation, found by similarity
Single-link clustering over bill embeddings, with the similarity threshold calibrated against the source data's own companion and successor relationships rather than guessed. Members are stamped with a family id, a role, a scope and a novelty score. Costs embeddings; no model calls.
Cross-state families are the entire point. Same-state companion bills are already published by the data source and are not a finding. Near-identical text filed in six states in the same session is.
Where a policy category appeared first
Policy category × state, where each cell is the year that category first showed up in that state — plus a corpus-wide ranking of which categories tend to precede which. Free, index-only, no model calls.
It is deliberately not built on the Novel lens's weave grid. They look alike and mean completely different things; sharing the machinery would have been a false economy.
| Policy category | AL | AZ | FL | ID | MO | OK | TN | TX | UT | WV |
|---|---|---|---|---|---|---|---|---|---|---|
| Category A | 2021 | 2020 | 2020 | 2022 | 2021 | 2022 | 2021 | 2020 | 2022 | 2023 |
| Category B | 2022 | 2021 | 2021 | 2022 | 2023 | 2022 | 2021 | 2021 | — | 2023 |
| Category C | 2023 | 2022 | 2022 | — | 2023 | 2023 | 2022 | 2022 | 2023 | — |
| Category D | — | 2023 | 2023 | — | — | 2023 | 2023 | 2023 | — | — |
| Category E | — | — | 2024 | — | — | — | 2024 | 2024 | — | — |
Charts that tell you what they're not showing you.
Four views over the corpus, all reading a free index-only endpoint — opening the panel spends nothing. What is more interesting than the charts is the set of rules they had to obey.
Grouping shows membership by state — never a derivation chain
markup_fidelity is encoded, not filtered — a garbled extraction stays visibly distinguishable from a real outlier
Only about 28% of the corpus has any roll call at all — this chart is over that subset, not over the corpus
No derivation chains
Clustering computes similarity, not descent. Drawing bill→bill parentage would render an unfalsifiable claim about who copied whom as if it were data. So the tree groups membership by state and stops there.
Every chart states its denominator
Only around 28% of bills have a recorded roll call. A margin chart over that subset, drawn without saying so, silently reads as a chart over the whole corpus. So it says so, on the chart.
Data quality is encoded, not hidden
How a bill's text was converted is recorded on the bill. Roughly a fifth of rows have no retrievable text at all and are kept as stubs rather than quietly dropped — an absence you can see beats a corpus that looks complete.
“Amendment aggressiveness” was built, exposed, measured, and deleted.
The idea was good: score how aggressively a bill rewrites existing statute by comparing inserted text against struck-through text. It was fully implemented and already exposed as a filterable facet in the document browser.
Then someone checked the distribution before shipping it. 98.6% of rows reported zero insertions — because two of the dominant PDF-to-text converters detect strikethrough and miss insertion markup entirely. The metric wasn't measuring legislative aggression. It was measuring which converter had processed the file.
It was reverted, and the reasoning was written down next to the renderer so that a future engineer with the same good idea finds the answer before rebuilding it.
A confident, filterable, completely wrong signal is worse than no signal, because people will act on it.
The point-in-time barrier is the GM wall, wearing a different hat.
“What was law as of a given date”
A real retrieval filter with exactly the same shape as the Campaign lens's GM-only wall: out-of-scope material is removed before context assembly, not filtered out of the answer afterwards. One mechanism, two domains, zero duplicated logic — which is the lens thesis in a single feature.
Its limitation ships disclosed in the code rather than discovered later: without a reliable repeal date it can include something since repealed, but it will never exclude something that was in force. The failure mode is known and it is the safe direction.
The tree stops working at ten thousand documents
So at corpus scale it is replaced by search plus lens-declared facet filtering, served straight from the index with flat paging. Entirely lens-agnostic — it works the same way in a novel project with too many scenes.
A default that is a judgement call
Corpus export runs as a background job, and including individual legislators' names is off by default. Naming people as template distributors in a file that will travel is a deliberate act, so it requires a deliberate click.
The Legislation lens ships the corpus pipeline, the browser, the two analyses, the charts, the point-in-time barrier and export. It does not yet ship lens-specific validation, and it carries no chat commands of its own. It is a working third lens, not a finished product in its category — and the reason it exists is to prove the engine generalises.