What is in the corpus, how it was assembled, and the places where it is wrong or incomplete. Read this before you quote anything.
Last built —
Everything here is a document the New York City Council publishes itself, through Legistar,
the legislative records system at legistar.council.nyc.gov. Two kinds of document
matter:
Legistar's own data interface now requires an access token that is not issued to the public, so the record was read the way a person reads it: by walking the public calendar, meeting pages and agenda items and following the attachment links.
Every meeting on the Council's calendar for the 2024, 2025 and 2026 sessions was enumerated and crawled — 1,740 meetings — and every transcript and testimony file attached to them was collected. Two consequences worth understanding:
Land use and zoning subcommittee sessions, stated meetings of the full Council and committee hearings are all present. Stated meetings can be filtered in or out; they are mostly votes and procedural business, with a public comment period.
The same transcript is uploaded against every bill heard that day, and re-uploaded whenever a bill returns to committee, each time getting a new file identifier. The raw crawl produced many thousands of attachment links for a far smaller number of real documents. Duplicates were collapsed twice: first on kind, hearing date and exact byte size, then on the SHA-1 of the downloaded file — which removed a further — re-uploads that were byte-identical under different identifiers. What survives is — distinct documents.
Three things are excluded, and none of them silently:
The practical shape of this: the question-and-answer record is close to complete for the period covered, and the written testimony is partial, weighted towards the submissions that were filed as text rather than as scans.
Text is extracted with pypdf. The transcripts are text-native rather than
scanned, which is why they are searchable at all, but they extract badly by default: one word
per line, a running committee header on every page and the stenographer's footer. All of that
is stripped and the prose reflowed. Written testimony is ordinary prose and is reflowed
differently, preserving paragraph breaks.
Any document that failed to download, exceeded the size ceiling, or yielded no extractable text is excluded and counted rather than silently dropped.
The searchable unit is a passage. For a transcript, a passage is one speaker turn, cut at the
speaker labels the stenographer wrote (CHAIRPERSON LEE:,
COMMISSIONER TISCH:). A search hit is displayed with the turns either side of it,
which is what makes a question and its answer read as a single exchange.
For written testimony, a passage is roughly a paragraph. Where a packet announces each submission with a heading of the form "Testimony of …", it is split on those headings and the witness's name is taken from the heading. Where a packet carries no such headings, it is left whole rather than cut at guessed boundaries.
The Council files its work by committee, and committees do not map onto subjects. Housing comes up in Housing and Buildings, in Finance, in Land Use and in General Welfare; a policing question can surface at a stated meeting. So each hearing also carries one or more topics, assigned from two things the record already states: the committee that sat, and the subject line of what was on the agenda that day — Oversight – Implementation of the Pied-à-Terre Tax, or the plain-language description of a bill.
The committee half of that is deterministic: a fixed table maps each committee to a topic. The subject half is a set of keyword rules that can only ever add a topic, never remove one, which is why a hearing often sits under several. This grouping is ours, not the Council's. It is a way of finding things, not an official classification, and it will put the occasional hearing in a place you would not have chosen. The committee filter, underneath the topics, is the Council's own filing and is exact.
Agenda subjects come from the meeting's own item list. Where a hearing held an oversight item, that item names the hearing and is shown as its title. Where a hearing worked through a dozen bills, no title is shown at all — picking one item out of twelve reads as the hearing's subject when it is only whatever sorted first.
Two different kinds of error turn up here, and they are treated differently.
Our errors are corrected. Pulling text out of a PDF introduces damage that has nothing
to do with what anyone said: ligature glyphs, so benefits arrives as benefits and
cannot be searched for; a colon standing in for a ti pair, so information becomes
informa:on; soft hyphens left inside words. Every one of those repairs is checked
against a word list before it is applied — a colon only becomes ti where the result is
a real word, which is why about:blank is left alone. A couple of dozen documents
extracted with no word spaces at all, running whole paragraphs together; their spacing was
rebuilt by segmenting against the vocabulary of the rest of the corpus, and any run that could
not be resolved was left exactly as it was rather than guessed at.
The record's errors are not corrected. Stenographers misspell names, witnesses make typos, and those stand as written. The one exception is committee labels: a cover page typed Committee on Conumser and Worker Protection would otherwise appear as a separate committee in the filters, so committee names are snapped to the Council's own published list where the match is near-exact. That corrects a label against an authoritative source; it never touches the transcript text.
Roles are inferred from the title in the speaker label, not from any roster: Council member and Chairperson and Speaker mean a member of the Council; Commissioner, Chief, Deputy Mayor and the like mean the administration; Mr. and Ms. mean a member of the public. Committee counsel and the sergeant at arms are marked as staff.
Council members are shown by full name where the surname in the transcript matches the current Council roster, taken from Legistar. Members who have since left the Council keep the label the stenographer used. No name is inferred or filled in from anywhere else. The appearance rosters printed at the front of transcripts are not used: they are frequently incomplete, sometimes list people who were called but never testified, and parsing them reliably was not possible without occasionally attaching the wrong name to a quotation.
The index is a set of static files; there is no server and no query is sent anywhere. Passages
are ranked with BM25. A phrase in double quotation marks must match in order — word positions
are recorded across every token, including the ones too common to index, so
"right to counsel" matches the phrase and not the three words scattered about.
For a handful of terms that appear in a very large share of all passages, positions are not
stored, and a phrase containing one of them is matched as words instead; the results line says
so when that happens.
The whole vocabulary of the corpus ships as one file, which is what makes the looser matching possible without a server. An unquoted word is matched together with its plural and tense variants, but only where those variants are words that actually occur in the corpus — a real stemmer would turn housing into hous and produce matches nobody asked for. A word that does not occur at all is matched against everything within one or two character edits of it, so a misremembered spelling still finds the passage. Both behaviours are reported above the results, so you can always see which words a search actually matched, and both are switched off by Exact words only.
Quotation marks are the precise instrument: inside them there is no variant matching and no
fuzzy matching, only the literal words in the literal order. evict* matches
everything beginning that way, -word excludes, word~ forces a loose
match, and speaker:, role:, committee:,
after: and before: narrow the field. Speaker and role are stored in
the index as reserved terms, so filtering by them costs a lookup rather than a scan.
Type a question rather than keywords and it is read as a question. Nothing here is a language model: it is a set of rules that pick out the parts of a question the record actually knows about. What did the Mamdani administration say is the reason behind rising assaults? becomes administration witnesses, from January 2026 on, about assaults — and the reading is printed above the results, where you can override it and search the words literally instead.
Two things make the difference between finding an answer and missing it. The first is that a question is not an and of its words: someone asking about rising assaults will not find the official who said they were up 9 percent if every word is required. So the words a question is about are scored rather than demanded, widened with synonyms, and a passage ranks by how much of the question it covers. The second is that one word in a question is usually the subject and the rest is grammar — so the most distinctive word anchors the search and must appear. Without that anchor a passage containing "because" and "up" outranks the one that is actually about assaults.
Where this runs out: the rules only know the synonyms they have been given, and they cannot tell that "the bulk of those assaults have been on members of the service" is an explanation while a sentence merely containing the word "because" is not. It finds the passage; the reading is yours.
For a synthesised answer rather than a ranked list, take an extract into Google NotebookLM: download the results of a search from the front page, or a whole committee from the packs page. What comes back from NotebookLM is a model's reading of those documents, not the documents; follow its citation back before quoting.
Every passage links back to the Legistar file it came from. That file, not this page, is the record.
Generated with the help of AI. The transcripts and testimony are the Council's own documents, reproduced as published; the segmentation into speakers and exchanges is done by software and will sometimes be wrong. Read the source document before quoting anything.