The audit files

Methodology

What this tracker covers

Audits, investigations and oversight reports on New York City agencies, programs and spending issued on or after January 1, 2025 by official watchdogs:

• The New York City Comptroller's Office, whose Bureau of Audit examines city agencies under the City Charter. Source: the reports section of comptroller.nyc.gov, filtered to the audit report type. This includes documents the office labels "audit," "follow-up audit," "review" and "letter report." City comptroller audits are included in full, including its occasional audits of non-city entities such as the Metropolitan Transportation Authority, because the office's audit portfolio is defined by its city oversight role.
• The Office of the New York State Comptroller, which audits New York City agencies under the state constitution and state law. Source: the state agency audit listings at osc.ny.gov, filtered to audits whose audited entity is a New York City government agency or a city public benefit corporation.
• The New York City Department of Investigation, the city's inspector general, whose investigative and policy reports carry Policy and Procedure Recommendations. Source: nyc.gov/doi. Press releases announcing arrests or indictments without an accompanying report are not included.
• Court-appointed monitors overseeing city agencies under federal court orders and consent decrees, including the Floyd v. City of New York monitor over the Police Department's stop, question and frisk practices (nypdmonitor.org), the Nunez v. City of New York independent monitor and remediation manager over the city's jails (tillidgroup.com), the federal co-monitors appointed under the 2019 agreement between the United States Department of Housing and Urban Development, the city and the New York City Housing Authority (nychamonitor.com) and the special master and independent auditor in L.V. v. New York City Department of Education, who track whether the city carries out special education hearing orders (advocatesforchildren.org). Short procedural court filings are not included.
• Independent city oversight bodies that evaluate an agency they do not run and publish reports with recommendations: the New York City Board of Correction, the charter-mandated oversight board for the city jails (nyc.gov/boc), and the Commission to Combat Police Corruption, created by mayoral executive order in 1995 to evaluate the Police Department's anti-corruption systems (nyc.gov/ccpc).

What counts as "New York City-related"

For state comptroller audits, an audit is included when the audited entity is a New York City agency (for example the Department of Social Services or New York City Public Schools) or a city authority such as the New York City Housing Authority. State audits of state agencies and state authorities that operate in the city, including the Metropolitan Transportation Authority, are excluded because their spending is not spending by the city, as are state audits of programs located outside the five boroughs.

Outside groups, and why they are separate

Think tanks, research centers and civic organizations also tell New York City government what to do, often in more detail than any audit. Their proposals are in the tracker, but they are off by default and are added only when a reader turns on "Add outside groups' proposals."

The two are kept apart because they mean different things. The separation says nothing about quality. A comptroller finding rests on a legal mandate: the office has standing to examine the agency, the agency is obliged to answer, and for city comptroller audits an official tracker records whether the fix happened. An outside report rests on argument alone. No agency has to respond to it, and no one publishes whether it was adopted. Mixing them in one undifferentiated list would imply an obligation that does not exist, so outside items carry a distinct badge, never carry an implementation status, and say plainly on each card that no agency is required to respond.

What qualifies: white papers, reports, studies and substantial policy briefs that make recommendations to New York City government, published on or after January 1, 2025. Testimony, public comments, letters, bill memos, op-eds, press releases, blog posts and commentary do not qualify. That bar excludes a great deal of real advocacy work, including organizations whose recommendations reach government almost entirely through testimony, and it excludes most of the output of organizations that publish mainly essays and columns.

Organizations are included on the basis of what they publish, not what they argue. The tracker takes no position on the merits of any proposal and does not label organizations by ideology; each proposal is attributed to the organization that made it, and readers can weigh the source themselves.

Watchdogs that are not here, and why

• The Independent Budget Office publishes rigorous fiscal analysis of city government, but its charter role is nonpartisan analysis rather than recommendations: its reports present findings and options without telling agencies what to do, so there is nothing to track for implementation. Its work is at ibo.nyc.gov.
• The Public Advocate's office is an elected office whose policy department, in its own words, develops policy recommendations and legislative proposals. Its reports argue for positions rather than evaluate an agency from outside it, so they are advocacy rather than audit. Its reports are at advocate.nyc.gov.
• The Civilian Complaint Review Board investigates individual complaints against police officers and recommends discipline case by case. Its annual and semi-annual reports are statistical accounts of that caseload, not evaluations of a program with recommendations to fix it, and its last issue-based policy report was published in 2020. Its reports are at nyc.gov/ccrb.
• The Mayor's Office of Operations produces the charter-mandated Mayor's Management Report, a performance-indicator publication written by the mayor's own office about the agencies the mayor runs. It reports metrics rather than independent findings and makes no recommendations. It is at nyc.gov/operations.

How the data is gathered

Scrapers fetch each watchdog's public listing pages, follow each report to its official page or PDF and extract the report text. Dates, titles, agencies and links come directly from those pages. The pipeline is built to fail loudly: if a source returns zero reports or a report yields no text, the update stops rather than silently publishing incomplete data.

How the plain-language summaries are written

Summaries are AI-assisted, written only from the text of each official report under rules that bar adding any fact, name or figure not present in the source, and are verified afterward by a script that checks every quoted phrase and every dollar figure against the source text. Findings and recommendations are restatements, so wording differs from the official documents; the official report is always one click away and is the authoritative version.

Since October 2026, new reports can also be summarized automatically when the scheduled checks find them. The pipeline sends the report's text to Anthropic's Claude (the Sonnet 5.5 model) with the same written rules, then publishes the draft only if it clears every check: each quote and each listed figure must appear word for word in the report text, every figure in the plain-language prose must appear in the report text, field lengths and topics must follow the rules, a city comptroller audit's recommendation count must match the comptroller's own tracker when the audit is listed there, and a second, separate Claude pass must find no statement the report does not support, no missing or merged recommendation, no editorializing and no unexplained acronym. A draft that fails any check is not published; it is set aside for a person. The pipeline never decides on its own that an outside report should be left out; it only suggests that, and a person decides. Cards with an automated summary that no person has yet read against the report say so. The code is scripts/draft_summaries.py.

How implementation status is tracked

Whether a recommendation was actually carried out is reported differently by each watchdog, and the tracker keeps those meanings distinct:

• City comptroller audits: per-recommendation statuses (implemented, partially implemented, not implemented, pending) come from the comptroller's official Audit Recommendations Tracker, which covers audits issued since 2022. These statuses are agency-reported to the comptroller and shown with their as-of date. Audits not yet in the tracker show no status chips.
• State comptroller audits: the state comptroller reports implementation through follow-up audits. Where a follow-up exists, its implementation counts appear in the card's implementation section.
• Court monitors: monitors assess compliance with court orders rather than tracking discrete recommendations; each monitor card carries the report's overall compliance assessment.
• Department of Investigation: DOI does not publish a per-recommendation status feed; acceptance or implementation is noted only when a DOI report itself states it.

What "findings" and "recommendations" mean here

Counts shown in the tracker are counts of items in these summaries, not official tallies. Reviews and letter reports sometimes present key takeaways and suggestions rather than formal findings and recommendations; the tracker treats them the same way. Full reports contain more detail, sub-findings and formal agency response letters than any summary can carry.

Updates

An automated weekly check re-runs the comptroller scrapers. If new reports appear, they are summarized automatically where the drafts pass the checks described above, and flagged for a person otherwise; the "last updated" date on the front page reflects the most recent data build. Outside groups' publications are re-crawled twice a month in the same way. Department of Investigation and monitor reports are added on the same cadence as they are published.

Limitations

• The tracker starts January 1, 2025. Earlier reports, including those a follow-up refers back to, are linked from the follow-ups themselves.
• Only published reports appear. Investigations and audits in progress are not public.
• City comptroller implementation statuses are agency-reported and cover only audits in the comptroller's tracker; independent verification of a status is beyond this project's scope.
• Agency names follow each office's own attribution, lightly normalized to natural word order.
• If a source website changes its structure, updates may pause until the scraper is fixed. The tracker shows its last good build date rather than partial data.

← Back to the files