Congressional Appropriations Analyzer

congress-approp is a Rust CLI tool and library that downloads U.S. federal appropriations bills from Congress.gov, extracts every spending provision into structured JSON using Claude, and verifies each dollar amount against the source text. The included dataset covers 32 enacted bills across FY2019–FY2026 with 34,568 provisions and $21.5 trillion in budget authority.

Dollar amounts are verified by deterministic string matching against the enrolled bill text — no LLM in the verification loop. 99.995% of extracted dollar amounts are confirmed present in the source (18,583 of 18,584). Every provision carries a source_span with exact byte offsets into the enrolled bill for independent verification.

Jump straight to working examples: Recipes & Demos — track any federal account across fiscal years, compare subcommittees with inflation adjustment, load the data in Python, and more. No API keys needed.

What’s Included

This book ships with 32 enacted appropriations bills across 4 congresses (116th–119th), covering FY2019 through FY2026. All twelve appropriations subcommittees are represented for FY2020–FY2024 and FY2026. You don’t need any API keys to explore them — just install the tool and start querying.

116th Congress (FY2019–FY2021) — 11 bills

117th Congress (FY2021–FY2023) — 7 bills

118th Congress (FY2024/FY2025) — 10 bills

119th Congress (FY2025/FY2026) — 4 bills

Totals: 32 bills, 34,568 provisions, $21.5 trillion in budget authority, 1,051 accounts tracked by Treasury Account Symbol across FY2019–FY2026.

What Can You Do?

“How did THUD funding change from FY2024 to FY2026?”

congress-approp enrich --dir data                    # Generate metadata (once, no API key)
congress-approp compare --base-fy 2024 --current-fy 2026 --subcommittee thud --dir data

82 accounts matched across fiscal years — Tenant-Based Rental Assistance up $6.1B (+18.7%), Transit Formula Grants reclassified at $14.6B, Capital Investment Grants down $505M.

“What’s the FY2026 MilCon-VA budget, and how much is advance?”

congress-approp summary --dir data --fy 2026 --subcommittee milcon-va --show-advance

┌───────────┬────────────────┬────────────┬─────────────────┬─────────────────┬─────────────────┬─────────────────┬─────────────────┐
│ Bill      ┆ Classification ┆ Provisions ┆     Current ($) ┆     Advance ($) ┆    Total BA ($) ┆ Rescissions ($) ┆      Net BA ($) │
╞═══════════╪════════════════╪════════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╡
│ H.R. 5371 ┆ Minibus        ┆        257 ┆ 101,742,083,450 ┆ 393,689,946,000 ┆ 495,432,029,450 ┆  16,499,000,000 ┆ 478,933,029,450 │
└───────────┴────────────────┴────────────┴─────────────────┴─────────────────┴─────────────────┴─────────────────┴─────────────────┘

79.5% of MilCon-VA is advance appropriations for the next fiscal year — without --show-advance, you’d overstate current-year VA spending by $394 billion.

“Trace VA Compensation and Pensions across all fiscal years”

congress-approp relate 118-hr9468:0 --dir data --fy-timeline

Shows every matching provision across FY2024–FY2026 with current/advance/supplemental split, plus deterministic hashes you can save as persistent links for future comparisons.

“Find everything about FEMA disaster relief”

congress-approp search --dir data --semantic "FEMA disaster relief funding" --top 5

Finds FEMA provisions across 5 different bills by meaning, not just keywords — even when the bill text says “Federal Emergency Management Agency—Disaster Relief Fund” instead of “FEMA.”

Key Concepts

• enrich generates bill metadata offline (no API keys) — enabling fiscal year filtering, subcommittee scoping, and advance appropriation detection.
• --fy 2026 filters any command to bills covering that fiscal year.
• --subcommittee thud scopes to a specific appropriations jurisdiction, resolving division letters automatically (Division D in one bill, Division F in another — both map to THUD).
• --show-advance separates current-year spending from advance appropriations (money enacted now but available in a future fiscal year). Critical for year-over-year comparisons.
• relate traces one provision across all bills with a fiscal year timeline.
• link suggest / link accept persist cross-bill relationships so compare --use-links can handle renames automatically.

Navigating This Book

This book is organized so you can jump to whatever fits your needs:

• Recipes & Demos — Worked examples for account tracking, fiscal year comparisons, inflation adjustment, Python/pandas integration, and data export. Interactive visualizations included.
• Getting Started — Install the tool and run your first query in under five minutes. Covers installation and first commands.
• Getting to Know the Tool — Background reading on what this tool does, who it’s for, and a primer on how federal appropriations work if you’re new to the domain.
• Tutorials — Step-by-step walkthroughs for common tasks: finding spending on a topic, comparing bills, tracking programs, exporting data, and more.
• How-To Guides — Task-oriented recipes for specific operations like downloading bills, extracting provisions, and generating embeddings.
• Explanation — Deep dives into how the extraction pipeline, verification, semantic search, provision types, and budget authority calculation work under the hood.
• Reference — Lookup material: CLI commands, JSON field definitions, provision types, environment variables, data directory layout, and the glossary.
• For Contributors — Architecture overview, code map, and guides for adding new provision types, commands, and tests.

Version

This documentation covers congress-approp v6.0.0.

• GitHub: https://github.com/cgorski/congress-appropriations
• crates.io: https://crates.io/crates/congress-appropriations

What This Tool Does

The Problem

Every year, Congress passes appropriations bills authorizing roughly $1.7 trillion in discretionary spending — the money that funds federal agencies, military operations, scientific research, infrastructure, veterans’ benefits, and thousands of other programs. These bills run to approximately 1,500 pages annually, published as XML on Congress.gov.

The text is public, but it’s practically unsearchable at the provision level. If you want to know how much Congress appropriated for a specific program, you have three options:

1. Read the bill yourself. The FY2024 omnibus alone is over 1,800 pages of dense legislative text with nested cross-references, “of which” sub-allocations, and provisions scattered across twelve divisions.
2. Read CBO cost estimates or committee reports. These are expert summaries, but they aggregate — you get totals by title or account, not individual provisions. They also don’t cover every bill type the same way.
3. Search Congress.gov full text. You can find keywords, but you can’t filter by provision type, sort by dollar amount, or compare the same program across bills.

None of these let you ask structured questions like “show me every rescission over $10 million” or “which programs got a different amount in the continuing resolution than in the omnibus” or “find all provisions related to opioid treatment, including ones that don’t use the word ‘opioid.’”

What This Tool Does

congress-approp turns appropriations bill text into structured, queryable, verified data:

• Downloads enrolled bill XML from Congress.gov via its official API — the authoritative, machine-readable source
• Extracts every spending provision into structured JSON using Claude, capturing account names, dollar amounts, agencies, availability periods, provision types, section references, and more
• Verifies every dollar amount against the source text using deterministic string matching — no LLM in the verification loop
• Generates semantic embeddings for meaning-based search, enabling search by meaning rather than exact keywords
• Provides CLI query tools to search, compare, summarize, and audit provisions across any number of extracted bills

The Trust Model

LLM extraction is not infallible. This tool is designed around a simple principle: the LLM extracts once; deterministic code verifies everything.

The verification pipeline runs after extraction and checks every claim the LLM made against the source bill text. No language model is involved in verification — it’s pure string matching with tiered fallback (exact → normalized → spaceless). The result across the included dataset:

Every extracted dollar amount can be traced back to an exact byte position in the enrolled bill text. The audit command shows this verification breakdown for any set of bills. If a number can’t be verified, it’s flagged — not silently accepted. For the full breakdown, see Accuracy Metrics.

The ~5% of provisions where raw_text isn’t a byte-identical substring are cases where the LLM truncated a very long provision or normalized whitespace. The verify-text command repairs these deterministically — and the dollar amounts in those provisions are still independently verified.

What’s Included

The tool ships with 32 enacted appropriations bills across 4 congresses (116th–119th), covering FY2019 through FY2026. Every major bill type is represented — omnibus, minibus, continuing resolutions, supplementals, and authorizations. See the Recipes & Demos page for the full bill inventory, or run congress-approp summary --dir data to see them all.

Each bill directory includes the source XML, extracted provisions (extraction.json), verification report, extraction metadata, TAS mapping, bill metadata, and pre-computed embeddings. No API keys are required to query this data.

Five Things You Can Do Right Now

All of these work immediately with the included example data — no API keys needed.

1. See budget totals for all included bills:

congress-approp summary --dir data

Shows each bill’s provision count, gross budget authority, rescissions, and net budget authority in a formatted table.

2. Search all appropriations provisions:

congress-approp search --dir data --type appropriation

Lists every appropriation-type provision across all bills with account name, amount, division, and agency.

3. Find FEMA funding:

congress-approp search --dir data --keyword "Federal Emergency Management"

Searches provision text for any mention of FEMA across all bills.

4. See what the continuing resolution changed:

congress-approp search --dir data/118-hr5860 --type cr_substitution

Shows the 13 “anomalies” — programs where the CR set a different funding level instead of continuing at the prior-year rate.

5. Audit verification status:

congress-approp audit --dir data

Displays a detailed verification breakdown for each bill: how many dollar amounts were verified, how many raw text excerpts matched the source, and the completeness coverage metric.

Who This Is For

congress-approp is built for anyone who needs to work with the details of federal appropriations bills — not just the headline numbers, but the individual provisions. This chapter describes five audiences and how each can get the most out of the tool.

Journalists & Policy Researchers

What you’d use this for:

• Fact-checking spending claims. A press release says “Congress cut Program X by 15%.” You can pull up every provision mentioning that program, compare the dollar amounts to the prior year’s bill, and confirm or refute the claim against the enrolled bill text — not a summary or a committee report, but the law itself.
• Comparing spending across fiscal years. “How did THUD funding change from FY2024 to FY2026?” Use compare --base-fy 2024 --current-fy 2026 --subcommittee thud and get a per-account comparison: Tenant-Based Rental Assistance up $6.1B (+18.7%), Capital Investment Grants down $505M. No need to know which bills or divisions to look at — the tool resolves that automatically.
• Finding provisions by topic. You’re writing a story about opioid treatment funding. Semantic search finds relevant provisions even when the bill text says “Substance Use Treatment and Prevention” instead of “opioid.” Combine with --fy 2026 --subcommittee labor-hhs to scope results to a specific year and jurisdiction.
• Separating advance from current-year spending. 79.5% of MilCon-VA budget authority is advance appropriations for the next fiscal year. Without --show-advance, a reporter comparing year-over-year VA spending would be off by hundreds of billions of dollars. The tool flags this automatically.
• Tracing a program across all bills. Use relate 118-hr9468:0 --fy-timeline to see VA Compensation and Pensions across FY2024–FY2026, with current/advance/supplemental split per year and links to every matching provision.

Start here: Getting Started → Find Spending on a Topic → Compare Two Bills → Enrich Bills with Metadata

API keys needed: None for querying pre-extracted example data (including FY filtering, subcommittee scoping, advance splits, and relate). OPENAI_API_KEY if you want semantic (meaning-based) search. CONGRESS_API_KEY + ANTHROPIC_API_KEY if you want to download and extract additional bills yourself.

Congressional Staffers & Analysts

What you’d use this for:

• Tracking program funding across bills. Use relate to trace a specific account — say, VA Compensation and Pensions — across all 14 bills with a fiscal year timeline showing the current-year, advance, and supplemental split. Save the matches as persistent links with link accept so you can reuse them in future comparisons.
• Subcommittee-level analysis. “What’s the FY2026 Defense budget?” Use summary --fy 2026 --subcommittee defense and get $836B in budget authority from H.R. 7148 Division A. The tool maps division letters to canonical jurisdictions automatically — Division A means Defense in H.R. 7148 but CJS in H.R. 6938.
• Identifying CR anomalies. Continuing resolutions fund the government at prior-year rates except for specific anomalies. The tool extracts every cr_substitution as structured data so you can see exactly which programs got different treatment: congress-approp search --dir data/118-hr5860 --type cr_substitution.
• Enriched bill classifications. The tool distinguishes omnibus (5+ subcommittees), minibus (2–4), full-year CR with appropriations (like H.R. 1968 with $1.786T in appropriations alongside a CR mechanism), and supplementals — not just the raw LLM classification.
• Exporting for briefings and spreadsheets. Every query command supports --format csv output. Pipe it to a file and open it in Excel: congress-approp compare --base-fy 2024 --current-fy 2026 --subcommittee thud --dir data --format csv > thud_compare.csv.

Start here: Getting Started → Compare Two Bills → Enrich Bills with Metadata → Track a Program Across Bills

API keys needed: None for querying pre-extracted data (including FY filtering, subcommittee scoping, advance splits, relate, and link management). Most staffers won’t need to run extractions themselves — the included example data covers 13 enacted bills across FY2024–FY2026.

Data Scientists & Developers

What you’d use this for:

• Building dashboards and visualizations. The --format json and --format jsonl output modes give you machine-readable provision data ready for ingestion into dashboards, notebooks, or databases. Every provision includes structured fields for amount, agency, account, division, section, provision type, and more.
• Integrating into data pipelines. congress-approp is both a CLI tool and a Rust library (congress_appropriations). You can call it from scripts via the CLI or embed it directly in Rust projects via the library API. The JSON schema is stable within major versions.
• Extending with new provision types or analysis. The extraction schema supports 11 provision types today. If you need to capture something new — say, a specific category of earmark or a new kind of spending limitation — the Adding a New Provision Type guide walks you through it.

Start here: Getting Started → Export Data for Spreadsheets and Scripts → Use the Library API from Rust → Architecture Overview

API keys needed: Depends on your workflow. None for querying existing extractions. OPENAI_API_KEY for generating embeddings (semantic search). CONGRESS_API_KEY + ANTHROPIC_API_KEY for downloading and extracting new bills.

Auditors & Oversight Staff

What you’d use this for:

• Validating extracted numbers. The audit command gives you a per-bill breakdown of verification status: how many dollar amounts were found in the source text, how many raw text excerpts matched byte-for-byte, and a completeness metric showing what percentage of dollar strings in the source were accounted for. Across the included dataset, 99.995% of dollar amounts are verified against the source text. See Accuracy Metrics for the full breakdown.
• Assessing extraction completeness. The verification report flags any dollar amount that appears in the source XML but isn’t captured by an extracted provision. A completeness percentage below 100% doesn’t necessarily indicate a missed provision — many dollar strings in bill text are statutory cross-references, loan guarantee ceilings, or old amounts being struck by amendments — but it gives you a starting point for investigation.
• Tracing numbers to source. Every verified dollar amount includes a character position in the source text. Every provision includes raw_text that can be matched against the bill XML. You can independently confirm any number the tool reports by opening the source file and checking the indicated position.

Start here: Getting Started → Verify Extraction Accuracy → LLM Reliability and Guardrails

API keys needed: None. All verification and audit operations work entirely offline against already-extracted data.

Contributors

What you’d use this for:

• Adding features. The tool is open source under MIT/Apache-2.0. Whether you want to add a new CLI subcommand, support a new bill format, or improve the extraction prompt, the contributor guides walk you through the codebase and conventions.
• Fixing bugs. The Testing Strategy chapter explains how the test suite is structured — including golden-file tests against the example bills — so you can reproduce issues and verify fixes.
• Understanding the architecture. The Architecture Overview and Code Map chapters explain how the pipeline stages connect, where each module lives, and how data flows from XML download through LLM extraction and verification to query output.

Start here: Architecture Overview → Code Map → Testing Strategy → Style Guide and Conventions

API keys needed: CONGRESS_API_KEY + ANTHROPIC_API_KEY if you’re working on download or extraction features. OPENAI_API_KEY if you’re working on embedding or semantic search features. None if you’re working on query, verification, or CLI features — the example data is sufficient.

How Federal Appropriations Work

This chapter covers the essentials of federal appropriations — fiscal years, bill types, provision structure, and key terminology. Readers already familiar with the appropriations process can skip to the tutorials.

The Federal Budget in 60 Seconds

The U.S. federal government spends roughly $6.7 trillion per year. That breaks down into three major categories:

This tool covers the 26% — discretionary spending — plus certain mandatory spending lines that appear as appropriation provisions in the bill text (for example, SNAP funding appears as a line item in the Agriculture appropriations division even though it’s technically mandatory spending). That’s why the budget authority total for H.R. 4366 is ~$846 billion, not the ~$1.7 trillion figure you’ll sometimes see for total discretionary spending (which includes all twelve bills plus defense), and certainly not the ~$6.7 trillion total federal budget.

The Fiscal Year

The federal fiscal year runs from October 1 through September 30. It’s named for the calendar year in which it ends, not the one in which it begins. So:

• FY2024 = October 1, 2023 – September 30, 2024
• FY2025 = October 1, 2024 – September 30, 2025

Bills are labeled by the fiscal year they fund, not the calendar year they were enacted in. The Consolidated Appropriations Act, 2024 (H.R. 4366) was signed into law on March 23, 2024 — nearly six months into the fiscal year it was supposed to fund from the start.

The Twelve Appropriations Bills

Each year, Congress is supposed to pass twelve individual appropriations bills, one for each subcommittee of the House and Senate Appropriations Committees:

1. Agriculture, Rural Development, FDA
2. Commerce, Justice, Science (CJS)
3. Defense
4. Energy and Water Development
5. Financial Services and General Government
6. Homeland Security
7. Interior, Environment
8. Labor, Health and Human Services, Education (Labor-HHS)
9. Legislative Branch
10. Military Construction, Veterans Affairs (MilCon-VA)
11. State, Foreign Operations
12. Transportation, Housing and Urban Development (THUD)

In practice, Congress rarely passes all twelve on time. Instead, it bundles them:

• An omnibus packages all (or nearly all) twelve bills into a single piece of legislation.
• A minibus bundles a few of the twelve together.
• Individual bills are occasionally passed on their own, but this has become increasingly rare.

When none of the twelve are done by October 1, Congress passes a continuing resolution to keep the government funded temporarily while it finishes negotiations.

Bill Types

The included dataset covers 32 enacted appropriations bills spanning all major bill types. Here’s what each one is, with the real example from this tool:

Regular / Omnibus

A regular appropriations bill provides new funding for one of the twelve subcommittee jurisdictions for the coming fiscal year. An omnibus combines multiple regular bills into one legislative vehicle, organized into lettered divisions (Division A, Division B, etc.). H.R. 4366, the Consolidated Appropriations Act, 2024, is an omnibus covering MilCon-VA, Agriculture, CJS, Energy-Water, Interior, THUD, and other matters across multiple divisions. It contains 2,364 provisions and authorizes $846 billion in budget authority.

Continuing Resolution

A continuing resolution (CR) provides temporary funding — usually at the prior fiscal year’s rate — for agencies whose regular appropriations bills haven’t been enacted yet. Most provisions in a CR simply say “continue at last year’s level,” but specific programs may get different treatment through anomalies (formally called CR substitutions). H.R. 5860, the Continuing Appropriations Act, 2024, contains 130 provisions including 13 CR substitutions — programs where Congress set a specific dollar amount rather than defaulting to the prior-year rate. It also includes mandatory spending extensions and other legislative riders.

Supplemental

A supplemental appropriation provides additional funding outside the regular annual cycle, typically in response to emergencies — natural disasters, military operations, public health crises, or (in this case) an unexpected funding shortfall. H.R. 9468, the Veterans Benefits Continuity and Accountability Supplemental Appropriations Act, 2024, contains 7 provisions providing $2.9 billion for VA Compensation and Pensions and Readjustment Benefits, plus reporting requirements and an Inspector General review.

Rescissions

A rescission bill cancels previously enacted budget authority. Rescissions also appear as individual provisions within larger bills — H.R. 4366 includes $24.7 billion in rescissions alongside its new appropriations.

Anatomy of a Provision

To see how bill text becomes structured data, let’s walk through a real example from H.R. 9468. Here’s what Congress wrote:

For an additional amount for ’‘Compensation and Pensions’’, $2,285,513,000, to remain available until expended.

And here is the structured JSON that congress-approp extracted from that sentence:

{
  "provision_type": "appropriation",
  "agency": "Department of Veterans Affairs",
  "account_name": "Compensation and Pensions",
  "amount": {
    "value": { "kind": "specific", "dollars": 2285513000 },
    "semantics": "new_budget_authority",
    "text_as_written": "$2,285,513,000"
  },
  "detail_level": "top_level",
  "availability": "to remain available until expended",
  "fiscal_year": 2024,
  "raw_text": "For an additional amount for ''Compensation and Pensions'', $2,285,513,000, to remain available until expended.",
  "confidence": 0.99
}

Here’s what each piece means:

• account_name: Pulled from the double-quoted name in the bill text (the ''Compensation and Pensions'' delimiters are a legislative drafting convention).
• amount: The dollar value is parsed to an integer (2285513000), the original text is preserved ("$2,285,513,000"), and the meaning is classified — this is new_budget_authority, meaning Congress is granting new spending authority, not referencing an existing amount.
• detail_level: This is a top_level appropriation — the full amount for the account, not a sub-allocation (“of which $X for Y”).
• availability: Captured from the bill text. “To remain available until expended” means this is no-year money — the agency can spend it over multiple fiscal years, unlike annual funds that expire at the end of the fiscal year.
• raw_text: The original bill text, verified against the source XML.
• Verification: The string $2,285,513,000 was found at character position 431 in the source XML. The raw_text is a byte-identical substring of the source starting at position 371.

Key Concepts

Budget Authority vs. Outlays

Budget authority (BA) is what Congress authorizes — the legal permission for agencies to enter into obligations (sign contracts, award grants, hire staff). Outlays are what the Treasury actually disburses. The two differ because agencies often obligate funds in one year but spend them over several years (especially for construction, procurement, and multi-year grants).

This tool reports budget authority, because that’s what the bill text specifies. When you see “$846B” for H.R. 4366, that’s the sum of new_budget_authority provisions at the top_level and line_item detail levels — what Congress authorized, not what agencies will spend this year.

Sub-Allocations Are Not Additional Money

Many provisions include “of which” clauses: “For the Office of Science, $8,220,000,000, of which $300,000,000 shall be for fusion energy research.” The $300 million is a sub-allocation — a directive about how to spend part of the $8.2 billion, not money on top of it. The tool captures sub-allocations at detail_level: "sub_allocation" and correctly excludes them from budget authority totals to avoid double-counting.

Advance Appropriations

Sometimes Congress enacts budget authority in this year’s bill but makes it available starting in the next fiscal year. These advance appropriations are included in the bill’s budget authority total (because the bill does enact them) but are noted in the provision’s notes field.

Congress Numbers

Each Congress spans two calendar years. The 118th Congress served from January 2023 through January 2025; the 119th Congress runs from January 2025 through January 2027. Bills are identified by their Congress — H.R. 4366 of the 118th Congress is an entirely different bill from H.R. 4366 of any other Congress. All three example bills in this tool are from the 118th Congress.

Essential Glossary

These five terms come up throughout the book. A comprehensive glossary is available in the Glossary reference chapter.

Installation

You will need: A computer running macOS or Linux, and an internet connection.

You will learn: How to install congress-approp and verify it’s working.

Install Rust

congress-approp is written in Rust and requires Rust 1.93 or later. If you don’t have Rust installed, the easiest way is via rustup:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

If you already have Rust, make sure it’s up to date:

rustup update

Verify your version:

rustc --version
# Should show 1.93.0 or later

Install from Source (Recommended)

Cloning the repository gives you the full dataset — 32 enacted appropriations bills (FY2019–FY2026) with pre-computed embeddings, ready to query with no API keys.

git clone https://github.com/cgorski/congress-appropriations.git
cd congress-appropriations
cargo install --path .

This compiles the project and places the congress-approp binary on your PATH. The first build takes a few minutes; subsequent builds are much faster.

Install from crates.io

If you just want the binary without cloning the full repository:

cargo install congress-appropriations

Note: The crates.io package does not include the data/ directory or pre-computed embedding vectors because they exceed the crates.io upload limit. If you install via crates.io, clone the repository separately to get the dataset, or download and extract your own bills.

Verify the Installation

Run the summary command against the included data:

congress-approp summary --dir data

You should see a table listing all 32 bills with their provision counts, budget authority, and rescissions. The last line confirms data integrity:

0 dollar amounts unverified across all bills. Run `congress-approp audit` for detailed verification.

If you see 32 bills and 34,568 total provisions across FY2019–FY2026, everything is working. You’re ready to start querying.

Tip: If you’re running from the cloned repo directory, data is a relative path that points to the included dataset. If you installed via cargo install and are running from a different directory, provide the full path to the data/ directory inside your clone.

API Keys (Optional)

No API keys are needed to query the pre-extracted dataset. Keys are only required if you want to download new bills, extract provisions from them, or use semantic search:

Set them in your shell when needed:

export CONGRESS_API_KEY="your-key-here"
export ANTHROPIC_API_KEY="your-key-here"
export OPENAI_API_KEY="your-key-here"

See Environment Variables and API Keys for details.

Rebuilding After Source Changes

If you modify the source code (or pull updates), rebuild and reinstall with:

cargo install --path .

For development iteration without reinstalling:

cargo build --release
./target/release/congress-approp summary --dir data

Next Steps

Next: Your First Query.

Your First Query

You will need: congress-approp installed (Installation), access to the data/ directory from the cloned repository.

You will learn: How to explore the included FY2024 appropriations data using five core commands — no API keys required.

This chapter walks through five core commands using the included dataset. Every command shown here produces output you can verify against the data files.

Step 1: See What Bills You Have

Start with the summary command to get an overview:

congress-approp summary --dir data

┌───────────┬───────────────────────┬────────────┬─────────────────┬─────────────────┬─────────────────┐
│ Bill      ┆ Classification        ┆ Provisions ┆ Budget Auth ($) ┆ Rescissions ($) ┆      Net BA ($) │
╞═══════════╪═══════════════════════╪════════════╪═════════════════╪═════════════════╪═════════════════╡
│ H.R. 4366 ┆ Omnibus               ┆       2364 ┆ 846,137,099,554 ┆  24,659,349,709 ┆ 821,477,749,845 │
│ H.R. 5860 ┆ Continuing Resolution ┆        130 ┆  16,000,000,000 ┆               0 ┆  16,000,000,000 │
│ H.R. 9468 ┆ Supplemental          ┆          7 ┆   2,882,482,000 ┆               0 ┆   2,882,482,000 │
│ TOTAL     ┆                       ┆       2501 ┆ 865,019,581,554 ┆  24,659,349,709 ┆ 840,360,231,845 │
└───────────┴───────────────────────┴────────────┴─────────────────┴─────────────────┴─────────────────┘

0 dollar amounts unverified across all bills. Run `congress-approp audit` for detailed verification.

Here’s what each column means:

The footer line — “0 dollar amounts unverified” — tells you that every extracted dollar amount was confirmed to exist in the source bill text. This is the headline trust metric.

Step 2: Search for Provisions

The search command finds provisions matching your criteria. Let’s start broad — all appropriation-type provisions across all bills:

congress-approp search --dir data --type appropriation

This returns a table with hundreds of rows. Let’s narrow it down. Find all provisions mentioning FEMA:

congress-approp search --dir data --keyword "Federal Emergency Management"

┌───┬───────────┬───────────────┬───────────────────────────────────────────────┬────────────────┬──────────┬─────┐
│ $ ┆ Bill      ┆ Type          ┆ Description / Account                         ┆     Amount ($) ┆ Section  ┆ Div │
╞═══╪═══════════╪═══════════════╪═══════════════════════════════════════════════╪════════════════╪══════════╪═════╡
│   ┆ H.R. 5860 ┆ other         ┆ Allows FEMA Disaster Relief Fund to be appor… ┆              — ┆ SEC. 128 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ appropriation ┆ Federal Emergency Management Agency—Disast…   ┆ 16,000,000,000 ┆ SEC. 129 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ appropriation ┆ Office of the Inspector General—Operations…   ┆      2,000,000 ┆ SEC. 129 ┆ A   │
└───┴───────────┴───────────────┴───────────────────────────────────────────────┴────────────────┴──────────┴─────┘
3 provisions found

$ = Amount status: ✓ found (unique), ≈ found (multiple matches), ✗ not found

Understanding the $ column — the verification status for each provision’s dollar amount:

Now try searching by account name. This matches against the structured account_name field rather than searching the full text:

congress-approp search --dir data --account "Child Nutrition"

┌───┬───────────┬───────────────┬─────────────────────────────────────────────┬────────────────┬─────────┬─────┐
│ $ ┆ Bill      ┆ Type          ┆ Description / Account                       ┆     Amount ($) ┆ Section ┆ Div │
╞═══╪═══════════╪═══════════════╪═════════════════════════════════════════════╪════════════════╪═════════╪═════╡
│ ✓ ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆ 33,266,226,000 ┆         ┆ B   │
│ ✓ ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆     18,004,000 ┆         ┆ B   │
│ ...                                                                                                          │
└───┴───────────┴───────────────┴─────────────────────────────────────────────┴────────────────┴─────────┴─────┘

The top result — $33.27 billion for Child Nutrition Programs — is the top-level appropriation. The smaller amounts below it are sub-allocations and reference amounts within the same account.

You can combine filters. For example, find all appropriations over $1 billion in Division A (MilCon-VA):

congress-approp search --dir data/118-hr4366 --type appropriation --division A --min-dollars 1000000000

Step 3: Look at the VA Supplemental

The smallest bill, H.R. 9468, is a good place to see the full picture. It has only 7 provisions:

congress-approp search --dir data/118-hr9468

┌───┬───────────┬───────────────┬───────────────────────────────────────────────┬───────────────┬──────────┬─────┐
│ $ ┆ Bill      ┆ Type          ┆ Description / Account                         ┆    Amount ($) ┆ Section  ┆ Div │
╞═══╪═══════════╪═══════════════╪═══════════════════════════════════════════════╪═══════════════╪══════════╪═════╡
│ ✓ ┆ H.R. 9468 ┆ appropriation ┆ Compensation and Pensions                     ┆ 2,285,513,000 ┆          ┆     │
│ ✓ ┆ H.R. 9468 ┆ appropriation ┆ Readjustment Benefits                         ┆   596,969,000 ┆          ┆     │
│   ┆ H.R. 9468 ┆ rider         ┆ Establishes that each amount appropriated o…  ┆             — ┆ SEC. 101 ┆     │
│   ┆ H.R. 9468 ┆ rider         ┆ Unless otherwise provided, the additional a…  ┆             — ┆ SEC. 102 ┆     │
│   ┆ H.R. 9468 ┆ directive     ┆ Requires the Secretary of Veterans Affairs …  ┆             — ┆ SEC. 103 ┆     │
│   ┆ H.R. 9468 ┆ directive     ┆ Requires the Secretary of Veterans Affairs …  ┆             — ┆ SEC. 103 ┆     │
│   ┆ H.R. 9468 ┆ directive     ┆ Requires the Inspector General of the Depar…  ┆             — ┆ SEC. 104 ┆     │
└───┴───────────┴───────────────┴───────────────────────────────────────────────┴───────────────┴──────────┴─────┘
7 provisions found

This is the complete bill: two appropriations ($2.3B for Comp & Pensions, $597M for Readjustment Benefits), two policy riders (SEC. 101 and 102 establishing that these amounts are additional to regular appropriations), and three directives requiring the VA Secretary and Inspector General to submit reports about the funding shortfall that necessitated this supplemental.

Notice how the two appropriations have ✓ in the dollar column, while the riders and directives show no symbol — they don’t carry dollar amounts, so there’s nothing to verify.

Step 4: See What the CR Changed

Continuing resolutions normally fund agencies at prior-year rates, but specific programs can get different treatment through “anomalies” — formally called CR substitutions. These are provisions that say “substitute $X for $Y,” setting a new level instead of continuing the old one.

congress-approp search --dir data/118-hr5860 --type cr_substitution

┌───┬───────────┬──────────────────────────────────────────┬───────────────┬───────────────┬──────────────┬──────────┬─────┐
│ $ ┆ Bill      ┆ Account                                  ┆       New ($) ┆       Old ($) ┆    Delta ($) ┆ Section  ┆ Div │
╞═══╪═══════════╪══════════════════════════════════════════╪═══════════════╪═══════════════╪══════════════╪══════════╪═════╡
│ ✓ ┆ H.R. 5860 ┆ Rural Housing Service—Rural Community…   ┆    25,300,000 ┆    75,300,000 ┆  -50,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Rural Utilities Service—Rural Water a…   ┆    60,000,000 ┆   325,000,000 ┆ -265,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆                                          ┆   122,572,000 ┆   705,768,000 ┆ -583,196,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ National Science Foundation—STEM Educ…   ┆    92,000,000 ┆   217,000,000 ┆ -125,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ National Oceanic and Atmospheric Admini… ┆    42,000,000 ┆    62,000,000 ┆  -20,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ National Science Foundation—Research …   ┆   608,162,000 ┆   818,162,000 ┆ -210,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Department of State—Administration of…   ┆    87,054,000 ┆   147,054,000 ┆  -60,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Bilateral Economic Assistance—Funds A…   ┆   637,902,000 ┆   937,902,000 ┆ -300,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Bilateral Economic Assistance—Departm…   ┆   915,048,000 ┆ 1,535,048,000 ┆ -620,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ International Security Assistance—Dep…   ┆    74,996,000 ┆   374,996,000 ┆ -300,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Office of Personnel Management—Salari…   ┆   219,076,000 ┆   190,784,000 ┆  +28,292,000 ┆ SEC. 126 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Department of Transportation—Federal …   ┆   617,000,000 ┆   570,000,000 ┆  +47,000,000 ┆ SEC. 137 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Department of Transportation—Federal …   ┆ 2,174,200,000 ┆ 2,221,200,000 ┆  -47,000,000 ┆ SEC. 137 ┆ A   │
└───┴───────────┴──────────────────────────────────────────┴───────────────┴───────────────┴──────────────┴──────────┴─────┘
13 provisions found

Notice how the table automatically changes shape for CR substitutions — it shows New, Old, and Delta columns instead of a single Amount. This tells you exactly which programs Congress funded above or below the prior-year rate:

• Most programs were cut: Migration and Refugee Assistance lost $620 million (-40.4%), NSF Research lost $210 million (-25.7%)
• Two programs increased: OPM Salaries and Expenses gained $28 million (+14.8%) and FAA Facilities and Equipment gained $47 million (+8.2%)
• Every dollar amount has ✓ — both the new and old amounts were verified in the source text

Step 5: Check Data Quality

The audit command shows how well the extraction held up against the source text:

congress-approp audit --dir data

┌───────────┬────────────┬──────────┬──────────┬───────┬───────┬──────────┬───────────┬──────────┬──────────┐
│ Bill      ┆ Provisions ┆ Verified ┆ NotFound ┆ Ambig ┆ Exact ┆ NormText ┆ Spaceless ┆ TextMiss ┆ Coverage │
╞═══════════╪════════════╪══════════╪══════════╪═══════╪═══════╪══════════╪═══════════╪══════════╪══════════╡
│ H.R. 4366 ┆       2364 ┆      762 ┆        0 ┆   723 ┆  2285 ┆       59 ┆         0 ┆       20 ┆    94.2% │
│ H.R. 5860 ┆        130 ┆       33 ┆        0 ┆     2 ┆   102 ┆       12 ┆         0 ┆       16 ┆    61.1% │
│ H.R. 9468 ┆          7 ┆        2 ┆        0 ┆     0 ┆     5 ┆        0 ┆         0 ┆        2 ┆   100.0% │
│ TOTAL     ┆       2501 ┆      797 ┆        0 ┆   725 ┆  2392 ┆       71 ┆         0 ┆       38 ┆          │
└───────────┴────────────┴──────────┴──────────┴───────┴───────┴──────────┴───────────┴──────────┴──────────┘

The key number: NotFound = 0 for every bill. Every dollar amount the tool extracted actually exists in the source bill text. Here’s a quick guide to the other columns:

For a deeper dive into what these numbers mean, see Verify Extraction Accuracy and What Coverage Means.

Step 6: Export to JSON

Every command supports --format json for machine-readable output. This is useful for piping to jq, loading into Python, or just seeing the full data:

congress-approp search --dir data/118-hr9468 --type appropriation --format json

[
  {
    "account_name": "Compensation and Pensions",
    "agency": "Department of Veterans Affairs",
    "amount_status": "found",
    "bill": "H.R. 9468",
    "description": "Compensation and Pensions",
    "division": "",
    "dollars": 2285513000,
    "match_tier": "exact",
    "old_dollars": null,
    "provision_index": 0,
    "provision_type": "appropriation",
    "quality": "strong",
    "raw_text": "For an additional amount for ''Compensation and Pensions'', $2,285,513,000, to remain available until expended.",
    "section": "",
    "semantics": "new_budget_authority"
  },
  {
    "account_name": "Readjustment Benefits",
    "agency": "Department of Veterans Affairs",
    "amount_status": "found",
    "bill": "H.R. 9468",
    "description": "Readjustment Benefits",
    "division": "",
    "dollars": 596969000,
    "match_tier": "exact",
    "old_dollars": null,
    "provision_index": 1,
    "provision_type": "appropriation",
    "quality": "strong",
    "raw_text": "For an additional amount for ''Readjustment Benefits'', $596,969,000, to remain available until expended.",
    "section": "",
    "semantics": "new_budget_authority"
  }
]

The JSON output includes every field for each provision — more detail than the table can show. Key fields to know:

• dollars: The dollar amount as an integer (no formatting)
• semantics: What the amount means — new_budget_authority counts toward budget totals
• raw_text: The verbatim excerpt from the bill text
• match_tier: How closely raw_text matched the source — exact means byte-identical
• quality: Overall quality assessment — strong, moderate, or weak
• provision_index: Position in the bill’s provision list (useful for --similar searches)

Other output formats are also available: --format csv for spreadsheets, --format jsonl for streaming one-object-per-line output. See Output Formats for details.

Enrich for Fiscal Year and Subcommittee Filtering

The example data includes pre-enriched metadata, but if you extract your own bills, run enrich to enable fiscal year and subcommittee filtering:

congress-approp enrich --dir data      # No API key needed — runs offline

Once enriched, you can scope any command to a specific fiscal year and subcommittee:

# FY2026 THUD subcommittee only
congress-approp summary --dir data --fy 2026 --subcommittee thud

# See advance vs current-year spending
congress-approp summary --dir data --fy 2026 --subcommittee milcon-va --show-advance

# Compare THUD across fiscal years
congress-approp compare --base-fy 2024 --current-fy 2026 --subcommittee thud --dir data

# Trace one provision across all bills
congress-approp relate 118-hr9468:0 --dir data --fy-timeline

See Enrich Bills with Metadata for the full guide.

What’s Next

Related chapters:

• Want to filter by fiscal year or subcommittee? → Enrich Bills with Metadata
• Want to find specific spending? → Find How Much Congress Spent on a Topic
• Want to compare bills across fiscal years? → Compare Two Bills
• Want to track a program across all bills? → Track a Program Across Bills
• Want to export data to Excel or Python? → Export Data for Spreadsheets and Scripts
• Want to understand the output better? → Understanding the Output (next chapter)
• Want to extract your own bills? → Extract Your Own Bill
• Want to search by meaning instead of keywords? → Use Semantic Search

Understanding the Output

You will need: congress-approp installed, access to the data/ directory.

You will learn: How to read every table the tool produces — what each column means, what the symbols indicate, and how to interpret the numbers.

Before diving into tutorials and specific tasks, let’s build a solid understanding of the output formats you’ll encounter. Every command in congress-approp uses consistent conventions, but the tables adapt their shape depending on what you’re looking at.

The Summary Table

The summary command gives you the bird’s-eye view:

congress-approp summary --dir data

┌───────────┬───────────────────────┬────────────┬─────────────────┬─────────────────┬─────────────────┐
│ Bill      ┆ Classification        ┆ Provisions ┆ Budget Auth ($) ┆ Rescissions ($) ┆      Net BA ($) │
╞═══════════╪═══════════════════════╪════════════╪═════════════════╪═════════════════╪═════════════════╡
│ H.R. 4366 ┆ Omnibus               ┆       2364 ┆ 846,137,099,554 ┆  24,659,349,709 ┆ 821,477,749,845 │
│ H.R. 5860 ┆ Continuing Resolution ┆        130 ┆  16,000,000,000 ┆               0 ┆  16,000,000,000 │
│ H.R. 9468 ┆ Supplemental          ┆          7 ┆   2,882,482,000 ┆               0 ┆   2,882,482,000 │
│ TOTAL     ┆                       ┆       2501 ┆ 865,019,581,554 ┆  24,659,349,709 ┆ 840,360,231,845 │
└───────────┴───────────────────────┴────────────┴─────────────────┴─────────────────┴─────────────────┘

0 dollar amounts unverified across all bills. Run `congress-approp audit` for detailed verification.

Column-by-column

The footer

The line below the table — “0 dollar amounts unverified across all bills” — is a quick trust check. It counts provisions across all loaded bills where the dollar amount string was not found in the source bill text. Zero means every extracted number was confirmed against the source. If this number is ever greater than zero, the audit command will show you exactly which provisions need review.

By-agency view

Add --by-agency to see budget authority broken down by parent department:

congress-approp summary --dir data --by-agency

This appends a second table showing every agency, its total budget authority, rescissions, and provision count, sorted by budget authority descending. For example, Department of Veterans Affairs shows ~$343B (which includes mandatory programs like Compensation and Pensions that appear as appropriation lines in the bill text).

The Search Table

The search command produces tables that adapt their columns based on what you’re searching for. This is one of the most important things to understand about the output.

Standard search table

For most searches, you see this layout:

congress-approp search --dir data/118-hr9468

┌───┬───────────┬───────────────┬───────────────────────────────────────────────┬───────────────┬──────────┬─────┐
│ $ ┆ Bill      ┆ Type          ┆ Description / Account                         ┆    Amount ($) ┆ Section  ┆ Div │
╞═══╪═══════════╪═══════════════╪═══════════════════════════════════════════════╪═══════════════╪══════════╪═════╡
│ ✓ ┆ H.R. 9468 ┆ appropriation ┆ Compensation and Pensions                     ┆ 2,285,513,000 ┆          ┆     │
│ ✓ ┆ H.R. 9468 ┆ appropriation ┆ Readjustment Benefits                         ┆   596,969,000 ┆          ┆     │
│   ┆ H.R. 9468 ┆ rider         ┆ Establishes that each amount appropriated o…  ┆             — ┆ SEC. 101 ┆     │
│   ┆ H.R. 9468 ┆ rider         ┆ Unless otherwise provided, the additional a…  ┆             — ┆ SEC. 102 ┆     │
│   ┆ H.R. 9468 ┆ directive     ┆ Requires the Secretary of Veterans Affairs …  ┆             — ┆ SEC. 103 ┆     │
│   ┆ H.R. 9468 ┆ directive     ┆ Requires the Secretary of Veterans Affairs …  ┆             — ┆ SEC. 103 ┆     │
│   ┆ H.R. 9468 ┆ directive     ┆ Requires the Inspector General of the Depar…  ┆             — ┆ SEC. 104 ┆     │
└───┴───────────┴───────────────┴───────────────────────────────────────────────┴───────────────┴──────────┴─────┘
7 provisions found

The $ column — verification symbols

The leftmost column tells you the verification status of each provision’s dollar amount:

CR substitution table

When you search for cr_substitution type provisions, the table automatically changes shape to show the old and new amounts:

congress-approp search --dir data/118-hr5860 --type cr_substitution

┌───┬───────────┬──────────────────────────────────────────┬───────────────┬───────────────┬──────────────┬──────────┬─────┐
│ $ ┆ Bill      ┆ Account                                  ┆       New ($) ┆       Old ($) ┆    Delta ($) ┆ Section  ┆ Div │
╞═══╪═══════════╪══════════════════════════════════════════╪═══════════════╪═══════════════╪══════════════╪══════════╪═════╡
│ ✓ ┆ H.R. 5860 ┆ Rural Housing Service—Rural Community…   ┆    25,300,000 ┆    75,300,000 ┆  -50,000,000 ┆ SEC. 101 ┆ A   │
│ ...                                                                                                                      │
│ ✓ ┆ H.R. 5860 ┆ Office of Personnel Management—Salari…   ┆   219,076,000 ┆   190,784,000 ┆  +28,292,000 ┆ SEC. 126 ┆ A   │
└───┴───────────┴──────────────────────────────────────────┴───────────────┴───────────────┴──────────────┴──────────┴─────┘
13 provisions found

Instead of a single Amount column, you get:

Semantic search table

When you use --semantic or --similar, a Sim (similarity) column appears at the left:

┌──────┬───────────┬───────────────┬───────────────────────────────────────┬────────────────┬─────┐
│ Sim  ┆ Bill      ┆ Type          ┆ Description / Account                 ┆     Amount ($) ┆ Div │
╞══════╪═══════════╪═══════════════╪═══════════════════════════════════════╪════════════════╪═════╡
│ 0.51 ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs              ┆ 33,266,226,000 ┆ B   │
│ 0.46 ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs              ┆     10,000,000 ┆ B   │
└──────┴───────────┴───────────────┴───────────────────────────────────────┴────────────────┴─────┘

The Sim score is the cosine similarity between your query and the provision’s embedding vector, ranging from 0 to 1:

Results are sorted by similarity descending and limited to --top N (default 20).

The Audit Table

The audit command provides the most detailed quality view:

congress-approp audit --dir data

┌───────────┬────────────┬──────────┬──────────┬───────┬───────┬──────────┬───────────┬──────────┬──────────┐
│ Bill      ┆ Provisions ┆ Verified ┆ NotFound ┆ Ambig ┆ Exact ┆ NormText ┆ Spaceless ┆ TextMiss ┆ Coverage │
╞═══════════╪════════════╪══════════╪══════════╪═══════╪═══════╪══════════╪═══════════╪══════════╪══════════╡
│ H.R. 4366 ┆       2364 ┆      762 ┆        0 ┆   723 ┆  2285 ┆       59 ┆         0 ┆       20 ┆    94.2% │
│ H.R. 5860 ┆        130 ┆       33 ┆        0 ┆     2 ┆   102 ┆       12 ┆         0 ┆       16 ┆    61.1% │
│ H.R. 9468 ┆          7 ┆        2 ┆        0 ┆     0 ┆     5 ┆        0 ┆         0 ┆        2 ┆   100.0% │
│ TOTAL     ┆       2501 ┆      797 ┆        0 ┆   725 ┆  2392 ┆       71 ┆         0 ┆       38 ┆          │
└───────────┴────────────┴──────────┴──────────┴───────┴───────┴──────────┴───────────┴──────────┴──────────┘

The audit table has two groups of columns: amount verification (left side) and text verification (right side).

Amount verification columns

These check whether the dollar amount string (e.g., "$2,285,513,000") exists in the source bill text:

The sum of Verified + Ambig equals the total number of provisions that have dollar amounts. NotFound should always be zero. Across the included example data, it is.

Text verification columns

These check whether the raw_text excerpt (the first ~150 characters of the bill language for each provision) is a substring of the source text:

Coverage column

Coverage is the percentage of all dollar-sign patterns found in the source bill text that were matched to an extracted provision. This measures completeness, not accuracy.

• 100% (H.R. 9468): Every dollar amount in the source was captured — perfect.
• 94.2% (H.R. 4366): Most dollar amounts were captured. The remaining 5.8% are typically statutory cross-references, loan guarantee ceilings, or old amounts being struck by amendments — dollar figures that appear in the text but aren’t independent provisions.
• 61.1% (H.R. 5860): Lower coverage is expected for continuing resolutions because most of the bill text consists of references to prior-year appropriations acts, which contain many dollar amounts that are contextual references, not new provisions.

Coverage below 100% does not mean the extracted numbers are wrong. It means the bill text contains dollar strings that aren’t captured as provisions. See What Coverage Means (and Doesn’t) for a detailed explanation.

Quick decision guide

After running audit, here’s how to interpret the results:

The Compare Table

The compare command shows account-level differences between two sets of bills:

congress-approp compare --base data/118-hr4366 --current data/118-hr9468

┌─────────────────────────────────────┬──────────────────────┬─────────────────┬───────────────┬──────────────────┬─────────┬──────────────┐
│ Account                             ┆ Agency               ┆        Base ($) ┆   Current ($) ┆        Delta ($) ┆     Δ % ┆ Status       │
╞═════════════════════════════════════╪══════════════════════╪═════════════════╪═══════════════╪══════════════════╪═════════╪══════════════╡
│ Compensation and Pensions           ┆ Department of Veter… ┆ 197,382,903,000 ┆ 2,285,513,000 ┆ -195,097,390,000 ┆  -98.8% ┆ changed      │
│ Readjustment Benefits               ┆ Department of Veter… ┆  13,774,657,000 ┆   596,969,000 ┆  -13,177,688,000 ┆  -95.7% ┆ changed      │
│ ...                                                                                                                                       │
│ Supplemental Nutrition Assistance … ┆ Department of Agric… ┆ 122,382,521,000 ┆             0 ┆ -122,382,521,000 ┆ -100.0% ┆ only in base │
└─────────────────────────────────────┴──────────────────────┴─────────────────┴───────────────┴──────────────────┴─────────┴──────────────┘

Results are sorted by the absolute value of Delta, largest changes first.

Interpreting cross-type comparisons: When comparing an omnibus to a supplemental (as above), most accounts will show “only in base” because the supplemental only touches a few accounts. The tool warns you about this: “Comparing Omnibus to Supplemental. Accounts in one but not the other may be expected.” The compare command is most informative when comparing bills of the same type — for example, an FY2023 omnibus to an FY2024 omnibus.

Output Formats

Every query command supports four output formats via --format:

Table (default)

congress-approp search --dir data/118-hr9468 --format table

Human-readable formatted table. Best for interactive use and quick exploration. Column widths adapt to content. Long text is truncated.

JSON

congress-approp search --dir data/118-hr9468 --format json

A JSON array of objects. Includes every field for each matching provision — more data than the table shows. Best for programmatic consumption, piping to jq, or loading into scripts.

JSONL (JSON Lines)

congress-approp search --dir data/118-hr9468 --format jsonl

One JSON object per line, no enclosing array. Best for streaming processing, piping to while read, or working with very large result sets. Each line is independently parseable.

CSV

congress-approp search --dir data/118-hr9468 --format csv > provisions.csv

Comma-separated values suitable for importing into Excel, Google Sheets, R, or pandas. Includes a header row. Dollar amounts are plain integers (not formatted with commas).

Tip: When exporting to CSV for Excel, make sure to import the file with UTF-8 encoding. Some bill text contains em-dashes (—) and other Unicode characters that may display incorrectly with the default Windows encoding.

For a detailed guide with examples and recipes for each format, see Output Formats.

Provision Types at a Glance

You’ll encounter these provision types throughout the tool. Use --list-types for a quick reference:

congress-approp search --dir data --list-types

Available provision types:
  appropriation                    Budget authority grant
  rescission                       Cancellation of prior budget authority
  cr_substitution                  CR anomaly (substituting $X for $Y)
  transfer_authority               Permission to move funds between accounts
  limitation                       Cap or prohibition on spending
  directed_spending                Earmark / community project funding
  mandatory_spending_extension     Amendment to authorizing statute
  directive                        Reporting requirement or instruction
  rider                            Policy provision (no direct spending)
  continuing_resolution_baseline   Core CR funding mechanism
  other                            Unclassified provisions

The distribution varies by bill type. In the FY2024 omnibus (H.R. 4366), the breakdown is:

The continuing resolution (H.R. 5860) has a very different profile: 49 riders, 44 mandatory spending extensions, 13 CR substitutions, and only 5 standalone appropriations. This reflects the CR’s structure — it mostly continues prior-year funding rather than setting new levels.

For detailed documentation of each provision type including all fields and real examples, see Provision Types.

Enriched Output

When you run congress-approp enrich --dir data (no API key needed), the tool generates bill metadata that enhances the output:

• Enriched classifications — the summary table shows “Full-Year CR with Appropriations” instead of “Continuing Resolution” for hybrid bills like H.R. 1968, and “Minibus” instead of “Omnibus” for bills covering only 2–4 subcommittees.
• Advance appropriation split — use --show-advance on summary to separate current-year spending from advance appropriations (money enacted now but available in a future fiscal year). This is critical for VA accounts where 79.5% of MilCon-VA budget authority is advance.
• Fiscal year and subcommittee filtering — use --fy 2026 and --subcommittee thud to scope any command to a specific year and jurisdiction, automatically resolving division letters across bills.

See Enrich Bills with Metadata for the full guide.

Next Steps

Related chapters:

• Enrich Bills with Metadata — enable FY filtering, subcommittee scoping, and advance splits
• Find How Much Congress Spent on a Topic — your first real research task
• Compare Two Bills — see what changed between bills
• Track a Program Across Bills — trace one account across fiscal years
• Filter and Search Provisions — all the search flags in one place

Recipes & Demos

Worked examples using the included 32-bill dataset (data/). All commands run locally against the pre-extracted data with no API keys unless noted. Semantic search requires OPENAI_API_KEY.

The book/cookbook/cookbook.py script reproduces all CSVs, charts, and JSON shown on this page. See Run All Demos Yourself at the bottom.

Dataset Overview

Subcommittee coverage by fiscal year

The --subcommittee filter requires bills with separate divisions per jurisdiction. FY2025 was funded through H.R. 1968, a full-year continuing resolution that wraps all 12 subcommittees into a single division — so --subcommittee cannot break it apart. Use trace or search --fy 2025 to access FY2025 data by account.

Quick Reference

# Track any federal account across all fiscal years (by FAS code or name search)
congress-approp trace "child nutrition" --dir data

# Budget totals for FY2026
congress-approp summary --dir data --fy 2026

# Find FEMA provisions across all bills covering FY2026
congress-approp search --dir data --keyword "Federal Emergency Management" --fy 2026

# Compare THUD funding FY2024 → FY2026 with inflation adjustment
congress-approp compare --base-fy 2024 --current-fy 2026 --subcommittee thud --dir data --use-authorities --real

# Verification quality across all 32 bills
congress-approp audit --dir data

Searching and Tracking Accounts

Keyword search

The --keyword flag searches the raw_text field — the verbatim bill language stored with each provision. It is case-insensitive. Combine with --type to filter by provision type, --fy by fiscal year, --agency by department, or --min-dollars / --max-dollars for dollar ranges. All filters are ANDed.

congress-approp search --dir data --keyword "veterans" --type appropriation

┌───┬───────────┬───────────────┬───────────────────────────────────────────────┬─────────────────┬─────────┬─────┐
│ $ ┆ Bill      ┆ Type          ┆ Description / Account                         ┆      Amount ($) ┆ Section ┆ Div │
╞═══╪═══════════╪═══════════════╪═══════════════════════════════════════════════╪═════════════════╪═════════╪═════╡
│ ✓ ┆ H.R. 133  ┆ appropriation ┆ Compensation and Pensions                     ┆   6,110,251,552 ┆         ┆ J   │
│ ✓ ┆ H.R. 133  ┆ appropriation ┆ Readjustment Benefits                         ┆  14,946,618,000 ┆         ┆ J   │
│ ✓ ┆ H.R. 133  ┆ appropriation ┆ General Operating Expenses, Veterans Benefit… ┆   3,180,000,000 ┆         ┆ J   │
│ ...                                                                                                              │

Column reference:

Tracking an account across fiscal years

The trace command follows a single federal account across every bill in the dataset using its Federal Account Symbol (FAS code) — a government-assigned identifier that persists through name changes and reorganizations.

Finding the FAS code by name:

congress-approp trace "child nutrition" --dir data

If the name matches multiple accounts, the tool lists them with their FAS codes. Use the code for the specific account:

congress-approp trace 012-3539 --dir data

TAS 012-3539: Child Nutrition Programs, Food and Nutrition Service, Agriculture
  Agency: Department of Agriculture

┌──────┬──────────────────────┬───────────┬──────────────────────────┐
│ FY   ┆ Budget Authority ($) ┆ Bill(s)   ┆ Account Name(s)          │
╞══════╪══════════════════════╪═══════════╪══════════════════════════╡
│ 2020 ┆       23,615,098,000 ┆ H.R. 1865 ┆ Child Nutrition Programs │
│ 2021 ┆       25,118,440,000 ┆ H.R. 133  ┆ Child Nutrition Programs │
│ 2022 ┆       26,883,922,000 ┆ H.R. 2471 ┆ Child Nutrition Programs │
│ 2023 ┆       28,545,432,000 ┆ H.R. 2617 ┆ Child Nutrition Programs │
│ 2024 ┆       33,266,226,000 ┆ H.R. 4366 ┆ Child Nutrition Programs │
│ 2026 ┆       37,841,674,000 ┆ H.R. 5371 ┆ Child Nutrition Programs │
└──────┴──────────────────────┴───────────┴──────────────────────────┘

  6 fiscal years, 6 bills, 175,270,792,000 total

FY2025 is absent here because H.R. 1968 (the full-year CR) continued FY2024 rates without a separate line item for this account.

Accounts with name changes demonstrate why FAS codes are necessary for cross-bill tracking:

congress-approp trace 070-0400 --dir data

TAS 070-0400: Operations and Support, United States Secret Service, Homeland Security
  Agency: Department of Homeland Security

┌──────┬──────────────────────┬────────────────┬─────────────────────────────────────────────┐
│ FY   ┆ Budget Authority ($) ┆ Bill(s)        ┆ Account Name(s)                             │
╞══════╪══════════════════════╪════════════════╪═════════════════════════════════════════════╡
│ 2020 ┆        2,336,401,000 ┆ H.R. 1158      ┆ United States Secret Service—Operations an… │
│ 2021 ┆        2,373,109,000 ┆ H.R. 133       ┆ United States Secret Service—Operations an… │
│ 2022 ┆        2,554,729,000 ┆ H.R. 2471      ┆ Operations and Support                      │
│ 2023 ┆        2,734,267,000 ┆ H.R. 2617      ┆ Operations and Support                      │
│ 2024 ┆        3,007,982,000 ┆ H.R. 2882      ┆ Operations and Support                      │
│ 2025 ┆          231,000,000 ┆ H.R. 9747 (CR) ┆ United States Secret Service—Operations an… │
└──────┴──────────────────────┴────────────────┴─────────────────────────────────────────────┘

  Name variants across bills:
    "Operations and Support" (117-hr2471, 117-hr2617, 118-hr2882) [prefix]
    "United States Secret Service—Operations and Sup…" (116-hr1158, 116-hr133, 118-hr9747) [canonical]

  6 fiscal years, 6 bills, 13,237,488,000 total

The account was renamed between the 116th and 117th Congress — the “United States Secret Service—” prefix was dropped. FAS code 070-0400 unifies both names. The FY2025 row shows $231M from H.R. 9747 (a CR supplement), not the full-year level.

Semantic search

When the official program name is unknown, semantic search matches provisions by meaning rather than keywords. Requires OPENAI_API_KEY (one API call per query, ~100ms).

export OPENAI_API_KEY="your-key"
congress-approp search --dir data --semantic "school lunch programs for kids" --top 3

┌──────┬───────────────────┬───────────────┬──────────────────────────┬────────────────┐
│ Sim  ┆ Bill              ┆ Type          ┆ Description / Account    ┆     Amount ($) │
╞══════╪═══════════════════╪═══════════════╪══════════════════════════╪════════════════╡
│ 0.52 ┆ H.R. 1865 (116th) ┆ appropriation ┆ Child Nutrition Programs ┆ 23,615,098,000 │
│ 0.51 ┆ H.R. 4366 (118th) ┆ appropriation ┆ Child Nutrition Programs ┆ 33,266,226,000 │
│ 0.51 ┆ H.R. 2471 (117th) ┆ appropriation ┆ Child Nutrition Programs ┆ 26,883,922,000 │
└──────┴───────────────────┴───────────────┴──────────────────────────┴────────────────┘

“school lunch programs for kids” shares no keywords with “Child Nutrition Programs”, but semantic search matches them by meaning. The Sim column is cosine similarity between the query and provision embeddings:

Scores reflect the full provision text (account name + agency + raw bill language), not just the account name, which is why good matches are often in the 0.45–0.55 range rather than near 1.0.

Additional examples (tested against the dataset):

Comparing Across Fiscal Years

Year-over-year comparison with inflation adjustment

congress-approp compare --base-fy 2024 --current-fy 2026 --subcommittee thud \
    --dir data --use-authorities --real

20 orphan(s) rescued via TAS authority matching
Comparing: H.R. 4366 (118th)  →  H.R. 7148 (119th)

┌─────────────────────────────────────┬──────────────────────┬────────────────┬────────────────┬─────────────────┬─────────┬───────────┬───┬──────────┐
│ Account                             ┆ Agency               ┆       Base ($) ┆    Current ($) ┆       Delta ($) ┆     Δ % ┆ Real Δ %* ┆   ┆ Status   │
╞═════════════════════════════════════╪══════════════════════╪════════════════╪════════════════╪═════════════════╪═════════╪═══════════╪═══╪══════════╡
│ Tenant-Based Rental Assistance      ┆ Department of Housi… ┆ 32,386,831,000 ┆ 38,438,557,000 ┆  +6,051,726,000 ┆  +18.7% ┆    +13.8% ┆ ▲ ┆ changed  │
│ Federal-Aid Highways                ┆ Federal Highway Adm… ┆ 60,834,782,888 ┆ 63,396,105,821 ┆  +2,561,322,933 ┆   +4.2% ┆     -0.1% ┆ ▼ ┆ changed  │
│ Operations                          ┆ Federal Aviation Ad… ┆ 12,729,627,000 ┆ 13,710,000,000 ┆    +980,373,000 ┆   +7.7% ┆     +3.2% ┆ ▲ ┆ changed  │
│ Facilities and Equipment            ┆ Federal Aviation Ad… ┆  3,191,250,000 ┆  4,000,000,000 ┆    +808,750,000 ┆  +25.3% ┆    +20.1% ┆ ▲ ┆ changed  │
│ Capital Investment Grants           ┆ Federal Transit Adm… ┆  2,205,000,000 ┆  1,700,000,000 ┆    -505,000,000 ┆  -22.9% ┆    -26.1% ┆ ▼ ┆ changed  │
│ Public Housing Fund                 ┆ Department of Housi… ┆  8,810,784,000 ┆  8,319,393,000 ┆    -491,391,000 ┆   -5.6% ┆     -9.5% ┆ ▼ ┆ changed  │
│ ...                                 ┆                      ┆                ┆                ┆                 ┆         ┆           ┆   ┆          │

Column reference:

The Federal-Aid Highways row illustrates why inflation adjustment matters: nominal +4.2%, but real -0.1%. The nominal increase does not keep pace with inflation.

The --real flag works on any compare command — any subcommittee, any fiscal year pair. No API key needed.

The “20 orphan(s) rescued via TAS authority matching” message indicates 20 accounts that would have appeared unmatched (different names between FY2024 and FY2026) were paired using their FAS codes.

Subcommittee budget authority across fiscal years

Individual subcommittee totals can be retrieved per fiscal year using summary --fy Y --subcommittee S. The book/cookbook/cookbook.py script runs all combinations; the resulting table:

FY2025 is omitted for individual subcommittees because it was funded through a full-year CR with all jurisdictions under one division — see the coverage note above.

All values are budget authority. These include mandatory spending programs that appear as appropriation lines (e.g., SNAP under Agriculture, Medicaid under Labor-HHS). The MilCon-VA figure ($495B for FY2026) includes $394B in advance appropriations — see the next section.

Advance vs. current-year appropriations

congress-approp summary --dir data --fy 2026 --subcommittee milcon-va --show-advance

┌───────────────────┬──────┬────────────────┬────────────┬─────────────────┬─────────────────┬─────────────────┬─────────────────┬─────────────────┐
│ Bill              ┆ FYs  ┆ Classification ┆ Provisions ┆     Current ($) ┆     Advance ($) ┆    Total BA ($) ┆ Rescissions ($) ┆      Net BA ($) │
╞═══════════════════╪══════╪════════════════╪════════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╡
│ H.R. 5371 (119th) ┆ 2026 ┆ Minibus        ┆        263 ┆ 101,839,976,450 ┆ 393,592,053,000 ┆ 495,432,029,450 ┆  16,499,000,000 ┆ 478,933,029,450 │
│ TOTAL             ┆      ┆                ┆        263 ┆ 101,839,976,450 ┆ 393,592,053,000 ┆ 495,432,029,450 ┆  16,499,000,000 ┆ 478,933,029,450 │
└───────────────────┴──────┴────────────────┴────────────┴─────────────────┴─────────────────┴─────────────────┴─────────────────┴─────────────────┘

79.4% of FY2026 MilCon-VA budget authority ($394B of $495B) is advance appropriations for FY2027. Only $102B is current-year spending. Without --show-advance, the total combines both, which can distort year-over-year comparisons by hundreds of billions of dollars.

The classification uses bill_meta.json generated by enrich (run once, no API key). The algorithm compares each provision’s availability dates against the bill’s fiscal year.

CR substitutions — what the continuing resolution changed

Continuing resolutions fund the government at prior-year rates, except for specific anomalies (CR substitutions) where Congress sets a different level.

congress-approp search --dir data/118-hr5860 --type cr_substitution

┌───┬───────────┬──────────────────────────────────────────┬───────────────┬───────────────┬──────────────┬──────────┬─────┐
│ $ ┆ Bill      ┆ Account                                  ┆       New ($) ┆       Old ($) ┆    Delta ($) ┆ Section  ┆ Div │
╞═══╪═══════════╪══════════════════════════════════════════╪═══════════════╪═══════════════╪══════════════╪══════════╪═════╡
│ ✓ ┆ H.R. 5860 ┆ Rural Housing Service—Rural Community…   ┆    25,300,000 ┆    75,300,000 ┆  -50,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Rural Utilities Service—Rural Water a…   ┆    60,000,000 ┆   325,000,000 ┆ -265,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ National Science Foundation—STEM Educ…   ┆    92,000,000 ┆   217,000,000 ┆ -125,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ National Science Foundation—Research …   ┆   608,162,000 ┆   818,162,000 ┆ -210,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Office of Personnel Management—Salari…   ┆   219,076,000 ┆   190,784,000 ┆  +28,292,000 ┆ SEC. 126 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Department of Transportation—Federal …   ┆   617,000,000 ┆   570,000,000 ┆  +47,000,000 ┆ SEC. 137 ┆ A   │
│ ...                                                                                                                      │
└───┴───────────┴──────────────────────────────────────────┴───────────────┴───────────────┴──────────────┴──────────┴─────┘
13 provisions found

The cr_substitution table shows New (the CR level), Old (the prior-year rate being replaced), and Delta (the difference). Negative delta = funding cut below the prior-year rate. The full dataset contains 123 CR substitutions across all bills.

To see all CR substitutions: congress-approp search --dir data --type cr_substitution

Working with the Data Programmatically

Loading extraction.json in Python

Each bill’s provisions are in data/{bill_dir}/extraction.json:

import json
from collections import Counter

ext = json.load(open('data/119-hr7148/extraction.json'))
provisions = ext['provisions']

# Count by type
type_counts = Counter(p['provision_type'] for p in provisions)
for ptype, count in type_counts.most_common():
    print(f"  {ptype}: {count}")

  appropriation: 1201
  limitation: 553
  rider: 325
  directive: 285
  transfer_authority: 107
  rescission: 98
  mandatory_spending_extension: 82
  other: 63
  directed_spending: 59
  continuing_resolution_baseline: 1

Field access patterns:

p = provisions[0]
p['provision_type']       # → 'appropriation'
p['account_name']         # → 'Military Personnel, Army'
p['agency']               # → 'Department of Defense'

# Dollar amount (defensive — some fields can be null)
amt = p.get('amount') or {}
value = (amt.get('value') or {}).get('dollars', 0) or 0
# → 54538366000

amt['semantics']          # → 'new_budget_authority'
#   'new_budget_authority' — counts toward budget totals
#   'rescission'           — cancellation of prior funds
#   'transfer_ceiling'     — max transfer amount (not new spending)
#   'limitation'           — spending cap
#   'reference_amount'     — sub-allocation or contextual (not counted)
#   'mandatory_spending'   — mandatory program in the appropriation text

p['detail_level']         # → 'top_level'
#   'top_level'       — main account appropriation (counts toward totals)
#   'line_item'       — numbered item within a section (counts)
#   'sub_allocation'  — "of which" breakdown (does NOT count)
#   'proviso_amount'  — amount in a "Provided, That" clause (does NOT count)

p['raw_text'][:80]        # → verbatim bill language
p['confidence']           # → 0.97 (LLM self-assessed; not calibrated above 0.90)
p['section']              # → '' (empty if no section number)
p['division']             # → 'A'

# Source span — exact byte position in the enrolled bill
span = p.get('source_span') or {}
span['start']             # → UTF-8 byte offset in the source text file
span['end']               # → exclusive end byte
span['file']              # → 'BILLS-119hr7148enr.txt'
span['verified']          # → True (source_bytes[start:end] == raw_text)

Filtering to top-level budget authority provisions (the ones counted in totals):

for p in provisions:
    if p.get('provision_type') != 'appropriation':
        continue
    amt = p.get('amount') or {}
    if amt.get('semantics') != 'new_budget_authority':
        continue
    dl = p.get('detail_level', '')
    if dl in ('sub_allocation', 'proviso_amount'):
        continue
    dollars = (amt.get('value') or {}).get('dollars', 0) or 0
    print(f"{p['account_name'][:50]:50s}  ${dollars:>15,}")

Building a pandas DataFrame from authorities.json

data/authorities.json contains the cross-bill account registry — 1,051 accounts with provisions, name variants, and rename events. To flatten it into a DataFrame:

import json
import pandas as pd

auth = json.load(open('data/authorities.json'))

rows = []
for a in auth['authorities']:
    for prov in a.get('provisions', []):
        for fy in prov.get('fiscal_years', []):
            rows.append({
                'fas_code': a['fas_code'],
                'agency_code': a['agency_code'],
                'agency': a['agency_name'],
                'title': a['fas_title'],
                'fiscal_year': fy,
                'dollars': prov.get('dollars', 0) or 0,
                'bill': prov['bill_identifier'],
                'bill_dir': prov['bill_dir'],
                'confidence': prov['confidence'],
                'method': prov['method'],
            })

df = pd.DataFrame(rows)

Key fields:

Common operations:

# Budget authority by fiscal year
df.groupby('fiscal_year')['dollars'].sum().sort_index()

# Top 10 agencies
df.groupby('agency')['dollars'].sum().sort_values(ascending=False).head(10)

# Pivot: one row per account, one column per FY
df.pivot_table(values='dollars', index=['fas_code', 'title'],
               columns='fiscal_year', aggfunc='sum', fill_value=0)

# Export
df.to_csv('budget_timeline.csv', index=False)

CLI CSV export and analysis

Export provisions from the CLI, then load in Python or a spreadsheet:

congress-approp search --dir data --type appropriation --fy 2026 --format csv > fy2026_approps.csv

import pandas as pd

df = pd.read_csv('fy2026_approps.csv')

CSV field reference:

Do not sum the dollars column directly. Filter to semantics == 'new_budget_authority' and exclude detail_level in ('sub_allocation', 'proviso_amount') to avoid double-counting. Or use congress-approp summary which handles this automatically.

ba = df[(df['semantics'] == 'new_budget_authority') &
        (~df['detail_level'].isin(['sub_allocation', 'proviso_amount']))]
print(f"FY2026 BA provisions: {len(ba)}")
print(f"Total: ${ba['dollars'].sum():,.0f}")

Other export formats: --format json (array), --format jsonl (one object per line for streaming), --format csv.

jq one-liners:

# Top 5 rescissions by dollar amount
congress-approp search --dir data --type rescission --format json | \
  jq 'sort_by(-.dollars) | .[0:5] | .[] | {bill, account_name, dollars}'

# Count provisions by type for FY2026
congress-approp search --dir data --fy 2026 --format json | \
  jq 'group_by(.provision_type) | map({type: .[0].provision_type, count: length}) | sort_by(-.count)'

Source span verification

Every provision carries a source_span with exact byte offsets into the enrolled bill text. To independently verify a provision:

import json

ext = json.load(open('data/118-hr9468/extraction.json'))
p = ext['provisions'][0]
span = p['source_span']

source_bytes = open(f"data/118-hr9468/{span['file']}", 'rb').read()
actual = source_bytes[span['start']:span['end']].decode('utf-8')

assert actual == p['raw_text']  # True

Account:  Compensation and Pensions
Dollars:  $2,285,513,000
Span:     bytes 371..482 in BILLS-118hr9468enr.txt
Match:    True

start and end are UTF-8 byte offsets. In Python, use open(path, 'rb').read()[start:end].decode('utf-8') — not character-based indexing.

To verify all provisions across multiple bills:

import json, os

for bill_dir in ['118-hr9468', '119-hr7148', '119-hr5371']:
    ext = json.load(open(f'data/{bill_dir}/extraction.json'))
    for i, p in enumerate(ext['provisions']):
        span = p.get('source_span') or {}
        if not span.get('file'):
            continue
        source = open(f'data/{bill_dir}/{span["file"]}', 'rb').read()
        actual = source[span['start']:span['end']].decode('utf-8')
        assert actual == p['raw_text'], f'{bill_dir} provision {i}: MISMATCH'
    print(f'{bill_dir}: {len(ext["provisions"])} provisions verified')

Visualizations

Generated by book/cookbook/cookbook.py. The images below are included in the repository; run the script to regenerate from the current data.

FY2026 Interactive Treemap

FY2026 budget authority ($5.6 trillion across 1,076 accounts) organized by jurisdiction → agency → account. The file is a self-contained HTML page — open it in your browser.

Hierarchy: jurisdiction (subcommittee) → agency (department) → account. Click to zoom. Color intensity encodes dollar amount.

Defense vs. Non-Defense Spending Trend

Dark blue = Defense. Light blue = all other subcommittees. Defense grew from $693B to $836B (+21%) over this period. Non-defense growth is primarily driven by mandatory spending programs (Medicaid, SNAP, VA Compensation) that appear as appropriation lines in the bill text. See Why the Numbers Might Not Match Headlines.

Top 6 Federal Accounts by Budget Authority

Each line is one Treasury Account Symbol (FAS code). The top accounts are dominated by mandatory programs that appear as appropriation line items: Medicaid, Health Care Trust Funds, and VA Compensation & Pensions.

Note on FY2025→FY2026 jumps: Some accounts show sharp increases between FY2025 and FY2026 (e.g., Medicaid $261B → $1,086B). This is because FY2025 was covered by a single full-year CR while FY2026 has multiple omnibus/minibus bills — the amounts are correct per bill, but the visual jump reflects different legislative coverage.

Verification Quality Heatmap

Each row is a bill; each column is a verification metric. Color intensity shows the percentage of provisions meeting that criterion.

Bills with low $ Verified percentages (e.g., CRs) are expected — most CR provisions do not carry dollar amounts.

Run All Demos Yourself

book/cookbook/cookbook.py runs 24 demos including everything above plus TAS resolution quality per bill, account rename events, directed spending analysis, advance appropriation breakdown, and more.

Setup

source .venv/bin/activate
pip install -r book/cookbook/requirements.txt

Run

python book/cookbook/cookbook.py

For semantic search demos (optional):

export OPENAI_API_KEY="your-key"
python book/cookbook/cookbook.py

Output

All files go to tmp/demo_output/:

Find How Much Congress Spent on a Topic

You will need: congress-approp installed, access to the data/ directory. For semantic search: OPENAI_API_KEY.

You will learn: Three ways to find spending provisions — by account name, by keyword, and by semantic meaning — and when to use each one.

This tutorial demonstrates three methods for finding spending provisions — by account name, by keyword, and by semantic meaning — and when each is appropriate.

Start with the Agency Rollup

If your question is about an entire department, the fastest answer is the by-agency summary:

congress-approp summary --dir data --by-agency

This prints the standard bill summary table, followed by a second table breaking down budget authority by parent department. Here’s the top of that second table:

┌─────────────────────────────────────────────────────┬─────────────────┬─────────────────┬────────────┐
│ Department                                           ┆ Budget Auth ($) ┆ Rescissions ($) ┆ Provisions │
╞═════════════════════════════════════════════════════╪═════════════════╪═════════════════╪════════════╡
│ Department of Veterans Affairs                       ┆ 343,238,707,982 ┆   9,799,155,560 ┆         51 │
│ Department of Agriculture                            ┆ 187,748,124,000 ┆     351,891,000 ┆        266 │
│ Department of Housing and Urban Development          ┆  75,743,762,466 ┆      85,000,000 ┆        116 │
│ Department of Energy                                 ┆  50,776,281,000 ┆               0 ┆         62 │
│ Department of Justice                                ┆  37,960,158,000 ┆   1,158,272,000 ┆        186 │
│ ...                                                                                                    │
└─────────────────────────────────────────────────────┴─────────────────┴─────────────────┴────────────┘

So the answer to “how much did the VA get?” is approximately $343 billion in budget authority across all bills in the dataset, with $9.8 billion in rescissions.

Important caveat: This total includes mandatory spending programs that appear as appropriation lines in the bill text. VA’s Compensation and Pensions account alone is $197 billion — that’s a mandatory entitlement, not discretionary spending, even though it appears in the appropriations bill. See Why the Numbers Might Not Match Headlines for more on this distinction.

Search by Account Name

When you know the program’s official name (or part of it), --account is the most precise filter. It matches against the structured account_name field:

congress-approp search --dir data --account "Child Nutrition"

┌───┬───────────┬───────────────┬─────────────────────────────────────────────┬────────────────┬─────────┬─────┐
│ $ ┆ Bill      ┆ Type          ┆ Description / Account                       ┆     Amount ($) ┆ Section ┆ Div │
╞═══╪═══════════╪═══════════════╪═════════════════════════════════════════════╪════════════════╪═════════╪═════╡
│ ✓ ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆ 33,266,226,000 ┆         ┆ B   │
│ ✓ ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆     18,004,000 ┆         ┆ B   │
│ ✓ ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆     21,005,000 ┆         ┆ B   │
│ ✓ ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆      5,000,000 ┆         ┆ B   │
│ ≈ ┆ H.R. 4366 ┆ limitation    ┆ Child Nutrition Programs                    ┆        500,000 ┆         ┆ B   │
│ ≈ ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆     10,000,000 ┆         ┆ B   │
│ ≈ ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆      1,000,000 ┆         ┆ B   │
│ ✓ ┆ H.R. 4366 ┆ appropriation ┆ McGovern-Dole International Food for Educ…  ┆    240,000,000 ┆         ┆ B   │
│ ≈ ┆ H.R. 4366 ┆ limitation    ┆ McGovern-Dole International Food for Educ…  ┆     24,000,000 ┆         ┆ B   │
└───┴───────────┴───────────────┴─────────────────────────────────────────────┴────────────────┴─────────┴─────┘

The top result — $33,266,226,000 — is the top-level appropriation for Child Nutrition Programs. The smaller amounts below it are sub-allocations (“of which $18,004,000 shall be for…”) and proviso amounts that break down how the top-level figure is to be spent. These sub-allocations have reference_amount semantics and are not counted again in the budget authority total — no double-counting.

The McGovern-Dole account also matches because it has “Child Nutrition” in its full name.

When to use --account vs. --keyword

• --account matches against the structured account_name field extracted by the LLM — the official name of the appropriations account.
• --keyword searches the full raw_text field — the actual bill language.

Sometimes the account name doesn’t contain the term you’re looking for, but the bill text does. Other times, the bill text doesn’t mention a term that is in the account name. Use both when you want to be thorough.

Search by Keyword in Bill Text

The --keyword flag searches the raw_text field — the excerpt of actual bill language stored with each provision. This finds provisions where the term appears anywhere in the source text, regardless of account name:

congress-approp search --dir data --keyword "Federal Emergency Management"

┌───┬───────────┬───────────────┬───────────────────────────────────────────────┬────────────────┬──────────┬─────┐
│ $ ┆ Bill      ┆ Type          ┆ Description / Account                         ┆     Amount ($) ┆ Section  ┆ Div │
╞═══╪═══════════╪═══════════════╪═══════════════════════════════════════════════╪════════════════╪══════════╪═════╡
│   ┆ H.R. 5860 ┆ other         ┆ Allows FEMA Disaster Relief Fund to be appor… ┆              — ┆ SEC. 128 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ appropriation ┆ Federal Emergency Management Agency—Disast…   ┆ 16,000,000,000 ┆ SEC. 129 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ appropriation ┆ Office of the Inspector General—Operations…   ┆      2,000,000 ┆ SEC. 129 ┆ A   │
└───┴───────────┴───────────────┴───────────────────────────────────────────────┴────────────────┴──────────┴─────┘
3 provisions found

This found three provisions: the $16B FEMA Disaster Relief Fund appropriation, a $2M Inspector General appropriation, and a non-dollar provision about how the fund can be apportioned. All three are in the continuing resolution (H.R. 5860), not the omnibus — because FEMA’s regular funding falls under the Homeland Security appropriations bill, which isn’t one of the divisions included in this particular omnibus.

Useful keywords for exploring

Here are some keywords that surface interesting provisions in the example data:

Combining filters

All search filters are combined with AND logic. Every provision in the result must match every filter you specify:

# Appropriations over $1 billion in Division A (MilCon-VA)
congress-approp search --dir data --type appropriation --division A --min-dollars 1000000000

# Rescissions from the Department of Justice
congress-approp search --dir data --type rescission --agency "Justice"

# Directives in the VA supplemental
congress-approp search --dir data/118-hr9468 --type directive

Search by Meaning (Semantic Search)

Keyword search has a fundamental limitation: it only finds provisions that use the exact words you search for. If you search for “school lunch” but the bill says “Child Nutrition Programs,” keyword search returns nothing.

Semantic search solves this. It uses embedding vectors to understand the meaning of your query and rank provisions by conceptual similarity — even when the words don’t overlap at all.

Prerequisites: Semantic search requires OPENAI_API_KEY (to embed your query text at search time) and pre-computed embeddings for the bills you’re searching. The included example data has pre-computed embeddings, so you just need the API key.

export OPENAI_API_KEY="your-key-here"
congress-approp search --dir data --semantic "school lunch programs for kids" --top 5

┌──────┬───────────┬───────────────┬─────────────────────────────────────────────┬────────────────┬─────┐
│ Sim  ┆ Bill      ┆ Type          ┆ Description / Account                       ┆     Amount ($) ┆ Div │
╞══════╪═══════════╪═══════════════╪═════════════════════════════════════════════╪════════════════╪═════╡
│ 0.51 ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆ 33,266,226,000 ┆ B   │
│ 0.46 ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆     10,000,000 ┆ B   │
│ 0.45 ┆ H.R. 4366 ┆ rider         ┆ Pilot project grant recipients shall be r…  ┆              — ┆ B   │
│ 0.45 ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆     18,004,000 ┆ B   │
│ 0.44 ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆      5,000,000 ┆ B   │
└──────┴───────────┴───────────────┴─────────────────────────────────────────────┴────────────────┴─────┘
5 provisions found

The query “school lunch programs for kids” shares no keywords with “Child Nutrition Programs”, but semantic search matches them by meaning. The similarity score of 0.51 reflects the conceptual relationship between the query and the provision text.

More semantic search examples

Try these queries against the example data to get a feel for how semantic search finds provisions that keyword search would miss:

# "Fixing roads and bridges" → finds Highway Infrastructure Programs, Federal-Aid Highways
congress-approp search --dir data --semantic "money for fixing roads and bridges" --top 5

# "Space exploration" → finds NASA Exploration, Space Operations, Space Technology
congress-approp search --dir data --semantic "space exploration" --top 5

# "Clean energy" → finds Energy Efficiency and Renewable Energy, Nuclear Energy
congress-approp search --dir data --semantic "clean energy research" --top 5

Combining semantic search with filters

You can narrow semantic results with hard filters. For example, find only appropriation-type provisions about clean energy with at least $100 million:

congress-approp search --dir data --semantic "clean energy" --type appropriation --min-dollars 100000000 --top 10

The filters are applied first (hard constraints that must match), then the remaining provisions are ranked by semantic similarity.

When semantic search doesn’t help

Semantic search is not always the right tool:

• Exact account name lookup: If you know the account name, use --account. It’s faster, deterministic, and doesn’t require an API key.
• No conceptual match: If nothing in the dataset relates to your query, similarity scores will be low (below 0.40). Low scores are an honest answer — the tool isn’t hallucinating relevance.
• Provision type distinction: Embeddings don’t strongly encode whether something is a rider vs. an appropriation. If you need only appropriations, add --type appropriation as a hard filter.

Get the Full Details in JSON

Once you’ve found interesting provisions in the table view, switch to JSON to see every field:

congress-approp search --dir data --account "Child Nutrition" --type appropriation --format json

This returns the full structured data for each matching provision, including fields the table truncates: raw_text (the full excerpt), semantics, detail_level, agency, division, notes, cross_references, and more.

For example, the top-level Child Nutrition Programs appropriation includes:

{
  "account_name": "Child Nutrition Programs",
  "agency": "Department of Agriculture",
  "bill": "H.R. 4366",
  "dollars": 33266226000,
  "semantics": "new_budget_authority",
  "detail_level": "top_level",
  "division": "B",
  "provision_type": "appropriation",
  "quality": "strong",
  "amount_status": "found",
  "match_tier": "exact",
  "raw_text": "For necessary expenses of the Food and Nutrition Service..."
}

Key fields to check:

• semantics: new_budget_authority means this counts toward the budget authority total. reference_amount means it’s a sub-allocation or contextual amount.
• detail_level: top_level is the main account appropriation. sub_allocation is an “of which” breakdown. line_item is a numbered item within a section.
• quality: strong means the dollar amount was verified and the raw text matched the source. moderate or weak means something didn’t check out as well.

Cross-Check Against the Source

For any provision you plan to cite, you can verify it directly against the bill XML. The raw_text field contains the excerpt, and the text_as_written dollar string can be searched in the source file:

# Find the dollar string in the source XML
grep '33,266,226,000' data/118-hr4366/BILLS-118hr4366enr.xml

If the string is found (which it will be — the audit confirms this), you know the extraction is accurate. For a full verification procedure, see Verify Extraction Accuracy.

Export for Further Analysis

Once you’ve identified the provisions you care about, export them for further work:

# CSV for Excel or Google Sheets
congress-approp search --dir data --account "Child Nutrition" --format csv > child_nutrition.csv

# JSON for Python, R, or jq
congress-approp search --dir data --agency "Veterans" --type appropriation --format json > va_appropriations.json

See Export Data for Spreadsheets and Scripts for detailed recipes.

Summary: Which Search Method to Use

For the most thorough search, try multiple approaches. Start with --account or --keyword for precision, then use --semantic to catch provisions you might have missed with different terminology.

Next Steps

• Compare Two Bills — see what changed between a CR and an omnibus
• Track a Program Across Bills — follow a specific account across bills using --similar
• Use Semantic Search — deeper dive into embedding-based search

Compare Two Bills

You will need: congress-approp installed, access to the data/ directory.

You will learn: How to use the compare command to see which accounts gained, lost, or changed funding between two sets of bills.

One of the most common questions in appropriations analysis is: “What changed?” Maybe you’re comparing a continuing resolution to the full-year omnibus to see which programs got different treatment. Maybe you’re comparing this year’s omnibus to last year’s. Or maybe a supplemental added emergency funding on top of the base bill and you want to see exactly where the money went.

The compare command answers these questions by matching accounts across two sets of bills and computing the dollar difference.

Your First Comparison

Let’s compare the FY2024 omnibus (H.R. 4366) to the VA supplemental (H.R. 9468) to see which accounts got additional emergency funding:

congress-approp compare --base data/118-hr4366 --current data/118-hr9468

The tool first prints a warning:

⚠  Comparing Omnibus to Supplemental. Accounts in one but not the other may be expected
    — this does not necessarily indicate policy changes.

This is important context. A supplemental only touches a handful of accounts, so most accounts from the omnibus will show up as “only in base.” That’s expected — the supplemental didn’t eliminate those programs.

The comparison table follows, sorted by largest absolute change first:

┌─────────────────────────────────────┬──────────────────────┬─────────────────┬───────────────┬──────────────────┬─────────┬──────────────┐
│ Account                             ┆ Agency               ┆        Base ($) ┆   Current ($) ┆        Delta ($) ┆     Δ % ┆ Status       │
╞═════════════════════════════════════╪══════════════════════╪═════════════════╪═══════════════╪══════════════════╪═════════╪══════════════╡
│ Compensation and Pensions           ┆ Department of Veter… ┆ 197,382,903,000 ┆ 2,285,513,000 ┆ -195,097,390,000 ┆  -98.8% ┆ changed      │
│ Supplemental Nutrition Assistance … ┆ Department of Agric… ┆ 122,382,521,000 ┆             0 ┆ -122,382,521,000 ┆ -100.0% ┆ only in base │
│ Medical Services                    ┆ Department of Veter… ┆  71,000,000,000 ┆             0 ┆  -71,000,000,000 ┆ -100.0% ┆ only in base │
│ Child Nutrition Programs            ┆ Department of Agric… ┆  33,266,226,000 ┆             0 ┆  -33,266,226,000 ┆ -100.0% ┆ only in base │
│ ...                                                                                                                                       │
│ Readjustment Benefits               ┆ Department of Veter… ┆  13,774,657,000 ┆   596,969,000 ┆  -13,177,688,000 ┆  -95.7% ┆ changed      │
│ ...                                                                                                                                       │
└─────────────────────────────────────┴──────────────────────┴─────────────────┴───────────────┴──────────────────┴─────────┴──────────────┘

Understanding the Columns

Status values

Interpreting Cross-Type Comparisons

The comparison above — omnibus vs. supplemental — is instructive but requires careful interpretation:

Why “Compensation and Pensions” shows -98.8%: The omnibus has $197B for Comp & Pensions (which includes mandatory spending). The supplemental has $2.3B. The compare command shows the raw dollar values in each set — it doesn’t add them together. The supplemental is additional funding on top of the omnibus, but the compare table shows the amounts within each set, not cumulative totals.

Why most accounts show “only in base”: The supplemental only funds two accounts (Comp & Pensions and Readjustment Benefits). Every other account in the omnibus has zero representation in the supplemental. This doesn’t mean those programs lost funding — it means the supplemental didn’t touch them.

The classification warning: The tool detects when you’re comparing different bill types (Omnibus vs. Supplemental, CR vs. Regular, etc.) and prints a warning. These cross-type comparisons can be misleading if you interpret “only in base” as “program eliminated.”

A More Natural Comparison: Filtering by Agency

To focus on just the accounts that matter, use --agency to narrow the comparison:

congress-approp compare --base data/118-hr4366 --current data/118-hr9468 --agency "Veterans"

This filters both sides to only show accounts from the Department of Veterans Affairs, making the comparison much easier to read. You’ll see the two “changed” accounts (Comp & Pensions and Readjustment Benefits) plus the VA accounts that are “only in base.”

When Compare Shines: Same-Type Comparisons

The compare command is most useful when comparing bills of the same type:

• FY2023 omnibus → FY2024 omnibus: See which programs gained or lost funding year over year
• House version → Senate version: Track differences during the conference process
• FY2024 omnibus → FY2025 omnibus: Year-over-year trend analysis

To do this, extract both bills into separate directories, then:

# Example: comparing two fiscal years (requires extracting both bills first)
congress-approp compare --base data/117/hr/2471 --current data/118/hr/4366

Accounts are matched by (agency, account_name) with automatic normalization. Results are sorted by the absolute value of the delta, so the biggest changes appear first.

Handling Account Name Mismatches

The compare command matches accounts by exact normalized name. If Congress renames an account between fiscal years — say, “Cybersecurity and Infrastructure Security Agency” becomes “CISA Operations and Support” — the compare command will show the old name as “only in base” and the new name as “only in current” rather than matching them.

For accounts with different names that represent the same program, use the --similar flag on search to find the semantic match:

congress-approp search --dir data --similar 118-hr9468:0 --top 5

This uses embedding vectors to match by meaning rather than account name. See Track a Program Across Bills for details.

The compare --use-links flag uses persistent cross-bill relationships (created via link accept) to inform the matching, handling renames automatically. See Track a Program Across Bills for the full link workflow.

Export Comparisons

Like all query commands, compare supports multiple output formats:

# CSV for Excel analysis
congress-approp compare --base data/118-hr4366 --current data/118-hr9468 --format csv > comparison.csv

# JSON for programmatic processing
congress-approp compare --base data/118-hr4366 --current data/118-hr9468 --format json

The JSON output includes every field for each account delta:

[
  {
    "account_name": "Compensation and Pensions",
    "agency": "Department of Veterans Affairs",
    "base_dollars": 197382903000,
    "current_dollars": 2285513000,
    "delta": -195097390000,
    "delta_pct": -98.84,
    "status": "changed"
  }
]

This is useful for building year-over-year tracking dashboards or automated change reports.

Practical Examples

Which programs got the biggest increases?

congress-approp compare --base data/fy2023 --current data/fy2024 --format json | \
  jq '[.[] | select(.delta > 0)] | sort_by(-.delta) | .[:10]'

Which programs were eliminated?

congress-approp compare --base data/fy2023 --current data/fy2024 --format json | \
  jq '[.[] | select(.status == "only in base")] | sort_by(-.base_dollars)'

What’s new this year?

congress-approp compare --base data/fy2023 --current data/fy2024 --format json | \
  jq '[.[] | select(.status == "only in current")] | sort_by(-.current_dollars)'

Summary

The compare command is your tool for answering “what changed?” at the account level:

• Use --base and --current to point at any two directories containing extracted bills
• Results are sorted by the absolute value of the change — biggest impacts first
• The --agency filter helps focus on specific departments
• Pay attention to the classification warning when comparing different bill types
• Export to CSV or JSON for further analysis
• For accounts that change names between bills, use --similar semantic matching

Next Steps

• Track a Program Across Bills — use embedding-based matching when account names differ
• Export Data for Spreadsheets and Scripts — advanced export recipes
• Why the Numbers Might Not Match Headlines — understand why budget authority figures may differ from public reports

Track a Program Across Bills

You will need: congress-approp installed, access to the data/ directory. Optionally: OPENAI_API_KEY for semantic search.

You will learn: How to follow a specific program’s funding across multiple bills using --similar, and how to interpret cross-bill matching results.

A single program — say, VA Compensation and Pensions — can appear in multiple bills within the same fiscal year: the full-year omnibus, a continuing resolution, and an emergency supplemental. Tracking it across all three tells you the complete funding story. But account names aren’t always consistent between bills, and keyword search only works when you know the exact terminology each bill uses.

The --similar flag solves this by using pre-computed embedding vectors to find provisions that mean the same thing, even when the words differ.

The Scenario

H.R. 9468 (the VA Supplemental) appropriated $2,285,513,000 for “Compensation and Pensions.” You want to find every related provision in the omnibus (H.R. 4366) and the continuing resolution (H.R. 5860).

Step 1: Identify the Source Provision

First, find the provision you want to track. You can use any search command to locate it:

congress-approp search --dir data/118-hr9468 --type appropriation

┌───┬───────────┬───────────────┬─────────────────────────────┬───────────────┬─────────┬─────┐
│ $ ┆ Bill      ┆ Type          ┆ Description / Account       ┆    Amount ($) ┆ Section ┆ Div │
╞═══╪═══════════╪═══════════════╪═════════════════════════════╪═══════════════╪═════════╪═════╡
│ ✓ ┆ H.R. 9468 ┆ appropriation ┆ Compensation and Pensions   ┆ 2,285,513,000 ┆         ┆     │
│ ✓ ┆ H.R. 9468 ┆ appropriation ┆ Readjustment Benefits       ┆   596,969,000 ┆         ┆     │
└───┴───────────┴───────────────┴─────────────────────────────┴───────────────┴─────────┴─────┘
2 provisions found

Compensation and Pensions is the first provision listed. To use --similar, you need the bill directory name and provision index. The directory is hr9468 (the directory name inside data/), and the index is 0 (first provision, zero-indexed).

You can also see the index in JSON output:

congress-approp search --dir data/118-hr9468 --type appropriation --format json

Look for the "provision_index": 0 field in the first result.

Step 2: Find Similar Provisions Across All Bills

Now use --similar to find the closest matches across every loaded bill:

congress-approp search --dir data --similar 118-hr9468:0 --top 10

┌──────┬───────────┬───────────────┬────────────────────────────────┬─────────────────┬─────┐
│ Sim  ┆ Bill      ┆ Type          ┆ Description / Account          ┆      Amount ($) ┆ Div │
╞══════╪═══════════╪═══════════════╪════════════════════════════════╪═════════════════╪═════╡
│ 0.86 ┆ H.R. 4366 ┆ appropriation ┆ Compensation and Pensions      ┆ 182,310,515,000 ┆ A   │
│ 0.78 ┆ H.R. 4366 ┆ appropriation ┆ Compensation and Pensions      ┆  15,072,388,000 ┆ A   │
│ 0.73 ┆ H.R. 4366 ┆ limitation    ┆ Compensation and Pensions      ┆      22,109,000 ┆ A   │
│ 0.70 ┆ H.R. 9468 ┆ appropriation ┆ Readjustment Benefits          ┆     596,969,000 ┆     │
│ 0.68 ┆ H.R. 4366 ┆ rescission    ┆ Medical Support and Compliance ┆   1,550,000,000 ┆ A   │
│ ...                                                                                         │
└──────┴───────────┴───────────────┴────────────────────────────────┴─────────────────┴─────┘

This is the complete picture of Comp & Pensions across the dataset:

1. 0.86 similarity — The omnibus’s main Comp & Pensions appropriation: $182.3 billion. This is the regular-year funding for the same account that the supplemental topped up by $2.3 billion.
2. 0.78 similarity — The omnibus’s advance appropriation for Comp & Pensions: $15.1 billion. This is money enacted in FY2024 but available for FY2025.
3. 0.73 similarity — A $22 million limitation on the Comp & Pensions account.
4. 0.70 similarity — Readjustment Benefits from the same supplemental. This is a different VA account, but conceptually close because it’s also VA mandatory benefits.
5. 0.68 similarity — A rescission of Medical Support and Compliance funds. Related VA account, lower similarity because it’s a different type of action (rescission vs. appropriation).

Why no CR matches?

The continuing resolution (H.R. 5860) doesn’t have a specific Comp & Pensions provision because CRs fund at the prior-year rate by default. Only the 13 programs with anomalies (CR substitutions) appear as explicit provisions. VA Comp & Pensions wasn’t one of them — it was simply continued at its prior-year level.

Step 3: How –similar Works Under the Hood

The --similar flag does not make any API calls. Here’s what happens:

1. It looks up the embedding vector for 118-hr9468:0 from the pre-computed vectors.bin file
2. It loads the embedding vectors for every provision in every bill under --dir
3. It computes the cosine similarity between the source vector and every other vector
4. It ranks by similarity descending and returns the top N results

Because everything is pre-computed and stored locally, this operation takes less than a millisecond for 2,500 provisions. The only prerequisite is that embeddings have been generated (via congress-approp embed) for all the bills you want to search.

Step 4: Interpret Similarity Scores

The similarity score tells you how closely related two provisions are in “meaning space”:

For cross-bill tracking, focus on matches above 0.75 — these are very likely the same account in a different bill.

Step 5: Track the Second Account

Repeat for Readjustment Benefits (provision index 1 in the supplemental):

congress-approp search --dir data --similar 118-hr9468:1 --top 5

┌──────┬───────────┬───────────────┬────────────────────────────────┬─────────────────┬─────┐
│ Sim  ┆ Bill      ┆ Type          ┆ Description / Account          ┆      Amount ($) ┆ Div │
╞══════╪═══════════╪═══════════════╪════════════════════════════════╪═════════════════╪═════╡
│ 0.88 ┆ H.R. 4366 ┆ appropriation ┆ Readjustment Benefits          ┆  13,399,805,000 ┆ A   │
│ 0.76 ┆ H.R. 9468 ┆ appropriation ┆ Compensation and Pensions      ┆   2,285,513,000 ┆     │
│ ...                                                                                         │
└──────┴───────────┴───────────────┴────────────────────────────────┴─────────────────┴─────┘

Top match at 0.88: the omnibus Readjustment Benefits account at $13.4 billion. The supplemental added $597 million on top of that.

When Account Names Differ Between Bills

The example data happens to use the same account names across bills, but this isn’t always the case. Continuing resolutions often use hierarchical names like:

• CR: "Rural Housing Service—Rural Community Facilities Program Account"
• Omnibus: "Rural Community Facilities Program Account"

Keyword matching would miss this, but --similar handles it because the embeddings capture the meaning of the provision, not just the words.

To demonstrate, let’s find the omnibus counterparts of the CR substitutions that have different naming conventions:

# First, find a CR substitution provision index
congress-approp search --dir data/118-hr5860 --type cr_substitution --format json
# Note: the first CR substitution (Rural Housing) is at some index — check provision_index

# Then find similar provisions in the omnibus
congress-approp search --dir data --similar hr5860:<INDEX> --top 3

Even though “Rural Housing Service—Rural Community Facilities Program Account” and “Rural Community Facilities Program Account” are different strings, the embedding similarity will be in the 0.75–0.80 range — high enough to confidently identify them as the same program.

Building a Funding Timeline

Once you can match accounts across bills, you can assemble a complete funding picture. For VA Comp & Pensions in FY2024:

With multiple fiscal years extracted, you could extend this to a multi-year timeline. The --similar command makes cross-year matching possible even when account names evolve.

Deep-Dive with Relate

The relate command provides a focused view of one provision across all bills, with a fiscal year timeline showing advance/current/supplemental splits:

# Trace VA Compensation and Pensions across all fiscal years
congress-approp relate 118-hr9468:0 --dir data --fy-timeline

Each match includes a deterministic 8-character hash that you can use to persist the relationship.

Persistent Links

You can save cross-bill relationships using the link system, so they persist across sessions and can be used by compare --use-links:

# Discover link candidates from embeddings
congress-approp link suggest --dir data --scope cross --limit 20

# Accept specific matches by hash (from relate or link suggest output)
congress-approp link accept --dir data a3f7b2c4 e5d1c8a9

# Or batch-accept all verified + high-confidence candidates
congress-approp link accept --dir data --auto

# Use accepted links in compare to handle renames
congress-approp compare --base-fy 2024 --current-fy 2026 --subcommittee thud --dir data --use-links

# View and manage saved links
congress-approp link list --dir data
congress-approp link remove --dir data a3f7b2c4

Links are stored at <dir>/links/links.json and are indexed by deterministic hashes — the same provision pair always produces the same hash, so you can script the workflow reliably. See Enrich Bills with Metadata and the CLI Reference for details.

This will enable automatic cross-year matching even when account names change, with human review for ambiguous cases.

Tips for Cross-Bill Tracking

1. Start from the smaller bill. If you’re tracking between a supplemental (7 provisions) and an omnibus (2,364 provisions), start from the supplemental and search into the omnibus. It’s easier to review 5–10 matches than 2,364.

2. Use --top 3 to reduce noise. You rarely need more than the top 3 matches. The best match is almost always the right one.

3. Combine with --type for precision. If you’re matching appropriations, add --type appropriation to exclude riders, directives, and other provision types from the results:

   congress-approp search --dir data --similar 118-hr9468:0 --type appropriation --top 5
   

4. Check both directions. If provision A in bill X matches provision B in bill Y at 0.85, provision B in bill Y should also match provision A in bill X at a similar score. If it doesn’t, something is off.

5. Low max similarity means the program is unique. If your source provision’s best match in another bill is below 0.55, the program may genuinely not exist in that bill. This is useful for identifying new programs or eliminated ones.

Summary

Next Steps

• Use Semantic Search — search by meaning using text queries instead of provision references
• Compare Two Bills — account-level comparison using name matching
• How Semantic Search Works — understand the embedding and cosine similarity mechanics

Extract Your Own Bill

You will need: congress-approp installed, CONGRESS_API_KEY (free), ANTHROPIC_API_KEY. Optionally: OPENAI_API_KEY for embeddings.

You will learn: How to go from zero to queryable data — downloading a bill from Congress.gov, extracting provisions with Claude, verifying the results, and optionally generating embeddings for semantic search.

The included example data covers three FY2024 bills, but there are dozens of enacted appropriations bills across recent congresses. This tutorial walks you through the full pipeline for extracting any bill you want.

Step 1: Get Your API Keys

You need two keys to run the full pipeline. A third is optional for semantic search.

Set them in your shell:

export CONGRESS_API_KEY="your-congress-key"
export ANTHROPIC_API_KEY="your-anthropic-key"
# Optional:
export OPENAI_API_KEY="your-openai-key"

Step 2: Test Connectivity

Verify that your API keys work before spending time on a full extraction:

congress-approp api test

This checks both the Congress.gov and Anthropic APIs. You should see confirmation that both are reachable and your keys are valid.

Step 3: Discover Available Bills

Use the api bill list command to see what appropriations bills exist for a given congress:

# List all appropriations bills for the 118th Congress (2023-2024)
congress-approp api bill list --congress 118

# List only enacted appropriations bills
congress-approp api bill list --congress 118 --enacted-only

The --enacted-only flag filters to bills that were signed into law — these are the ones that actually became binding spending authority. You’ll see a list with bill type, number, title, and status.

Congress numbers

Each Congress spans two years:

Bill type codes

When downloading a specific bill, you need the bill type code:

Most enacted appropriations bills originate in the House (hr), since the Constitution requires revenue and spending bills to originate there.

Step 4: Download the Bill

Download a single bill

If you know the specific bill you want:

congress-approp download --congress 118 --type hr --number 9468 --output-dir data

This fetches the enrolled (final, signed into law) XML from Congress.gov and saves it to data/118/hr/9468/BILLS-118hr9468enr.xml.

Download all enacted bills for a congress

To get everything at once:

congress-approp download --congress 118 --enacted-only --output-dir data

This scans for all enacted appropriations bills in the 118th Congress and downloads their enrolled XML. It may take a minute or two depending on how many bills there are.

Preview without downloading

Use --dry-run to see what would be downloaded without actually fetching anything:

congress-approp download --congress 118 --enacted-only --output-dir data --dry-run

Step 5: Preview the Extraction (Dry Run)

Before making any LLM API calls, preview what the extraction will look like:

congress-approp extract --dir data/118/hr/9468 --dry-run

The dry run shows you:

• Chunk count: How many chunks the bill will be split into. Small bills (like the VA supplemental) are a single chunk. The FY2024 omnibus splits into 75 chunks.
• Estimated input tokens: How many tokens will be sent to the LLM. This helps you estimate cost before committing.

Here’s what to expect for different bill sizes:

Step 6: Run the Extraction

Now run the actual extraction:

congress-approp extract --dir data/118/hr/9468

For the small VA supplemental, this completes in under a minute. Here’s what happens:

1. Parse: The XML is parsed to extract clean text and identify chunk boundaries
2. Extract: Each chunk is sent to Claude with a detailed system prompt defining every provision type
3. Merge: Provisions from all chunks are combined into a single list
4. Compute: Budget authority totals are computed from the individual provisions (never trusting the LLM’s arithmetic)
5. Verify: Every dollar amount and text excerpt is checked against the source XML
6. Write: All artifacts are saved to disk

Controlling parallelism

For large bills with many chunks, you can control how many LLM calls run simultaneously:

# Default: 5 concurrent calls
congress-approp extract --dir data/118/hr/4366

# Faster but uses more API quota
congress-approp extract --dir data/118/hr/4366 --parallel 8

# Conservative — one at a time
congress-approp extract --dir data/118/hr/4366 --parallel 1

Higher parallelism is faster but may hit API rate limits. The default of 5 is a good balance.

Using a different model

By default, extraction uses claude-opus-4-6. You can override this:

# Via flag
congress-approp extract --dir data/118/hr/9468 --model claude-sonnet-4-20250514

# Via environment variable
export APPROP_MODEL="claude-sonnet-4-20250514"
congress-approp extract --dir data/118/hr/9468

Caution: The system prompt and expected output format are tuned for Claude Opus. Other models may produce lower-quality extractions with more classification errors or missing provisions. Always check the audit output after extracting with a non-default model.

Progress display

For multi-chunk bills, a progress dashboard shows real-time status:

  5/42, 187 provs [4m 23s] 842 tok/s | 📝A-IIb ~8K 180/s | 🤔B-I ~3K | 📝B-III ~1K 95/s

This tells you: 5 of 42 chunks complete, 187 provisions extracted so far, running for 4 minutes 23 seconds, with three chunks currently being processed.

Step 7: Check the Output Files

After extraction, your bill directory contains several new files:

data/118/hr/9468/
├── BILLS-118hr9468enr.xml     ← Source XML (downloaded in Step 4)
├── extraction.json            ← All provisions with amounts, accounts, sections
├── verification.json          ← Deterministic checks against source text
├── metadata.json              ← Model name, prompt version, timestamps, source hash
├── tokens.json                ← LLM token usage (input, output, cache hits)
└── chunks/                    ← Per-chunk LLM artifacts (thinking traces, raw responses)

Step 8: Verify the Extraction

Run the audit command to check quality:

congress-approp audit --dir data/118/hr/9468

┌───────────┬────────────┬──────────┬──────────┬───────┬───────┬──────────┬───────────┬──────────┬──────────┐
│ Bill      ┆ Provisions ┆ Verified ┆ NotFound ┆ Ambig ┆ Exact ┆ NormText ┆ Spaceless ┆ TextMiss ┆ Coverage │
╞═══════════╪════════════╪══════════╪══════════╪═══════╪═══════╪══════════╪═══════════╪══════════╪══════════╡
│ H.R. 9468 ┆          7 ┆        2 ┆        0 ┆     0 ┆     5 ┆        0 ┆         0 ┆        2 ┆   100.0% │
└───────────┴────────────┴──────────┴──────────┴───────┴───────┴──────────┴───────────┴──────────┴──────────┘

What to check:

1. NotFound should be 0. If any dollar amounts weren’t found in the source text, investigate with audit --verbose.
2. Exact should be high. This means the raw text excerpts are byte-identical to the source — the LLM copied the text faithfully.
3. Coverage ideally ≥ 90%. Coverage below 100% isn’t necessarily a problem — see What Coverage Means.

If NotFound > 0, run the verbose audit to see which provisions failed:

congress-approp audit --dir data/118/hr/9468 --verbose

This lists each problematic provision with its dollar string, allowing you to manually check against the source XML.

Step 9: Query Your Data

All the same commands you used with the example data now work on your extracted bill:

# Summary
congress-approp summary --dir data/118/hr/9468

# Search for specific provisions
congress-approp search --dir data/118/hr/9468 --type appropriation

# Compare with the examples
congress-approp compare --base data/118-hr4366 --current data/118/hr/9468

You can also point --dir at a parent directory to load multiple bills at once:

# Load everything under data/
congress-approp summary --dir data

# Search across all extracted bills
congress-approp search --dir data --keyword "Veterans Affairs"

The loader walks recursively from whatever --dir you specify, finding every extraction.json file.

Step 10 (Optional): Generate Embeddings

If you want semantic search and --similar matching for your newly extracted bill, generate embeddings:

export OPENAI_API_KEY="your-key"
congress-approp embed --dir data/118/hr/9468

This sends each provision’s text to OpenAI’s text-embedding-3-large model and saves the vectors locally. For a small bill (7 provisions), this takes a few seconds. For the omnibus (2,364 provisions), about 30 seconds.

Preview token usage

congress-approp embed --dir data/118/hr/9468 --dry-run

Shows how many provisions would be embedded and estimated token count without making any API calls.

After embedding

Now semantic search works on your bill:

congress-approp search --dir data --semantic "school lunch programs" --top 5
congress-approp search --dir data --similar 118-hr9468:0 --top 5

The embed command writes two files:

• embeddings.json — Metadata: model name, dimensions, provision count, SHA-256 of the extraction it was built from
• vectors.bin — Binary float32 vectors (count × dimensions × 4 bytes)

See Generate Embeddings for detailed options.

Re-Extracting a Bill

If you want to re-extract a bill — perhaps with a newer model or after a schema update — simply run extract again. It will overwrite the existing extraction.json and verification.json.

After re-extracting, the embeddings become stale. The tool detects this via the hash chain and warns you:

⚠ H.R. 9468: embeddings are stale (extraction.json has changed)

Run embed again to regenerate them.

If you only need to re-verify without re-extracting (for example, after a schema upgrade), use the upgrade command instead:

congress-approp upgrade --dir data/118/hr/9468

This re-deserializes the existing extraction through the current code’s schema, re-runs verification, and updates the files — no LLM calls needed. See Upgrade Extraction Data for details.

Estimating Costs

The tokens.json file records exact token usage after extraction. Here are typical numbers from the example bills:

Embedding costs are much lower — approximately $0.01 per bill for text-embedding-3-large.

Use extract --dry-run and embed --dry-run to preview token counts before committing to API calls.

Quick Reference: Full Pipeline

Here’s the complete sequence for extracting a bill from scratch:

# 1. Set API keys
export CONGRESS_API_KEY="..."
export ANTHROPIC_API_KEY="..."
export OPENAI_API_KEY="..."  # optional, for embeddings

# 2. Find the bill
congress-approp api bill list --congress 118 --enacted-only

# 3. Download
congress-approp download --congress 118 --type hr --number 4366 --output-dir data

# 4. Preview extraction
congress-approp extract --dir data/118/hr/4366 --dry-run

# 5. Extract
congress-approp extract --dir data/118/hr/4366 --parallel 6

# 6. Verify
congress-approp audit --dir data/118/hr/4366

# 7. Generate embeddings (optional)
congress-approp embed --dir data/118/hr/4366

# 8. Query
congress-approp summary --dir data
congress-approp search --dir data --type appropriation

Troubleshooting

“No XML files found”

Make sure you downloaded the bill first (congress-approp download). The extract command looks for BILLS-*.xml files in the specified directory.

“Rate limited” errors during extraction

Reduce parallelism: extract --parallel 2. Anthropic’s API has per-minute token limits that can be exceeded with high concurrency on large bills.

Low coverage after extraction

Run audit --verbose to see which dollar amounts in the source text weren’t captured. Common causes:

• Statutory cross-references: Dollar amounts from other laws cited in the bill text — correctly excluded
• Struck amounts: “Striking ‘$50,000’ and inserting ‘$75,000’” — the old amount shouldn’t be extracted
• Loan guarantee ceilings: Not budget authority — correctly excluded

If legitimate provisions are missing, consider re-extracting with a higher-capability model.

Stale embeddings warning

After re-extracting, the hash chain detects that extraction.json has changed but embeddings.json still references the old version. Run congress-approp embed --dir <path> to regenerate.

Next Steps

• Verify Extraction Accuracy — detailed guide for auditing results
• Generate Embeddings — embedding options and configuration
• Filter and Search Provisions — all search flags for querying your new data

Export Data for Spreadsheets and Scripts

You will need: congress-approp installed, access to the data/ directory.

You will learn: How to get appropriations data into Excel, Google Sheets, Python, R, and shell pipelines using the four output formats: CSV, JSON, JSONL, and table.

The congress-approp CLI is great for interactive exploration, but most analysis workflows eventually need the data in another tool — a spreadsheet for a briefing, a pandas DataFrame for statistical analysis, or a jq pipeline for automation. Every query command supports four output formats via the --format flag, and this tutorial shows you how to use each one effectively.

CSV for Spreadsheets

CSV is the most portable format for getting data into Excel, Google Sheets, LibreOffice Calc, or any other spreadsheet application.

Basic export

congress-approp search --dir data --type appropriation --format csv > appropriations.csv

This writes a file with a header row and one row per matching provision. Here’s what the first few lines look like:

bill,provision_type,account_name,description,agency,dollars,old_dollars,semantics,detail_level,section,division,raw_text,amount_status,match_tier,quality,provision_index
H.R. 9468,appropriation,Compensation and Pensions,Compensation and Pensions,Department of Veterans Affairs,2285513000,,new_budget_authority,,,,For an additional amount for ''Compensation and Pensions''...,found,exact,strong,0
H.R. 9468,appropriation,Readjustment Benefits,Readjustment Benefits,Department of Veterans Affairs,596969000,,new_budget_authority,,,,For an additional amount for ''Readjustment Benefits''...,found,exact,strong,1

Columns in CSV output

The CSV includes the same fields as the JSON output, flattened into columns:

⚠️ Don’t sum the dollars column directly. The export includes sub-allocations and reference amounts that would double-count money already in a parent line item. Without filtering, a naive sum can overcount budget authority by 2x or more.

To compute correct budget authority totals:

• Filter to semantics == new_budget_authority
• Exclude detail_level == sub_allocation and detail_level == proviso_amount

Or use congress-approp summary which does this correctly and automatically.

Computing totals correctly

In Excel or Google Sheets:

1. Open the CSV
2. Add a filter on the semantics column → select only new_budget_authority
3. Add a filter on the detail_level column → deselect sub_allocation and proviso_amount
4. Sum the filtered dollars column

With jq (command line):

congress-approp search --dir data --type appropriation --format jsonl \
  | jq -s '[.[] | select(.semantics == "new_budget_authority" and .detail_level != "sub_allocation" and .detail_level != "proviso_amount") | .dollars] | add'

With Python:

import csv
with open("provisions.csv") as f:
    rows = list(csv.DictReader(f))
ba = sum(int(r["dollars"]) for r in rows
         if r["dollars"]
         and r["semantics"] == "new_budget_authority"
         and r["detail_level"] not in ("sub_allocation", "proviso_amount"))
print(f"Budget Authority: ${ba:,}")

Tip: When you export to CSV/JSON/JSONL, the tool prints a summary to stderr showing how many provisions have each semantics type and the budget authority total. Watch for this — it tells you immediately whether filtering is needed.

Opening in Excel

1. Open Excel
2. File → Open → navigate to your .csv file
3. If Excel doesn’t auto-detect columns, use Data → From Text/CSV and select UTF-8 encoding
4. The dollars column will be numeric — you can format it as currency or with comma separators

Gotchas to watch for:

• Large numbers: Excel may display very large dollar amounts in scientific notation (e.g., 8.46E+11). Format the column as Number with 0 decimal places.
• Leading zeros: Not an issue here since bill numbers don’t have leading zeros, but be aware that CSV import can strip them in other contexts.
• UTF-8 characters: Bill text contains em-dashes (—), curly quotes, and other Unicode characters. Make sure your import specifies UTF-8 encoding. On Windows, this sometimes requires the “From Text/CSV” import wizard rather than a simple File → Open.
• Commas in text: The raw_text and description fields may contain commas. The CSV output properly quotes these fields, but some older CSV parsers may not handle quoted fields correctly.

Opening in Google Sheets

1. Go to Google Sheets → File → Import → Upload
2. Select your .csv file
3. Import location: “Replace current sheet” or “Insert new sheet”
4. Separator type: Comma (should auto-detect)
5. Google Sheets handles UTF-8 natively — no encoding issues

Useful CSV exports

# All appropriations across all example bills
congress-approp search --dir data --type appropriation --format csv > all_appropriations.csv

# Just the VA accounts
congress-approp search --dir data --agency "Veterans" --format csv > va_provisions.csv

# Rescissions over $100 million
congress-approp search --dir data --type rescission --min-dollars 100000000 --format csv > big_rescissions.csv

# CR substitutions with old and new amounts
congress-approp search --dir data --type cr_substitution --format csv > cr_anomalies.csv

# Everything in Division A (MilCon-VA)
congress-approp search --dir data/118-hr4366 --division A --format csv > milcon_va.csv

# Summary table as CSV
congress-approp summary --dir data --format csv > bill_summary.csv

JSON for Programmatic Use

JSON output includes every field for each matching provision as an array of objects. It’s the richest output format and the best choice for Python, JavaScript, R, or any other programming language.

Basic export

congress-approp search --dir data/118-hr9468 --type appropriation --format json

[
  {
    "account_name": "Compensation and Pensions",
    "agency": "Department of Veterans Affairs",
    "amount_status": "found",
    "bill": "H.R. 9468",
    "description": "Compensation and Pensions",
    "division": "",
    "dollars": 2285513000,
    "match_tier": "exact",
    "old_dollars": null,
    "provision_index": 0,
    "provision_type": "appropriation",
    "quality": "strong",
    "raw_text": "For an additional amount for ''Compensation and Pensions'', $2,285,513,000, to remain available until expended.",
    "section": "",
    "semantics": "new_budget_authority"
  },
  {
    "account_name": "Readjustment Benefits",
    "agency": "Department of Veterans Affairs",
    "amount_status": "found",
    "bill": "H.R. 9468",
    "description": "Readjustment Benefits",
    "division": "",
    "dollars": 596969000,
    "match_tier": "exact",
    "old_dollars": null,
    "provision_index": 1,
    "provision_type": "appropriation",
    "quality": "strong",
    "raw_text": "For an additional amount for ''Readjustment Benefits'', $596,969,000, to remain available until expended.",
    "section": "",
    "semantics": "new_budget_authority"
  }
]

Five jq One-Liners Every Analyst Needs

If you have jq installed (a lightweight JSON processor), you can do filtering and aggregation directly from the command line:

1. Total budget authority across all appropriations:

congress-approp search --dir data --type appropriation --format json | \
  jq '[.[] | select(.semantics == "new_budget_authority") | .dollars] | add'

862137099554

2. Top 10 accounts by dollar amount:

congress-approp search --dir data --type appropriation --format json | \
  jq '[.[] | select(.dollars != null)] | sort_by(-.dollars) | .[:10] | .[] | "\(.dollars)\t\(.account_name)"'

3. Group by agency and sum budget authority:

congress-approp search --dir data --type appropriation --format json | \
  jq 'group_by(.agency) | map({
    agency: .[0].agency,
    total: [.[] | .dollars // 0] | add,
    count: length
  }) | sort_by(-.total) | .[:10]'

4. Find all provisions in Division A over $1 billion:

congress-approp search --dir data --format json | \
  jq '[.[] | select(.division == "A" and (.dollars // 0) > 1000000000)]'

5. Extract just account names (unique, sorted):

congress-approp search --dir data --type appropriation --format json | \
  jq '[.[].account_name] | unique | sort | .[]'

Loading JSON in Python

import json

# Method 1: From a file
with open("appropriations.json") as f:
    provisions = json.load(f)

# Method 2: From subprocess
import subprocess
result = subprocess.run(
    ["congress-approp", "search", "--dir", "data",
     "--type", "appropriation", "--format", "json"],
    capture_output=True, text=True
)
provisions = json.loads(result.stdout)

# Work with the data
for p in provisions:
    if p["dollars"] and p["dollars"] > 1_000_000_000:
        print(f"{p['account_name']}: ${p['dollars']:,.0f}")

Loading JSON in pandas

import pandas as pd
import json

# Load search output
df = pd.read_json("appropriations.json")

# Basic analysis
print(f"Total provisions: {len(df)}")
print(f"Total BA: ${df[df['semantics'] == 'new_budget_authority']['dollars'].sum():,.0f}")
print(f"\nBy agency:")
print(df.groupby("agency")["dollars"].sum().sort_values(ascending=False).head(10))

Loading JSON in R

library(jsonlite)

provisions <- fromJSON("appropriations.json")

# Filter to appropriations with budget authority
ba <- provisions[provisions$semantics == "new_budget_authority" & !is.na(provisions$dollars), ]

# Top 10 by dollars
head(ba[order(-ba$dollars), c("account_name", "agency", "dollars")], 10)

JSONL for Streaming

JSONL (JSON Lines) outputs one JSON object per line, with no enclosing array brackets. This is ideal for:

• Streaming processing (each line is independently parseable)
• Piping to while read loops in shell scripts
• Processing very large result sets without loading everything into memory
• Tools like xargs and parallel

Basic usage

congress-approp search --dir data --type appropriation --format jsonl

Each line is a complete JSON object:

{"account_name":"Compensation and Pensions","agency":"Department of Veterans Affairs","amount_status":"found","bill":"H.R. 9468","description":"Compensation and Pensions","division":"","dollars":2285513000,...}
{"account_name":"Readjustment Benefits","agency":"Department of Veterans Affairs","amount_status":"found","bill":"H.R. 9468","description":"Readjustment Benefits","division":"","dollars":596969000,...}
...

Shell processing examples

# Count provisions per bill
congress-approp search --dir data --format jsonl | \
  jq -r '.bill' | sort | uniq -c | sort -rn

# Extract account names line by line
congress-approp search --dir data --type appropriation --format jsonl | \
  while IFS= read -r line; do
    echo "$line" | jq -r '.account_name'
  done

# Filter and reformat in one pipeline
congress-approp search --dir data --type rescission --format jsonl | \
  jq -r 'select(.dollars > 1000000000) | "\(.bill)\t$\(.dollars)\t\(.account_name)"'

When to use JSONL vs. JSON

Working with extraction.json Directly

Sometimes the CLI search output doesn’t give you exactly what you need. The raw extraction.json file contains the complete data with nested structures that the CLI flattens.

Structure

{
  "schema_version": "1.0",
  "bill": {
    "identifier": "H.R. 9468",
    "classification": "supplemental",
    "short_title": "Veterans Benefits Continuity and Accountability Supplemental Appropriations Act, 2024",
    "fiscal_years": [2024],
    "divisions": [],
    "public_law": null
  },
  "provisions": [
    {
      "provision_type": "appropriation",
      "account_name": "Compensation and Pensions",
      "agency": "Department of Veterans Affairs",
      "amount": {
        "value": { "kind": "specific", "dollars": 2285513000 },
        "semantics": "new_budget_authority",
        "text_as_written": "$2,285,513,000"
      },
      "detail_level": "top_level",
      "availability": "to remain available until expended",
      "fiscal_year": 2024,
      "confidence": 0.99,
      "raw_text": "For an additional amount for ''Compensation and Pensions'', $2,285,513,000, to remain available until expended.",
      "notes": ["Supplemental appropriation under Veterans Benefits Administration heading", "No-year funding"],
      "cross_references": [],
      "section": "",
      "division": null,
      "title": null,
      "provisos": [],
      "earmarks": [],
      "parent_account": null,
      "program": null
    }
  ],
  "summary": { ... },
  "chunk_map": []
}

Key differences from CLI JSON output:

• Nested amount object with value, semantics, and text_as_written sub-fields
• notes array — explanatory annotations the LLM added
• cross_references array — references to other laws and sections
• provisos array — “Provided, That” conditions
• earmarks array — community project funding items
• confidence float — LLM self-assessed confidence (0.0–1.0)
• availability string — fund availability period

Flattening nested data in Python

import json
import pandas as pd

with open("data/118-hr9468/extraction.json") as f:
    data = json.load(f)

# Flatten provisions with nested amounts
rows = []
for p in data["provisions"]:
    row = {
        "provision_type": p["provision_type"],
        "account_name": p.get("account_name", ""),
        "agency": p.get("agency", ""),
        "section": p.get("section", ""),
        "division": p.get("division", ""),
        "confidence": p.get("confidence", 0),
        "raw_text": p.get("raw_text", ""),
        "notes": "; ".join(p.get("notes", [])),
    }

    # Flatten the amount
    amt = p.get("amount")
    if amt:
        val = amt.get("value", {})
        row["dollars"] = val.get("dollars") if val.get("kind") == "specific" else None
        row["semantics"] = amt.get("semantics", "")
        row["text_as_written"] = amt.get("text_as_written", "")

    rows.append(row)

df = pd.DataFrame(rows)
print(df[["provision_type", "account_name", "dollars", "semantics"]].to_string())

Finding provisions with specific notes

The notes field contains useful annotations that the CLI doesn’t display:

import json

with open("data/118-hr4366/extraction.json") as f:
    data = json.load(f)

# Find all provisions noted as advance appropriations
for i, p in enumerate(data["provisions"]):
    notes = p.get("notes", [])
    for note in notes:
        if "advance" in note.lower():
            acct = p.get("account_name", "unknown")
            amt = p.get("amount", {}).get("value", {}).get("dollars", "N/A")
            print(f"[{i}] {acct}: ${amt:,} — {note}")

Summary: Choosing the Right Format

For most analysis tasks, start with --format json or --format csv. Only read extraction.json directly when you need nested fields like notes, cross_references, or provisos that the CLI output flattens away.

Next Steps

• Filter and Search Provisions — all search flags for narrowing results before export
• extraction.json Fields — complete field reference for the raw JSON
• Output Formats — format reference with full column lists

Use Semantic Search

You will need: congress-approp installed, access to the data/ directory, OPENAI_API_KEY environment variable set.

You will learn: How to find provisions by meaning instead of keywords, how to interpret similarity scores, how to use --similar for cross-bill matching, and when semantic search is (and isn’t) the right tool.

Keyword search finds provisions that contain the exact words you type. Semantic search finds provisions that mean what you’re looking for — even when the words are completely different. This is the difference between searching for “school lunch” (zero results in appropriations language) and finding “$33 billion for Child Nutrition Programs” (the actual provision that funds school lunches).

This tutorial walks through setup, real queries against the example data, and practical techniques for getting the best results.

Prerequisites

Semantic search requires two things:

1. Pre-computed embeddings for the bills you want to search. The included example data already has these — you don’t need to generate them.
2. OPENAI_API_KEY set in your environment. This is needed at query time to embed your search text (a single API call, ~100ms, costs fractions of a cent).

export OPENAI_API_KEY="your-key-here"

If you’re working with your own extracted bills that don’t have embeddings yet, generate them first:

congress-approp embed --dir your-data-directory

See Generate Embeddings for details.

Your First Semantic Search

The following example searches for a concept using everyday language that shares no keywords with the matching provision:

congress-approp search --dir data --semantic "school lunch programs for kids" --top 5

┌──────┬───────────┬───────────────┬─────────────────────────────────────────────┬────────────────┬─────┐
│ Sim  ┆ Bill      ┆ Type          ┆ Description / Account                       ┆     Amount ($) ┆ Div │
╞══════╪═══════════╪═══════════════╪═════════════════════════════════════════════╪════════════════╪═════╡
│ 0.51 ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆ 33,266,226,000 ┆ B   │
│ 0.46 ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆     10,000,000 ┆ B   │
│ 0.45 ┆ H.R. 4366 ┆ rider         ┆ Pilot project grant recipients shall be r…  ┆              — ┆ B   │
│ 0.45 ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆     18,004,000 ┆ B   │
│ 0.44 ┆ H.R. 4366 ┆ appropriation ┆ Child Nutrition Programs                    ┆      5,000,000 ┆ B   │
└──────┴───────────┴───────────────┴─────────────────────────────────────────────┴────────────────┴─────┘
5 provisions found

Not a single word in “school lunch programs for kids” appears in “Child Nutrition Programs” — and yet it’s the top result at 0.51 similarity. The embedding model understands that school lunches and child nutrition are the same concept.

Compare this to a keyword search for the same phrase:

congress-approp search --dir data --keyword "school lunch"

0 provisions found

Zero results. Keyword search can only find provisions containing the literal words “school lunch,” which no provision in any of these bills does.

Understanding the Sim Column

When you use --semantic or --similar, the table gains a Sim column showing the cosine similarity between your query and each provision’s embedding vector. Scores range from 0 to 1:

Key insight: A score of 0.51 for “school lunch” → “Child Nutrition Programs” is strong for a conceptual translation query. Scores above 0.80 typically occur only when comparing the same program in different bills.

More Queries to Try

These examples demonstrate different types of semantic matching. Try each one against the example data:

Layperson → Bureaucratic Translation

The most common use case — you know what you want in plain English, but the bill uses formal government terminology:

# Plain language → official program names
congress-approp search --dir data --semantic "money for fixing roads and bridges" --top 5
# → Highway Infrastructure Programs, Federal-Aid Highways, National Infrastructure Investments

congress-approp search --dir data --semantic "space exploration and rockets" --top 5
# → Exploration (NASA), Space Operations, Space Technology

congress-approp search --dir data --semantic "fighting wildfires" --top 5
# → Wildland Fire Management, Wildfire Suppression Operations Reserve Fund

congress-approp search --dir data --semantic "help for homeless veterans" --top 5
# → Homeless Assistance Grants, various VA provisions

Topic Discovery

When you’re exploring a policy area without knowing specific program names:

# What's in the bill about clean energy?
congress-approp search --dir data --semantic "clean energy research" --top 10

# What about drug enforcement?
congress-approp search --dir data --semantic "drug enforcement and narcotics control" --top 10

# Nuclear weapons and defense?
congress-approp search --dir data --semantic "nuclear weapons maintenance and modernization" --top 10

News Story → Provisions

Paste a phrase from a news article to find the relevant provisions:

# From a headline about the opioid crisis
congress-approp search --dir data --semantic "opioid crisis drug treatment" --top 5

# From a story about border security
congress-approp search --dir data --semantic "border wall construction and immigration enforcement" --top 5

# From a story about scientific research funding
congress-approp search --dir data --semantic "federal funding for scientific research grants" --top 10

Combining Semantic Search with Filters

Semantic search provides the ranking (which provisions are most relevant to your query). Hard filters provide constraints (which provisions are even eligible to appear). When combined, the filters apply first, then semantic ranking orders the remaining results.

Filter by provision type

If you only want appropriation-type provisions (not riders, directives, or limitations):

congress-approp search --dir data --semantic "clean energy" --type appropriation --top 5

This is useful because semantic search doesn’t distinguish provision types — a rider about clean energy policy scores as high as an appropriation for clean energy funding. Adding --type appropriation ensures you only see provisions with dollar amounts.

Filter by dollar range

Find large provisions about a topic:

congress-approp search --dir data --semantic "scientific research" --type appropriation --min-dollars 1000000000 --top 5

This returns only appropriations of $1 billion or more that are semantically related to scientific research.

Filter by division

Focus on a specific part of the omnibus:

# Only Division A (MilCon-VA)
congress-approp search --dir data --semantic "veterans health care" --division A --top 5

# Only Division B (Agriculture)
congress-approp search --dir data --semantic "farm subsidies" --division B --top 5

Combine multiple filters

congress-approp search --dir data \
  --semantic "renewable energy and climate" \
  --type appropriation \
  --min-dollars 100000000 \
  --division D \
  --top 10

This finds the top 10 appropriations of $100M+ in Division D (Energy and Water) related to renewable energy and climate.

Finding Similar Provisions with –similar

While --semantic embeds a text query and searches for matching provisions, --similar takes an existing provision and finds the most similar provisions across all loaded bills. This is the cross-bill matching tool.

Basic usage

The syntax is --similar <bill_directory>:<provision_index>:

congress-approp search --dir data --similar 118-hr9468:0 --top 5

┌──────┬───────────┬───────────────┬────────────────────────────────┬─────────────────┬─────┐
│ Sim  ┆ Bill      ┆ Type          ┆ Description / Account          ┆      Amount ($) ┆ Div │
╞══════╪═══════════╪═══════════════╪════════════════════════════════╪═════════════════╪═════╡
│ 0.86 ┆ H.R. 4366 ┆ appropriation ┆ Compensation and Pensions      ┆ 182,310,515,000 ┆ A   │
│ 0.78 ┆ H.R. 4366 ┆ appropriation ┆ Compensation and Pensions      ┆  15,072,388,000 ┆ A   │
│ 0.73 ┆ H.R. 4366 ┆ limitation    ┆ Compensation and Pensions      ┆      22,109,000 ┆ A   │
│ 0.70 ┆ H.R. 9468 ┆ appropriation ┆ Readjustment Benefits          ┆     596,969,000 ┆     │
│ 0.68 ┆ H.R. 4366 ┆ rescission    ┆ Medical Support and Compliance ┆   1,550,000,000 ┆ A   │
└──────┴───────────┴───────────────┴────────────────────────────────┴─────────────────┴─────┘
5 provisions found

Here 118-hr9468:0 means “provision index 0 in the hr9468 directory” — that’s the VA Supplemental’s Compensation and Pensions appropriation. The top match in the omnibus is the same account at 0.86 similarity.

Key difference from –semantic

Because --similar doesn’t make any API calls, it’s instant and free. It looks up the source provision’s pre-computed vector and computes cosine similarity against every other provision’s vector locally.

Finding the provision index

To use --similar, you need the provision index. There are several ways to find it:

Method 1: Use --format json and look for the provision_index field:

congress-approp search --dir data/118-hr9468 --type appropriation --format json | \
  jq '.[] | "\(.provision_index): \(.account_name) $\(.dollars)"'

"0: Compensation and Pensions $2285513000"
"1: Readjustment Benefits $596969000"

Method 2: In the table output, count rows from the top (zero-indexed). The first row is index 0, the second is index 1, and so on within each bill.

Method 3: For a specific account, search for it and note the provision_index in the JSON output.

Cross-bill matching with different naming conventions

CRs and omnibus bills often use different naming conventions for the same account. Embeddings handle this because they capture meaning, not just words:

• CR: "Rural Housing Service—Rural Community Facilities Program Account"
• Omnibus: "Rural Community Facilities Program Account"

Despite the different names, --similar will match these at approximately 0.78 similarity — well above the threshold for confident matching.

When Semantic Search Doesn’t Work

Semantic search has limitations. Here are situations where other approaches work better:

Exact account name lookups

If you know the precise account name, --account is faster, deterministic, and doesn’t require an API key:

# Better than semantic search for exact lookups
congress-approp search --dir data --account "Child Nutrition Programs"

No conceptual match in the dataset

If you search for a topic that genuinely isn’t in the bills, similarity scores will be low — and that’s the correct answer:

congress-approp search --dir data --semantic "cryptocurrency regulation bitcoin blockchain" --top 3

┌──────┬───────────┬───────────────┬───────────────────────────────┬─────────────┬─────┐
│ Sim  ┆ Bill      ┆ Type          ┆ Description / Account         ┆  Amount ($) ┆ Div │
╞══════╪═══════════╪═══════════════╪═══════════════════════════════╪═════════════╪═════╡
│ 0.30 ┆ H.R. 4366 ┆ appropriation ┆ Regulation and Technology     ┆  62,400,000 ┆ E   │
│ 0.29 ┆ H.R. 4366 ┆ appropriation ┆ Regulation and Technology     ┆      40,000 ┆ E   │
│ 0.29 ┆ H.R. 4366 ┆ appropriation ┆ Regulation and Technology     ┆ 116,186,000 ┆ E   │
└──────┴───────────┴───────────────┴───────────────────────────────┴─────────────┴─────┘
3 provisions found

Scores of 0.29–0.30 are well below any meaningful threshold. The tool correctly surfaces the closest things it has (NRC “Regulation and Technology” — the word “regulation” provides a weak signal) but the low scores tell you: nothing in this dataset is actually about cryptocurrency.

Treat scores below 0.40 as “no meaningful match.”

Distinguishing provision types by embedding

Embeddings capture what the provision is about, not what type of action it is. A rider that prohibits funding for abortions and an appropriation for reproductive health services may score highly similar because they’re about the same topic — even though they represent opposite policy actions.

If provision type matters, always combine semantic search with --type:

# Find appropriations about reproductive health, not policy riders
congress-approp search --dir data --semantic "reproductive health" --type appropriation --top 5

Query instability

Different phrasings of the same question can produce somewhat different results. In experiments, five different phrasings of a FEMA-related query shared only one common provision in their top-5 results. This is a known property of embedding models.

Mitigation: If the topic matters, try 2–3 different phrasings and take the union of results. A future --multi-query feature will automate this.

Cost and Performance

Semantic search is fast and inexpensive:

Embedding generation (one-time per bill):

The embedding model is text-embedding-3-large with 3,072 dimensions. Vectors are stored as binary float32 files that load in milliseconds.

How It Works Under the Hood

For a detailed technical explanation, see How Semantic Search Works. In brief:

1. At embed time: Each provision’s meaningful text (account name + agency + bill text) is sent to OpenAI’s embedding model, which returns a 3,072-dimensional vector. These vectors are stored in vectors.bin.

2. At query time (–semantic): Your search text is sent to the same model (one API call). The returned vector is compared to every stored provision vector using cosine similarity (the dot product of normalized vectors). Results are ranked by similarity.

3. At query time (–similar): The source provision’s vector is looked up from the stored vectors.bin. No API call needed — everything is local.

4. The math: Cosine similarity measures the angle between two vectors in 3,072-dimensional space. Vectors pointing in the same direction (similar meaning) have high cosine similarity; vectors pointing in different directions (different meanings) have low similarity.

Tips for Effective Semantic Search

1. Be descriptive, not terse. “Federal funding for scientific research at universities” works better than just “science.” Longer queries give the embedding model more context.

2. Use domain language when you know it. “SNAP benefits supplemental nutrition” will rank higher than “food stamps for poor people” because the embedding model has seen more formal language in its training data.

3. Combine with hard filters. Semantic search ranks; filters constrain. Use them together:

   congress-approp search --dir data --semantic "your query" --type appropriation --min-dollars 1000000 --top 10
   

4. Try both --semantic and --similar. If you find one good provision via semantic search, switch to --similar with that provision’s index to find related provisions across other bills without additional API calls.

5. Trust low scores. If the best match is below 0.40, the topic likely isn’t in the dataset. Don’t force an interpretation.

6. Check results with keyword search. After semantic search finds a promising account, verify with --account or --keyword to make sure you’re seeing the complete picture:

   # Semantic search found "Child Nutrition Programs" — now get everything for that account
   congress-approp search --dir data --account "Child Nutrition"
   

Quick Reference

Next Steps

• How Semantic Search Works — the full technical explanation of embeddings, cosine similarity, and vector storage
• Track a Program Across Bills — using --similar for cross-bill matching
• Generate Embeddings — creating embeddings for your own extracted bills

Download Bills from Congress.gov

You will need: congress-approp installed, CONGRESS_API_KEY environment variable set.

You will learn: How to discover available appropriations bills, download their enrolled XML, and set up a data directory for extraction.

This guide covers every option for downloading bill XML from Congress.gov. If you just want the quick path, skip to Quick Reference at the end.

Set Up Your API Key

The Congress.gov API requires a free API key. Sign up at api.congress.gov/sign-up — approval is usually instant.

Set the key in your environment:

export CONGRESS_API_KEY="your-key-here"

You can verify connectivity with:

congress-approp api test

Discover Available Bills

Before downloading, you’ll usually want to see what’s available. The api bill list command queries Congress.gov for appropriations bills:

List all appropriations bills for a congress

congress-approp api bill list --congress 118

This returns every bill in the 118th Congress (2023–2024) that Congress.gov classifies as an appropriations bill — introduced, passed, vetoed, or enacted.

List only enacted bills

Most of the time you only want bills that became law:

congress-approp api bill list --congress 118 --enacted-only

The --enacted-only flag filters to bills signed by the President (or with a veto override). These are the authoritative spending laws.

Congress numbers

Each Congress spans two years. Here are the recent ones:

Note that fiscal years don’t align perfectly with congresses — a bill enacted in the 118th Congress might fund FY2024 (which started October 1, 2023) or FY2025.

Get metadata for a specific bill

If you know which bill you want, you can inspect its metadata before downloading:

congress-approp api bill get --congress 118 --type hr --number 4366

Check available text versions

Bills have multiple text versions (introduced, engrossed, enrolled, etc.). To see what’s available:

congress-approp api bill text --congress 118 --type hr --number 4366

This lists every text version with its format (XML, PDF, HTML) and download URL. For extraction, you want the enrolled (enr) version — the final text signed into law.

Bill Type Codes

When specifying a bill, you need the type code:

Most enacted appropriations bills originate in the House (hr), since the Constitution requires spending bills to originate there. Joint resolutions (hjres, sjres) are sometimes used for continuing resolutions.

Download a Single Bill

To download one specific bill’s enrolled XML:

congress-approp download --congress 118 --type hr --number 9468 --output-dir data

This creates the directory structure and saves the XML:

data/
└── 118/
    └── hr/
        └── 9468/
            └── BILLS-118hr9468enr.xml

The file name follows the Government Publishing Office convention: BILLS-{congress}{type}{number}enr.xml.

Only the enrolled version is downloaded

By default, the tool downloads only the enrolled version (the final text signed into law). This is the version you need for extraction and analysis — one XML file per bill, no clutter.

If you need other text versions (for example, to compare the House-passed version to the final enrolled version), you can request specific versions or all versions:

# Download only the introduced version
congress-approp download --congress 118 --type hr --number 4366 --output-dir data --version ih

# Download all available text versions (introduced, engrossed, enrolled, etc.)
congress-approp download --congress 118 --type hr --number 4366 --output-dir data --all-versions

Available version codes for --version:

Tip: For extraction and analysis, always use the enrolled version (the default). Non-enrolled versions may have different XML structures that the parser doesn’t support. The --all-versions flag is for advanced workflows like tracking how a bill changed during the legislative process.

Download multiple formats

You can download both XML (for extraction) and PDF (for reading) at once:

congress-approp download --congress 118 --type hr --number 4366 --output-dir data --format xml,pdf

Download All Enacted Bills for a Congress

To batch-download every enacted appropriations bill:

congress-approp download --congress 118 --enacted-only --output-dir data

This scans Congress.gov for all enacted appropriations bills in the specified congress, then downloads the enrolled XML for each one. The process may take a minute or two depending on how many bills exist and the API’s response time.

Each bill gets its own directory:

data/
└── 118/
    └── hr/
        ├── 4366/
        │   └── BILLS-118hr4366enr.xml
        ├── 5860/
        │   └── BILLS-118hr5860enr.xml
        └── 9468/
            └── BILLS-118hr9468enr.xml

Preview Without Downloading

Use --dry-run to see what would be downloaded without actually fetching anything:

congress-approp download --congress 118 --enacted-only --output-dir data --dry-run

This queries the API and lists each bill that would be downloaded, along with the file size and output path. Useful for estimating how much data you’re about to pull down.

Choosing an Output Directory

The --output-dir flag controls where bills are saved. The default is ./data. You can use any directory structure you like:

# Default location
congress-approp download --congress 118 --type hr --number 4366

# Custom location
congress-approp download --congress 118 --type hr --number 4366 --output-dir ~/appropriations-data

# Organized by fiscal year (your choice of structure)
congress-approp download --congress 118 --type hr --number 4366 --output-dir data/fy2024

The tool creates intermediate directories as needed. Later, when you run extract, search, summary, and other commands, you point --dir at whatever directory contains your bills — the loader walks recursively to find all extraction.json files.

Handling Rate Limits and Errors

The Congress.gov API has rate limits (typically 5,000 requests per hour for registered users). If you’re downloading many bills in quick succession, you may encounter rate limiting.

Symptoms: HTTP 429 (Too Many Requests) errors, or slow responses.

Solutions:

• Wait a few minutes and retry
• Download bills one at a time rather than in batch
• The tool handles most retries automatically, but persistent rate limiting may require reducing your request frequency

Other common issues:

After Downloading

Once you have the XML, the next step is extraction:

# Preview extraction (no API calls)
congress-approp extract --dir data/118/hr/9468 --dry-run

# Run extraction
congress-approp extract --dir data/118/hr/9468

See Extract Provisions from a Bill for the full extraction guide, or Extract Your Own Bill for the end-to-end tutorial.

Quick Reference

# Set your API key
export CONGRESS_API_KEY="your-key"

# Test connectivity
congress-approp api test

# List enacted bills for a congress
congress-approp api bill list --congress 118 --enacted-only

# Download a single bill
congress-approp download --congress 118 --type hr --number 4366 --output-dir data

# Download all enacted bills for a congress
congress-approp download --congress 118 --enacted-only --output-dir data

# Preview without downloading
congress-approp download --congress 118 --enacted-only --output-dir data --dry-run

# Check available text versions for a bill
congress-approp api bill text --congress 118 --type hr --number 4366

Full Command Reference

congress-approp download [OPTIONS] --congress <CONGRESS>

Options:
    --congress <CONGRESS>      Congress number (e.g., 118 for 2023-2024)
    --type <TYPE>              Bill type: hr, s, hjres, sjres
    --number <NUMBER>          Bill number (used with --type for single-bill download)
    --output-dir <OUTPUT_DIR>  Output directory [default: ./data]
    --enacted-only             Only download enacted (signed into law) bills
    --format <FORMAT>          Download format: xml, pdf [comma-separated] [default: xml]
    --version <VERSION>        Text version filter: enr, ih, eh, es, is
    --all-versions             Download all text versions instead of just enrolled
    --dry-run                  Show what would be downloaded without fetching

Next Steps

• Extract Provisions from a Bill — turn downloaded XML into structured data
• Extract Your Own Bill — the full end-to-end tutorial
• Environment Variables and API Keys — all API key configuration options

Extract Provisions from a Bill

You will need: congress-approp installed, downloaded bill XML (see Download Bills), ANTHROPIC_API_KEY environment variable set.

You will learn: How to run the extraction pipeline, control parallelism and model selection, interpret the output files, and handle common issues.

Extraction is the core step of the pipeline — it sends bill text to Claude, which identifies and classifies every spending provision, then deterministic verification checks every dollar amount against the source. This guide covers all the options and considerations.

Prerequisites

1. Downloaded bill XML. You need at least one BILLS-*.xml file in a bill directory. See Download Bills from Congress.gov.
2. Anthropic API key. Set it in your environment:

export ANTHROPIC_API_KEY="your-key-here"

Preview Before Extracting

Always start with a dry run to see what the extraction will involve:

congress-approp extract --dir data/118/hr/9468 --dry-run

The dry run shows you:

• Bill identifier parsed from the XML
• Chunk count — how many pieces the bill will be split into for parallel processing
• Estimated input tokens — helps you estimate API cost before committing

Typical chunk counts by bill size:

No API calls are made during a dry run.

Run Extraction

Single bill

congress-approp extract --dir data/118/hr/9468

For a small bill (like the VA supplemental), this completes in under a minute. The tool:

1. Parses the XML to extract clean text and identify structural boundaries (divisions, titles)
2. Splits large bills into chunks at division and title boundaries
3. Sends each chunk to Claude with a ~300-line system prompt defining every provision type
4. Merges provisions from all chunks into a single list
5. Computes budget authority totals from the individual provisions (never trusting the LLM’s arithmetic)
6. Verifies every dollar amount and text excerpt against the source XML
7. Writes all artifacts to disk

Multiple bills

Point --dir at a parent directory to extract all bills found underneath:

congress-approp extract --dir data

The tool walks recursively, finds every directory containing a BILLS-*.xml file, and extracts each one. Bills that already have extraction.json are automatically skipped — you can safely re-run the same command after a partial failure and it picks up where it left off. To force re-extraction of already-processed bills, use --force:

# Re-extract everything, even bills that already have extraction.json
congress-approp extract --dir data --force

Enrolled versions only

When a bill directory contains multiple XML versions (enrolled, introduced, engrossed, etc.), the extract command automatically uses only the enrolled version (*enr.xml). Non-enrolled versions are ignored. If no enrolled version exists, all available versions are processed.

This means you don’t need to worry about cleaning up extra XML files — the tool picks the right one automatically.

Resilient processing

If an XML file fails to parse (for example, a non-enrolled version with a different XML structure), the tool logs a warning and continues to the next bill instead of aborting the entire run:

⚠ Skipping data/118/hr/2872/BILLS-118hr2872eas.xml: Failed to parse ... (not a parseable bill XML?)

This means one bad file won’t kill a multi-bill extraction run.

Chunk failure handling

Large bills are split into many chunks for parallel extraction. If any chunk permanently fails after all retries (typically due to API rate limiting or empty responses), the tool aborts that bill by default — it does not write extraction.json. This prevents garbage partial extractions from being saved and mistaken for valid data.

✗ 7148: 113 of 115 chunks failed for H.R. 7148. Aborting to prevent partial extraction.
  Use --continue-on-error to save partial results.
  No extraction.json written for this bill.

The tool then continues to the next bill in the queue. Since no extraction.json was written for the failed bill, re-running the same command will automatically retry it.

If you explicitly want partial results (for example, a bill where 59 of 92 chunks succeeded and you want the 1,600+ provisions that were extracted), use --continue-on-error:

congress-approp extract --dir data/118/hr/2882 --parallel 6 --continue-on-error

This saves the partial extraction.json with whatever chunks succeeded. The audit command will show lower coverage for these partial extractions.

Extract all downloaded bills with parallelism

congress-approp extract --dir data --parallel 6

Controlling Parallelism

The --parallel flag controls how many LLM API calls run simultaneously. This affects both speed and API rate limit usage:

# Default: 5 concurrent calls
congress-approp extract --dir data/118/hr/4366

# Faster — good for large bills if your API quota allows
congress-approp extract --dir data/118/hr/4366 --parallel 8

# Conservative — avoids rate limits, good for debugging
congress-approp extract --dir data/118/hr/4366 --parallel 1

For the FY2024 omnibus (75 chunks), --parallel 6 completes in approximately 60 minutes. At --parallel 1, it would take several hours.

Progress display

For multi-chunk bills, a live progress dashboard shows extraction status:

  5/42, 187 provs [4m 23s] 842 tok/s | 📝A-IIb ~8K 180/s | 🤔B-I ~3K | 📝B-III ~1K 95/s

Reading left to right:

• 5/42 — 5 of 42 chunks complete
• 187 provs — 187 provisions extracted so far
• [4m 23s] — elapsed time
• 842 tok/s — average token throughput
• The remaining items show currently active chunks: 📝 = receiving response, 🤔 = model is thinking

Choosing a Model

By default, extraction uses claude-opus-4-6, which produces the highest quality results. You can override this:

# Via command-line flag
congress-approp extract --dir data/118/hr/9468 --model claude-sonnet-4-20250514

# Via environment variable (useful for scripting)
export APPROP_MODEL="claude-sonnet-4-20250514"
congress-approp extract --dir data/118/hr/9468

The command-line flag takes precedence over the environment variable.

Quality warning: The system prompt and expected output format are specifically tuned for Claude Opus. Other models may produce:

• More classification errors (e.g., marking an appropriation as a rider)
• Missing provisions (especially sub-allocations and proviso amounts)
• Inconsistent JSON formatting (handled by from_value.rs resilient parsing, but still)
• Lower coverage scores in the audit

Always check audit output after extracting with a non-default model.

The model name is recorded in metadata.json so you always know which model produced a given extraction.

Output Files

After extraction, the bill directory contains:

data/118/hr/9468/
├── BILLS-118hr9468enr.xml     ← Source XML (unchanged)
├── extraction.json            ← All provisions with amounts, accounts, sections
├── verification.json          ← Deterministic checks against source text
├── metadata.json              ← Model name, prompt version, timestamps, source hash
├── tokens.json                ← LLM token usage (input, output, cache hits)
└── chunks/                    ← Per-chunk LLM artifacts (gitignored)
    ├── 01JRWN9T5RR0JTQ6C9FYYE96A8.json
    └── ...

extraction.json

The main output. Contains:

• bill — Identifier, classification, short title, fiscal years, divisions
• provisions — Array of every extracted provision with full structured data
• summary — LLM-generated summary statistics (used for diagnostics, never for computation)
• chunk_map — Links each provision to the chunk it was extracted from
• schema_version — Version of the extraction schema

This is the file all query commands (search, summary, compare, audit) read.

verification.json

Deterministic verification of every provision against the source text. No LLM involved:

• amount_checks — Was each dollar string found in the source?
• raw_text_checks — Is each raw text excerpt a substring of the source?
• completeness — How many dollar strings in the source were captured?
• summary — Roll-up metrics (verified, not_found, ambiguous, match tiers)

metadata.json

Extraction provenance:

• model — Which LLM model was used
• prompt_version — Hash of the system prompt
• extraction_timestamp — When the extraction ran
• source_xml_sha256 — SHA-256 hash of the source XML (for the hash chain)

tokens.json

API token usage:

• total_input — Total input tokens across all chunks
• total_output — Total output tokens
• total_cache_read — Tokens served from prompt cache (reduces cost)
• total_cache_create — Tokens added to prompt cache
• calls — Number of API calls made

chunks/ directory

Per-chunk LLM artifacts stored with ULID filenames. Each file contains:

• The model’s thinking content (internal reasoning)
• The raw JSON response before parsing
• The parsed provisions for that chunk
• A conversion report showing any type coercions or missing fields

These are permanent provenance records — useful for debugging why a particular provision was classified a certain way. They are gitignored by default (not part of the hash chain, not needed for downstream operations).

Verify After Extraction

Always run the audit after extracting:

congress-approp audit --dir data/118/hr/9468

What to check:

For a detailed verification procedure, see Verify Extraction Accuracy.

Re-Extracting a Bill

To re-extract (for example, with a newer model or after prompt improvements), use the --force flag:

# Re-extract even though extraction.json already exists
congress-approp extract --dir data/118/hr/9468 --force

Without --force, the extract command skips bills that already have extraction.json. This makes it safe to re-run extract --dir data after failures — bills that succeeded are skipped, and bills that failed (no extraction.json written) are retried automatically.

After re-extraction:

• extraction.json and verification.json are overwritten
• metadata.json and tokens.json are overwritten
• A new set of chunk artifacts is created in chunks/
• Embeddings become stale — the tool will warn you, and you’ll need to run embed again

Upgrade without re-extracting

If you only need to re-verify against a newer schema (no LLM calls), use upgrade instead:

congress-approp upgrade --dir data/118/hr/9468

This re-deserializes the existing extraction through the current code’s schema, re-runs verification, and updates the files. Much faster and free. See Upgrade Extraction Data.

Handling Large Bills

Omnibus bills (1,000+ pages) require special attention:

Chunk splitting

Large bills are automatically split into chunks at XML <division> and <title> boundaries. This is semantic chunking — each chunk contains a complete legislative section with full context. The FY2024 omnibus (H.R. 4366) splits into approximately 75 chunks.

If a single title or division exceeds the maximum chunk token limit (~3,000 tokens), it’s further split at paragraph boundaries. This is rare but happens for very long sections.

Time estimates

Handling interruptions

If extraction is interrupted (network error, rate limit, crash), you’ll need to re-run it from the beginning. There is no checkpoint/resume mechanism — the tool extracts all chunks and merges them atomically.

Troubleshooting

“N of M chunks failed … Aborting”

This means some LLM API calls failed after all retries — typically due to rate limiting on large bills. The tool did not write extraction.json to prevent saving garbage data.

Fix: Wait a few minutes for API quotas to reset, then re-run the same command. Since no extraction.json was written, the failed bill will be retried automatically. If the bill is very large (90+ chunks), try reducing parallelism:

congress-approp extract --dir data/119/hr/7148 --parallel 3

If you want to save whatever chunks succeeded (accepting an incomplete extraction), add --continue-on-error:

congress-approp extract --dir data/119/hr/7148 --parallel 6 --continue-on-error --force

“All bills already extracted”

This means every bill directory already has extraction.json. Use --force to re-extract:

congress-approp extract --dir data/118/hr/9468 --force

“No XML files found”

Make sure you downloaded the bill first. The extract command looks for files matching BILLS-*.xml in the specified directory.

ls data/118/hr/9468/BILLS-*.xml

“Rate limited” or 429 errors

Reduce parallelism:

congress-approp extract --dir data/118/hr/4366 --parallel 2

Anthropic’s API has per-minute token limits. High concurrency on large bills can exceed these limits.

Low provision count

If a large bill produces surprisingly few provisions, check:

1. The XML file — is it the correct version? Some partial texts are available on Congress.gov.
2. The audit output — low coverage combined with low provision count suggests the extraction missed sections.
3. The chunk artifacts — look in chunks/ for any chunks that produced zero provisions or error responses.

“Unexpected token” or JSON parsing errors

The from_value.rs resilient parser handles most LLM output quirks automatically. If you see parsing warnings in the verbose output, they’re usually minor (a missing field defaulting to empty, a string where a number was expected being coerced). The conversion.json report in each chunk directory shows exactly what was adjusted.

If extraction fails entirely, try with --parallel 1 to isolate which chunk is problematic, then examine that chunk’s artifacts in chunks/.

Quick Reference

# Set API key
export ANTHROPIC_API_KEY="your-key"

# Preview extraction (no API calls)
congress-approp extract --dir data/118/hr/9468 --dry-run

# Extract a single bill
congress-approp extract --dir data/118/hr/9468

# Extract with higher parallelism
congress-approp extract --dir data/118/hr/4366 --parallel 8

# Extract all bills under a directory (skips already-extracted bills)
congress-approp extract --dir data --parallel 6

# Re-extract a bill that was already extracted
congress-approp extract --dir data/118/hr/9468 --force

# Save partial results even when some chunks fail
congress-approp extract --dir data/118/hr/2882 --parallel 6 --continue-on-error

# Verify after extraction
congress-approp audit --dir data/118/hr/9468

Full Command Reference

congress-approp extract [OPTIONS]

Options:
    --dir <DIR>            Data directory containing downloaded bill XML [default: ./data]
    --dry-run              Show what would be extracted without calling LLM
    --parallel <PARALLEL>  Parallel LLM calls [default: 5]
    --model <MODEL>        LLM model override [env: APPROP_MODEL=]
    --force                Re-extract bills even if extraction.json already exists
    --continue-on-error    Save partial results when some chunks fail (default: abort bill)

Next Steps

• Verify Extraction Accuracy — detailed audit and verification guide
• Generate Embeddings — enable semantic search for extracted bills
• Filter and Search Provisions — query your newly extracted data

Generate Embeddings

You will need: congress-approp installed, extracted bill data (with extraction.json), OPENAI_API_KEY environment variable set.

You will learn: How to generate embedding vectors for semantic search and --similar matching, configure embedding options, detect and handle staleness, and manage embedding storage.

Embeddings are what power semantic search (--semantic) and cross-bill matching (--similar). Each provision’s text is converted into a 3,072-dimensional vector that captures its meaning. Provisions about similar topics — even with completely different wording — will have vectors pointing in similar directions.

You only need to generate embeddings once per bill. After that, all semantic operations use the stored vectors locally, with the single exception of --semantic queries which make one small API call to embed your query text.

Prerequisites

1. Extracted bill data. You need extraction.json in each bill directory. See Extract Provisions from a Bill.
2. OpenAI API key. Embeddings use OpenAI’s text-embedding-3-large model.

export OPENAI_API_KEY="your-key-here"

Note: The included example data (data/118-hr4366, data/118-hr5860, data/118-hr9468) ships with pre-generated embeddings. You don’t need to run embed for the examples unless you want to regenerate them.

Generate Embeddings

Single bill directory

congress-approp embed --dir data/118/hr/9468

For a small bill (7 provisions), this takes a few seconds. For the FY2024 omnibus (2,364 provisions), about 30 seconds.

All bills under a directory

congress-approp embed --dir data

The tool walks recursively, finds every directory with an extraction.json, and generates embeddings for each one. Bills that already have up-to-date embeddings are skipped automatically.

Preview without calling the API

congress-approp embed --dir data --dry-run

Shows how many provisions would be embedded and the estimated token count for each bill, without making any API calls.

What Gets Created

The embed command writes two files to each bill directory:

embeddings.json

A small JSON metadata file (~200 bytes, human-readable):

{
  "schema_version": "1.0",
  "model": "text-embedding-3-large",
  "dimensions": 3072,
  "count": 7,
  "extraction_sha256": "a1b2c3d4e5f6...",
  "vectors_file": "vectors.bin",
  "vectors_sha256": "f6e5d4c3b2a1..."
}

vectors.bin

A binary file containing raw little-endian float32 vectors. The file size is exactly count × dimensions × 4 bytes:

There is no header in the file — the count and dimensions come from embeddings.json. Vectors are stored in provision order (provision 0 first, then provision 1, etc.).

Embedding Options

Model

The default model is text-embedding-3-large, which provides the best quality embeddings available from OpenAI. You can override this:

congress-approp embed --dir data --model text-embedding-3-small

Warning: All embeddings in a dataset must use the same model. You cannot compare vectors from different models. If you change models, regenerate embeddings for all bills.

Dimensions

By default, the tool requests the full 3,072 dimensions from text-embedding-3-large. You can request fewer dimensions for smaller storage at the cost of some quality:

congress-approp embed --dir data --dimensions 1024

Experimental results from this project’s testing:

Since binary vector files load in under 2ms regardless of size, there is little practical reason to truncate dimensions.

Warning: Like models, all embeddings in a dataset must use the same dimension count. Cosine similarity between vectors of different dimensions is undefined.

Batch size

Provisions are sent to the API in batches. The default batch size is 100 provisions per API call:

congress-approp embed --dir data --batch-size 50

Smaller batch sizes make more API calls but reduce the impact of a single failed call. The default of 100 is efficient for most use cases.

How Provision Text Is Built

Each provision is embedded using a deterministic text representation built by build_embedding_text(). The text concatenates the provision’s meaningful fields:

Account: Child Nutrition Programs | Agency: Department of Agriculture | Text: For necessary expenses of the Food and Nutrition Service...

The exact fields included depend on the provision type:

• Appropriations/Rescissions: Account name, agency, program, raw text
• CR Substitutions: Account name, reference act, reference section, raw text
• Directives/Riders: Description, raw text
• Other types: Description or LLM classification, raw text

This deterministic construction means the same provision always produces the same embedding text, regardless of when or where you run the command.

Staleness Detection

The hash chain connects embeddings to their source extraction:

extraction.json ──sha256──▶ embeddings.json (extraction_sha256)
vectors.bin ──sha256──▶ embeddings.json (vectors_sha256)

If you re-extract a bill (producing a new extraction.json), the embeddings become stale. Commands that use embeddings will warn you:

⚠ H.R. 4366: embeddings are stale (extraction.json has changed)

This warning is advisory — the tool still works, but similarity results may not match the current provisions. To fix it, regenerate embeddings:

congress-approp embed --dir data/118/hr/4366

The embed command automatically detects stale embeddings and regenerates them. Up-to-date embeddings are skipped.

Skipping Up-to-Date Bills

When you run embed on a directory with multiple bills, the tool checks each one:

1. Does embeddings.json exist?
2. Does extraction_sha256 in embeddings.json match the current SHA-256 of extraction.json?
3. Does vectors_sha256 in embeddings.json match the current SHA-256 of vectors.bin?

If all three checks pass, the bill is skipped with a message like:

Skipping H.R. 9468: embeddings up to date

This makes it safe to run embed --dir data repeatedly — it only does work where needed.

Cost Estimates

Embedding generation is inexpensive compared to extraction:

The text-embedding-3-large model charges per token. Even the largest omnibus bill with 2,364 provisions uses only a few tens of thousands of tokens total, which costs pennies.

Use --dry-run to preview the exact token count before committing.

Reading Vectors in Python

If you want to work with the embeddings outside of congress-approp:

import json
import struct

# Load metadata
with open("data/118-hr9468/embeddings.json") as f:
    meta = json.load(f)

dims = meta["dimensions"]  # 3072
count = meta["count"]       # 7

# Load vectors
with open("data/118-hr9468/vectors.bin", "rb") as f:
    raw = f.read()

# Parse into list of vectors
vectors = []
for i in range(count):
    start = i * dims * 4
    end = start + dims * 4
    vec = struct.unpack(f"<{dims}f", raw[start:end])
    vectors.append(vec)

# Vectors are L2-normalized (norm ≈ 1.0), so cosine similarity = dot product
def cosine(a, b):
    return sum(x * y for x, y in zip(a, b))

# Compare provision 0 to provision 1
print(f"Similarity: {cosine(vectors[0], vectors[1]):.4f}")

You can also load the vectors into numpy for faster computation:

import numpy as np

vectors = np.frombuffer(raw, dtype=np.float32).reshape(count, dims)

# Cosine similarity matrix
similarity_matrix = vectors @ vectors.T

After Generating Embeddings

Once embeddings are generated, you can use:

• Semantic search: congress-approp search --dir data --semantic "your query" --top 10
• Similar provisions: congress-approp search --dir data --similar 118-hr9468:0 --top 5

The --similar flag does not make any API calls — it uses the stored vectors directly. The --semantic flag makes one API call to embed your query text (~100ms).

Troubleshooting

“OPENAI_API_KEY environment variable not set”

Set your API key:

export OPENAI_API_KEY="your-key-here"

“No extraction.json found”

You need to extract the bill before generating embeddings. Run congress-approp extract first.

Embeddings stale warning after re-extraction

This is expected. Run congress-approp embed --dir <path> to regenerate.

Very large vectors.bin file

The omnibus bill produces a ~29 MB vectors.bin file. This is expected for 2,364 provisions × 3,072 dimensions × 4 bytes per float. The file loads in under 2ms despite its size.

These files are excluded from the crates.io package (via Cargo.toml exclude field) because they exceed the 10 MB upload limit. They are included in the git repository for users who clone.

Quick Reference

# Set API key
export OPENAI_API_KEY="your-key"

# Generate embeddings for one bill
congress-approp embed --dir data/118/hr/9468

# Generate embeddings for all bills
congress-approp embed --dir data

# Preview without API calls
congress-approp embed --dir data --dry-run

# Use a different model
congress-approp embed --dir data --model text-embedding-3-small

# Use fewer dimensions
congress-approp embed --dir data --dimensions 1024

# Smaller batch size
congress-approp embed --dir data --batch-size 50

Full Command Reference

congress-approp embed [OPTIONS]

Options:
    --dir <DIR>                Data directory [default: ./data]
    --model <MODEL>            Embedding model [default: text-embedding-3-large]
    --dimensions <DIMENSIONS>  Request this many dimensions from the API [default: 3072]
    --batch-size <BATCH_SIZE>  Provisions per API batch [default: 100]
    --dry-run                  Preview without calling API

Next Steps

• Use Semantic Search — put your new embeddings to work
• Track a Program Across Bills — cross-bill matching with --similar
• Data Integrity and the Hash Chain — how staleness detection works

Verify Extraction Accuracy

You will need: congress-approp installed, access to extracted bill data (the data/ directory works).

You will learn: How to run a full verification audit, interpret every metric, trace individual provisions back to source XML, and decide whether extraction quality is sufficient for your use case.

Extraction uses an LLM to classify and structure provisions from bill text. Verification uses deterministic code — no LLM involved — to check every claim the extraction made against the source. This guide walks you through the complete verification workflow.

Step 1: Run the Audit

The audit command is your primary verification tool:

congress-approp audit --dir data

┌───────────┬────────────┬──────────┬──────────┬───────┬───────┬──────────┬───────────┬──────────┬──────────┐
│ Bill      ┆ Provisions ┆ Verified ┆ NotFound ┆ Ambig ┆ Exact ┆ NormText ┆ Spaceless ┆ TextMiss ┆ Coverage │
╞═══════════╪════════════╪══════════╪══════════╪═══════╪═══════╪══════════╪═══════════╪══════════╪══════════╡
│ H.R. 4366 ┆       2364 ┆      762 ┆        0 ┆   723 ┆  2285 ┆       59 ┆         0 ┆       20 ┆    94.2% │
│ H.R. 5860 ┆        130 ┆       33 ┆        0 ┆     2 ┆   102 ┆       12 ┆         0 ┆       16 ┆    61.1% │
│ H.R. 9468 ┆          7 ┆        2 ┆        0 ┆     0 ┆     5 ┆        0 ┆         0 ┆        2 ┆   100.0% │
│ TOTAL     ┆       2501 ┆      797 ┆        0 ┆   725 ┆  2392 ┆       71 ┆         0 ┆       38 ┆          │
└───────────┴────────────┴──────────┴──────────┴───────┴───────┴──────────┴───────────┴──────────┴──────────┘

Column Guide:
  Verified   Dollar amount string found at exactly one position in source text
  NotFound   Dollar amounts NOT found in source — not present in source, review manually
  Ambig      Dollar amounts found multiple times in source — correct but position uncertain
  Exact      raw_text is byte-identical substring of source — verbatim copy
  NormText   raw_text matches after whitespace/quote/dash normalization — content correct
  Spaceless  raw_text matches only after removing all spaces — PDF artifact, review
  TextMiss   raw_text not found at any tier — may be paraphrased, review manually
  Coverage   Percentage of dollar strings in source text matched to a provision

Key:
  NotFound = 0 and Coverage = 100%   →  All amounts captured and found in source
  NotFound = 0 and Coverage < 100%   →  Extracted amounts correct, but bill has more
  NotFound > 0                       →  Some amounts need manual review

This is a lot of information. Let’s break it down column by column.

Step 2: Check for Unverifiable Amounts (The Critical Metric)

The single most important number in the audit is the NotFound column. It counts provisions where the extracted dollar string (e.g., "$2,285,513,000") was not found anywhere in the source bill text.

Across the included example data: NotFound = 0 for every bill. 99.995% of extracted dollar amounts were confirmed to exist in the source text. See Accuracy Metrics for the full breakdown.

Verified vs. Ambiguous

The remaining provisions with dollar amounts fall into two categories:

• Verified: The dollar string was found at exactly one position in the source. This provides the strongest attribution — you know exactly where in the bill this amount comes from.
• Ambiguous (Ambig): The dollar string was found at multiple positions. The amount is correct — it’s definitely in the bill — but it appears more than once, so you can’t automatically pin it to a single location.

Ambiguous matches are common and expected. Round numbers like $5,000,000 can appear 50+ times in a large omnibus bill. In H.R. 4366, 723 of 1,485 provisions with dollar amounts are ambiguous — mostly because common round-number amounts recur throughout the bill’s 2,364 provisions.

Ambiguous does not mean inaccurate. The amount is verified to exist in the source; only the precise location is uncertain.

Provisions without dollar amounts

Not all provisions have dollar amounts. Riders, directives, and some policy provisions carry no dollars. These provisions don’t appear in the Verified/NotFound/Ambig counts. In the example data:

• H.R. 4366: 2,364 provisions, 1,485 with dollar amounts (762 verified + 723 ambiguous), 879 without
• H.R. 5860: 130 provisions, 35 with dollar amounts (33 verified + 2 ambiguous), 95 without
• H.R. 9468: 7 provisions, 2 with dollar amounts (2 verified + 0 ambiguous), 5 without

Step 3: Examine Raw Text Matching

The right side of the audit table checks whether each provision’s raw_text excerpt (the first ~150 characters of the bill language) is a substring of the source text. This is checked in four tiers:

Tier 1: Exact (best)

The raw_text is a byte-identical substring of the source bill text. This means the LLM copied the text perfectly — not a single character was changed.

In the example data: approximately 95.5% of provisions match at the Exact tier across the 13-bill dataset. This is excellent and provides strong evidence that the provision is attributed to the correct location in the bill.

Tier 2: Normalized

The raw_text matches after normalizing whitespace, curly quotes (" → "), and em-dashes (— → -). These differences arise from the XML-to-text conversion process — the source XML uses Unicode characters that the LLM may render differently.

In the example data: 71 provisions (2.8%) match at the Normalized tier. The content is correct; only formatting details differ.

Tier 3: Spaceless

The raw_text matches only after removing all spaces. This catches cases where word boundaries differ — for example, (1)not less than vs. (1) not less than. This is typically caused by XML tags being stripped without inserting spaces.

In the example data: 0 provisions match at the Spaceless tier.

Tier 4: No Match (TextMiss)

The raw_text was not found at any tier. Possible causes:

• Truncation: The LLM truncated a very long provision and the truncated text doesn’t appear as-is in the source.
• Paraphrasing: The LLM rephrased the statutory language (especially common for complex amendments like “Section X is amended by striking Y and inserting Z”).
• Concatenation: The LLM combined text from adjacent sections into one raw_text string.

In the example data: 38 provisions (1.5%) are TextMiss. Examining them reveals they are all non-dollar provisions — statutory amendments (riders and mandatory spending extensions) where the LLM slightly reformatted section references. No provision with a dollar amount has a TextMiss in the example data.

What TextMiss does and doesn’t mean

TextMiss does NOT mean the provision is fabricated. The provision’s other fields (account_name, description, dollar amounts) may still be correct — it’s only the raw_text excerpt that doesn’t match. Dollar amounts are verified independently through the amount checks.

TextMiss DOES mean you should review manually if the provision is important to your analysis. Use audit --verbose to see which provisions are affected.

Step 4: Use Verbose Mode for Details

When any metric raises a concern, use --verbose to see specific problematic provisions:

congress-approp audit --dir data --verbose

This adds a list of individual provisions that didn’t pass verification at the highest tier. For each one, you’ll see:

• The provision index
• The provision type and account name (if applicable)
• The dollar string (if applicable) and whether it was found
• The raw text preview and which match tier it achieved

This gives you enough information to manually check any provision against the source XML.

Step 5: Trace a Specific Provision to Source

For any provision you want to verify yourself — perhaps one you plan to cite in a report or story — here’s how to trace it back to the source:

1. Get the provision details

congress-approp search --dir data/118-hr9468 --type appropriation --format json

Look for the provision you’re interested in. Note the dollars, raw_text, and provision_index fields.

For example, provision 0 of H.R. 9468:

{
  "dollars": 2285513000,
  "raw_text": "For an additional amount for ''Compensation and Pensions'', $2,285,513,000, to remain available until expended.",
  "provision_index": 0,
  "amount_status": "found",
  "match_tier": "exact"
}

2. Verify the dollar string in the source XML

Search for the text_as_written dollar string in the source file:

grep '$2,285,513,000' data/118-hr9468/BILLS-118hr9468enr.xml

If it’s found (and amount_status is “found”), the amount is verified. If found exactly once, the attribution is unambiguous.

3. Read the surrounding context

To see what the bill actually says around that dollar amount:

grep -B2 -A5 '2,285,513,000' data/118-hr9468/BILLS-118hr9468enr.xml

Or in Python for cleaner output:

import re

with open("data/118-hr9468/BILLS-118hr9468enr.xml") as f:
    text = f.read()

idx = text.find("2,285,513,000")
if idx >= 0:
    # Get surrounding context, strip XML tags
    start = max(0, idx - 200)
    end = min(len(text), idx + 200)
    context = re.sub(r'<[^>]+>', ' ', text[start:end])
    context = re.sub(r'\s+', ' ', context).strip()
    print(f"Context: ...{context}...")

4. Compare to the extracted data

Does the context match what the provision claims? Is the account name correct? Is the amount attributed to the right program? The structured raw_text field should be recognizable in the source context.

For the VA Supplemental example, the source text reads:

For an additional amount for ’‘Compensation and Pensions’’, $2,285,513,000, to remain available until expended.

And the extracted raw_text is identical — byte-for-byte.

Step 6: Interpret Coverage

The Coverage column shows the percentage of dollar-sign patterns in the source bill text that were matched to an extracted provision. This measures extraction completeness, not accuracy.

100% coverage (H.R. 9468)

Every dollar amount in the source was captured by a provision. This is ideal and common for small, simple bills.

94.2% coverage (H.R. 4366)

Most dollar amounts were captured, but 5.8% were not. For a 1,500-page omnibus, this is expected. The unmatched dollar strings are typically:

• Statutory cross-references: Dollar amounts from other laws cited in the bill text (e.g., “as authorized under section 1241(a)” where the referenced section contains a dollar amount)
• Loan guarantee ceilings: “$3,500,000,000 for guaranteed farm ownership loans” — these are loan volume limits, not budget authority
• Struck amounts: “Striking ‘$50,000’ and inserting ‘$75,000’” — the old amount being struck shouldn’t be an independent provision
• Proviso sub-references: Amounts in conditions that don’t constitute independent provisions

61.1% coverage (H.R. 5860)

Continuing resolutions have inherently lower coverage because most of the bill text consists of references to prior-year appropriations acts. Those referenced acts contain many dollar amounts that appear in the CR’s text but aren’t new provisions — they’re contextual citations. Only the 13 CR substitutions and a few standalone appropriations are genuine new provisions in this bill.

When low coverage IS concerning

Coverage below 60% on a regular appropriations bill (not a CR) may indicate that the extraction missed entire sections. Investigate by:

1. Running audit --verbose to see which dollar amounts are unaccounted for
2. Checking whether major accounts you expect are present in search --type appropriation
3. Comparing the provision count to what you’d expect for a bill of that size

See What Coverage Means (and Doesn’t) for a detailed explanation.

Step 7: Cross-Check with External Sources

For high-stakes analysis, cross-check the tool’s totals against independent sources:

CBO cost estimates

The Congressional Budget Office publishes cost estimates for most appropriations bills. These aggregate numbers can serve as a sanity check for the tool’s budget authority totals. Note that CBO estimates may use slightly different accounting conventions (e.g., including or excluding advance appropriations differently).

Committee reports

The House and Senate Appropriations Committees publish detailed reports accompanying each bill. These contain account-level funding tables that can be compared to the tool’s per-account breakdowns.

Known sources of discrepancy

Even with perfect extraction, the tool’s totals may differ from external sources because:

• Mandatory spending lines (SNAP, VA Comp & Pensions) appear as appropriation provisions in the bill text but are not “discretionary” in the budget sense
• Advance appropriations are enacted in the current bill but available in a future fiscal year
• Sub-allocations use reference_amount semantics and are excluded from budget authority totals, while some external sources include them
• Transfer authorities have dollar ceilings that are not new spending

See Why the Numbers Might Not Match Headlines for a comprehensive explanation.

Step 8: Decide Whether to Re-Extract

Based on your audit results, here’s a decision framework:

Re-extraction vs. upgrade

• Re-extract (congress-approp extract --dir <path>): Makes new LLM API calls. Use when you want a fresh extraction, possibly with a different model or after prompt improvements.
• Upgrade (congress-approp upgrade --dir <path>): No LLM calls. Re-deserializes existing data through the current schema and re-runs verification. Use when the schema or verification logic has been updated but the extraction itself is fine.

Automated Verification in Scripts

For CI/CD or automated pipelines, you can check verification programmatically:

# Check that no dollar amounts are unverifiable across all bills
congress-approp summary --dir data --format json | python3 -c "
import sys, json
bills = json.load(sys.stdin)
# The summary footer reports unverified count
# Check budget authority totals as a regression guard
expected = {'H.R. 4366': 846137099554, 'H.R. 5860': 16000000000, 'H.R. 9468': 2882482000}
for b in bills:
    assert b['budget_authority'] == expected[b['identifier']], \
        f\"{b['identifier']} budget authority mismatch: {b['budget_authority']} != {expected[b['identifier']]}\"
print('All budget authority totals match expected values')
"

This is the same check used in the project’s integration test suite to guard against data regressions.

Quick Decision Table

Next Steps

• What Coverage Means (and Doesn’t) — detailed explanation of the coverage metric
• How Verification Works — the technical design of the verification pipeline
• LLM Reliability and Guardrails — understanding the trust model and known failure modes

Filter and Search Provisions

You will need: congress-approp installed, access to the data/ directory. For semantic search: OPENAI_API_KEY.

You will learn: Every filter flag available on the search command, how to combine them, and practical recipes for common queries.

The search command is the most versatile tool in congress-approp. It supports ten filter flags that can be combined freely — all filters use AND logic, meaning every provision in the results must match every filter you specify. This guide covers each flag with real examples from the included data.

Quick Reference: All Search Flags

Filter by Provision Type (--type)

The most common filter. Restricts results to a single provision type.

# All appropriations across all bills
congress-approp search --dir data --type appropriation

# All rescissions
congress-approp search --dir data --type rescission

# CR substitutions (anomalies) — table auto-adapts to show New/Old/Delta columns
congress-approp search --dir data --type cr_substitution

# Reporting requirements and instructions to agencies
congress-approp search --dir data --type directive

# Policy provisions (no direct spending)
congress-approp search --dir data --type rider

Available provision types

Use --list-types to see all valid values:

congress-approp search --dir data --list-types

Available provision types:
  appropriation                    Budget authority grant
  rescission                       Cancellation of prior budget authority
  cr_substitution                  CR anomaly (substituting $X for $Y)
  transfer_authority               Permission to move funds between accounts
  limitation                       Cap or prohibition on spending
  directed_spending                Earmark / community project funding
  mandatory_spending_extension     Amendment to authorizing statute
  directive                        Reporting requirement or instruction
  rider                            Policy provision (no direct spending)
  continuing_resolution_baseline   Core CR funding mechanism
  other                            Unclassified provisions

Type distribution by bill

Not every bill contains every type. Here’s the distribution across the example data:

Filter by Agency (--agency)

Matches the agency field using a case-insensitive substring search:

# All provisions from the Department of Veterans Affairs
congress-approp search --dir data --agency "Veterans"

# All provisions from the Department of Energy
congress-approp search --dir data --agency "Energy"

# All NASA provisions
congress-approp search --dir data --agency "Aeronautics"

# All DOJ provisions
congress-approp search --dir data --agency "Justice"

The --agency flag matches against the structured agency field that the LLM extracted — typically the full department name (e.g., “Department of Veterans Affairs”). You only need to provide a substring; the match is case-insensitive.

Tip: Some provisions don’t have an agency field (riders, directives, and some other types). These will never appear in agency-filtered results.

Combine with type for focused results

# Only VA appropriations
congress-approp search --dir data --agency "Veterans" --type appropriation

# Only VA rescissions
congress-approp search --dir data --agency "Veterans" --type rescission

# DOJ directives
congress-approp search --dir data --agency "Justice" --type directive

Filter by Account Name (--account)

Matches the account_name field using a case-insensitive substring search. This is more specific than --agency — it targets the individual appropriations account:

# All provisions for Child Nutrition Programs
congress-approp search --dir data --account "Child Nutrition"

# All provisions for the FBI
congress-approp search --dir data --account "Federal Bureau of Investigation"

# All provisions for Disaster Relief
congress-approp search --dir data --account "Disaster Relief"

# All provisions for Medical Services (VA)
congress-approp search --dir data --account "Medical Services"

The account name is extracted from the bill text — it’s usually the text between '' delimiters in the legislative language (e.g., ''Compensation and Pensions'').

Account vs. Agency

Many provisions under the same agency have different account names. Use --agency for a department-wide view and --account when you know the specific program.

Gotcha: “Salaries and Expenses”

The account name “Salaries and Expenses” appears under dozens of different agencies. If you search --account "Salaries and Expenses" without an agency filter, you’ll get results from across the entire government. Combine with --agency to narrow:

congress-approp search --dir data --account "Salaries and Expenses" --agency "Justice"

Filter by Keyword in Bill Text (--keyword)

Searches the raw_text field — the actual bill language excerpt stored with each provision. This is a case-insensitive substring match:

# Find provisions mentioning FEMA
congress-approp search --dir data --keyword "Federal Emergency Management"

# Find provisions with "notwithstanding" (often signals important policy exceptions)
congress-approp search --dir data --keyword "notwithstanding"

# Find provisions about transfer authority
congress-approp search --dir data --keyword "may transfer"

# Find provisions about reporting requirements
congress-approp search --dir data --keyword "shall submit a report"

# Find provisions referencing a specific public law
congress-approp search --dir data --keyword "Public Law 118"

Keyword vs. Account vs. Semantic

For the most thorough search, try all three approaches. Start with --keyword or --account for precision, then use --semantic to find provisions you might have missed.

Filter by Bill (--bill)

Restricts results to a specific bill by its identifier string:

# Only provisions from H.R. 4366
congress-approp search --dir data --bill "H.R. 4366"

# Only provisions from H.R. 9468
congress-approp search --dir data --bill "H.R. 9468"

The value must match the bill identifier as it appears in the data (e.g., “H.R. 4366”, including the space and period). This is a case-sensitive exact match.

Alternative: Point --dir at a specific bill directory. Instead of --bill, you can scope the search by directory:

# These are equivalent for single-bill searches:
congress-approp search --dir data --bill "H.R. 4366"
congress-approp search --dir data/118-hr4366

The --dir approach is simpler for single-bill searches. The --bill flag is useful when you have multiple bills loaded via a parent directory and want to filter to one.

Filter by Division (--division)

Omnibus bills are organized into lettered divisions (Division A, Division B, etc.), each covering a different set of agencies. The --division flag scopes results to a single division:

# Division A = MilCon-VA in H.R. 4366
congress-approp search --dir data/118-hr4366 --division A

# Division B = Agriculture in H.R. 4366
congress-approp search --dir data/118-hr4366 --division B

# Division C = Commerce, Justice, Science in H.R. 4366
congress-approp search --dir data/118-hr4366 --division C

# Division D = Energy and Water in H.R. 4366
congress-approp search --dir data/118-hr4366 --division D

The division letter is a single character (A, B, C, etc.). Bills without divisions (like the VA supplemental H.R. 9468) have no division field, so --division effectively returns no results for those bills.

Combine with type for division-level analysis

# All appropriations in MilCon-VA (Division A) over $1 billion
congress-approp search --dir data/118-hr4366 --division A --type appropriation --min-dollars 1000000000

# All rescissions in Commerce-Justice-Science (Division C)
congress-approp search --dir data/118-hr4366 --division C --type rescission

# All riders in Agriculture (Division B)
congress-approp search --dir data/118-hr4366 --division B --type rider

Filter by Dollar Range (--min-dollars, --max-dollars)

Filters provisions by the absolute value of their dollar amount:

# Provisions of $1 billion or more
congress-approp search --dir data --min-dollars 1000000000

# Provisions between $100 million and $500 million
congress-approp search --dir data --min-dollars 100000000 --max-dollars 500000000

# Small provisions under $1 million
congress-approp search --dir data --max-dollars 1000000

# Large rescissions
congress-approp search --dir data --type rescission --min-dollars 1000000000

The filter uses the absolute value of the dollar amount, so rescissions (which may be stored as negative values internally) are compared by their magnitude.

Provisions without dollar amounts (riders, directives, etc.) are excluded from results when --min-dollars or --max-dollars is specified.

Combining Multiple Filters

All filters use AND logic — every filter must match for a provision to appear. This lets you build very specific queries:

# VA appropriations over $1 billion in Division A
congress-approp search --dir data \
  --agency "Veterans" \
  --type appropriation \
  --division A \
  --min-dollars 1000000000

# DOJ rescissions in Division C
congress-approp search --dir data \
  --agency "Justice" \
  --type rescission \
  --division C

# Provisions mentioning "notwithstanding" in the omnibus under $10 million
congress-approp search --dir data/118-hr4366 \
  --keyword "notwithstanding" \
  --max-dollars 10000000

# Energy-related appropriations in Division D between $100M and $1B
congress-approp search --dir data/118-hr4366 \
  --division D \
  --type appropriation \
  --min-dollars 100000000 \
  --max-dollars 1000000000

Filter order doesn’t matter

The tool applies filters in the order that’s most efficient internally. The command-line order of flags has no effect on results — these two commands produce identical output:

congress-approp search --dir data --type appropriation --agency "Veterans"
congress-approp search --dir data --agency "Veterans" --type appropriation

Semantic Search (--semantic)

Semantic search ranks provisions by meaning similarity instead of keyword matching. It requires pre-computed embeddings and an OPENAI_API_KEY:

export OPENAI_API_KEY="your-key"

# Find provisions about school lunch programs (no keyword overlap with "Child Nutrition Programs")
congress-approp search --dir data --semantic "school lunch programs for kids" --top 5

# Find provisions about road and bridge infrastructure
congress-approp search --dir data --semantic "money for fixing roads and bridges" --top 5

Combining semantic search with hard filters

Hard filters apply first (constraining which provisions are eligible), then semantic ranking orders the remaining results:

# Appropriations about clean energy, at least $100M
congress-approp search --dir data \
  --semantic "clean energy research" \
  --type appropriation \
  --min-dollars 100000000 \
  --top 10

For a full tutorial on semantic search, see Use Semantic Search.

Find Similar Provisions (--similar)

Find provisions most similar to a specific one across all loaded bills. The syntax is --similar <bill_directory>:<provision_index>:

# Find provisions similar to VA Supplemental provision 0 (Comp & Pensions)
congress-approp search --dir data --similar 118-hr9468:0 --top 5

# Find provisions similar to omnibus provision 620 (FBI Salaries and Expenses)
congress-approp search --dir data --similar hr4366:620 --top 5

Unlike --semantic, the --similar flag does not make any API calls — it uses pre-computed vectors directly. This makes it instant and free.

You can also combine --similar with hard filters:

# Find appropriations similar to a specific provision
congress-approp search --dir data --similar 118-hr9468:0 --type appropriation --top 5

For a full tutorial, see Track a Program Across Bills.

Controlling the Number of Results (--top)

The --top flag limits results for semantic and similar searches (default 20). It has no effect on non-semantic searches (which return all matching provisions):

# Top 3 results
congress-approp search --dir data --semantic "veterans health care" --top 3

# Top 50 results
congress-approp search --dir data --semantic "veterans health care" --top 50

Output Formats (--format)

All search results can be output in four formats:

# Human-readable table (default)
congress-approp search --dir data --type appropriation --format table

# JSON array (full fields, for programmatic use)
congress-approp search --dir data --type appropriation --format json

# JSON Lines (one object per line, for streaming)
congress-approp search --dir data --type appropriation --format jsonl

# CSV (for spreadsheets)
congress-approp search --dir data --type appropriation --format csv > provisions.csv

JSON and CSV include more fields than the table view — notably raw_text, semantics, detail_level, amount_status, match_tier, quality, and provision_index.

For detailed format documentation and recipes, see Export Data for Spreadsheets and Scripts and Output Formats.

Practical Recipes

Here are battle-tested queries for common analysis tasks:

Find the biggest appropriations in a bill

congress-approp search --dir data/118-hr4366 --type appropriation --min-dollars 10000000000 --format table

Find all provisions for a specific agency

congress-approp search --dir data --agency "Department of Energy" --format table

Export all rescissions to a spreadsheet

congress-approp search --dir data --type rescission --format csv > rescissions.csv

Find reporting requirements for the VA

congress-approp search --dir data --keyword "Veterans Affairs" --type directive

Find all provisions that override other law

congress-approp search --dir data --keyword "notwithstanding"

Find which mandatory programs were extended in the CR

congress-approp search --dir data/118-hr5860 --type mandatory_spending_extension --format json

Find provisions in a specific dollar range

# "Small" appropriations: $1M to $10M
congress-approp search --dir data --type appropriation --min-dollars 1000000 --max-dollars 10000000

# "Large" appropriations: over $10B
congress-approp search --dir data --type appropriation --min-dollars 10000000000

Count provisions by type across all bills

congress-approp search --dir data --format json | \
  jq 'group_by(.provision_type) | map({type: .[0].provision_type, count: length}) | sort_by(-.count)'

Export everything and filter later

If you’re not sure what you need yet, export all provisions and filter in your analysis tool:

# All provisions, all fields, all bills
congress-approp search --dir data --format json > all_provisions.json

# Or as CSV for Excel
congress-approp search --dir data --format csv > all_provisions.csv

Tips

1. Start broad, then narrow. Begin with --type or --agency alone, see how many results you get, then add more filters to focus.

2. Use --format json to see all fields. The table view truncates long text and hides some fields. JSON shows everything.

3. Use --dir scoping for single-bill searches. Instead of --bill "H.R. 4366", use --dir data/118-hr4366 — it’s simpler and slightly faster.

4. Combine keyword and account searches. An account name search finds provisions named after a program. A keyword search finds provisions that mention a program in their text. Use both for completeness.

5. Try semantic search as a second pass. After keyword/account search gives you the obvious results, run a semantic search on the same topic to find provisions you might have missed because the bill uses different terminology.

6. Check --list-types when unsure. If you can’t remember the exact type name, --list-types shows all valid values with descriptions.

Next Steps

• Find How Much Congress Spent on a Topic — tutorial combining multiple search techniques
• Use Semantic Search — deep dive into meaning-based search
• Output Formats — detailed format reference
• CLI Command Reference — complete reference for all commands

Work with CR Substitutions

You will need: congress-approp installed, access to the data/ directory.

You will learn: What CR substitutions are in legislative context, how to find and interpret them, how to match them to their omnibus counterparts, and how to export them for analysis.

Continuing resolutions (CRs) fund the government at prior-year rates — but not uniformly. Specific programs get different treatment through anomalies, formally known as CR substitutions. These are provisions that say “substitute $X for $Y,” replacing one dollar amount with another. They’re politically significant because they reveal which programs Congress chose to fund above or below the default rate.

The tool extracts CR substitutions as structured data with both the new and old amounts, making them easy to find, compare, and analyze.

What a CR Substitution Looks Like

In bill text, a CR substitution looks like this:

…shall be applied by substituting “$25,300,000” for “$75,300,000”…

This means: instead of continuing the Rural Community Facilities Program at its prior-year level of $75.3 million, fund it at $25.3 million — a $50 million cut.

The tool captures both sides:

{
  "provision_type": "cr_substitution",
  "account_name": "Rural Housing Service—Rural Community Facilities Program Account",
  "new_amount": {
    "value": { "kind": "specific", "dollars": 25300000 },
    "semantics": "new_budget_authority",
    "text_as_written": "$25,300,000"
  },
  "old_amount": {
    "value": { "kind": "specific", "dollars": 75300000 },
    "semantics": "new_budget_authority",
    "text_as_written": "$75,300,000"
  },
  "raw_text": "except section 521(a)(2) shall be applied by substituting ''$25,300,000'' for ''$75,300,000''",
  "section": "SEC. 101",
  "division": "A"
}

Both dollar amounts — the new and the old — are independently verified against the source bill text.

Find All CR Substitutions

The --type cr_substitution filter finds every anomaly in a continuing resolution:

congress-approp search --dir data/118-hr5860 --type cr_substitution

┌───┬───────────┬──────────────────────────────────────────┬───────────────┬───────────────┬──────────────┬──────────┬─────┐
│ $ ┆ Bill      ┆ Account                                  ┆       New ($) ┆       Old ($) ┆    Delta ($) ┆ Section  ┆ Div │
╞═══╪═══════════╪══════════════════════════════════════════╪═══════════════╪═══════════════╪══════════════╪══════════╪═════╡
│ ✓ ┆ H.R. 5860 ┆ Rural Housing Service—Rural Community…   ┆    25,300,000 ┆    75,300,000 ┆  -50,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Rural Utilities Service—Rural Water a…   ┆    60,000,000 ┆   325,000,000 ┆ -265,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆                                          ┆   122,572,000 ┆   705,768,000 ┆ -583,196,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ National Science Foundation—STEM Educ…   ┆    92,000,000 ┆   217,000,000 ┆ -125,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ National Oceanic and Atmospheric Admini… ┆    42,000,000 ┆    62,000,000 ┆  -20,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ National Science Foundation—Research …   ┆   608,162,000 ┆   818,162,000 ┆ -210,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Department of State—Administration of…   ┆    87,054,000 ┆   147,054,000 ┆  -60,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Bilateral Economic Assistance—Funds A…   ┆   637,902,000 ┆   937,902,000 ┆ -300,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Bilateral Economic Assistance—Departm…   ┆   915,048,000 ┆ 1,535,048,000 ┆ -620,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ International Security Assistance—Dep…   ┆    74,996,000 ┆   374,996,000 ┆ -300,000,000 ┆ SEC. 101 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Office of Personnel Management—Salari…   ┆   219,076,000 ┆   190,784,000 ┆  +28,292,000 ┆ SEC. 126 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Department of Transportation—Federal …   ┆   617,000,000 ┆   570,000,000 ┆  +47,000,000 ┆ SEC. 137 ┆ A   │
│ ✓ ┆ H.R. 5860 ┆ Department of Transportation—Federal …   ┆ 2,174,200,000 ┆ 2,221,200,000 ┆  -47,000,000 ┆ SEC. 137 ┆ A   │
└───┴───────────┴──────────────────────────────────────────┴───────────────┴───────────────┴──────────────┴──────────┴─────┘
13 provisions found

$ = Amount status: ✓ found (unique), ≈ found (multiple matches), ✗ not found

Notice how the table automatically changes shape when you search for CR substitutions — instead of a single Amount column, you get three:

Every dollar amount has ✓ verification — both the new and old amounts were found in the source bill text. All 13 CR substitutions in H.R. 5860 are fully verified.

Interpret the Results

Which programs were cut?

Eleven of the thirteen CR substitutions are negative deltas — Congress funded these programs below the prior-year level during the temporary spending period. The largest cuts:

Which programs got more?

Only two programs received increases:

Missing account names

The third row in the table has no account name — just $122,572,000 / $705,768,000. This happens when the CR language references a section of law rather than naming an account directly:

except section 521(d)(1) shall be applied by substituting ''$122,572,000'' for ''$705,768,000''

Section 521(d)(1) refers to the rental assistance voucher program under the Housing Act of 1949. The tool captures the amounts and the raw text but can’t always infer the account name when the bill text uses a statutory reference instead.

You can see the full details in JSON:

congress-approp search --dir data/118-hr5860 --type cr_substitution --format json

The raw_text field will show the full excerpt for each provision, including the statutory reference.

Export CR Substitutions

CSV for spreadsheets

congress-approp search --dir data/118-hr5860 --type cr_substitution --format csv > cr_anomalies.csv

The CSV includes the dollars column (new amount), old_dollars column (old amount), and all other fields. You can compute the delta in Excel as =A2-B2 or use the dollars and old_dollars columns directly.

JSON for scripts

congress-approp search --dir data/118-hr5860 --type cr_substitution --format json > cr_anomalies.json

JSON output includes every field:

{
  "account_name": "Rural Housing Service—Rural Community Facilities Program Account",
  "amount_status": "found",
  "bill": "H.R. 5860",
  "description": "Rural Housing Service—Rural Community Facilities Program Account",
  "division": "A",
  "dollars": 25300000,
  "match_tier": "exact",
  "old_dollars": 75300000,
  "provision_index": 3,
  "provision_type": "cr_substitution",
  "quality": "strong",
  "raw_text": "except section 521(a)(2) shall be applied by substituting ''$25,300,000'' for ''$75,300,000''",
  "section": "SEC. 101",
  "semantics": "new_budget_authority"
}

Sort by largest cut using jq

congress-approp search --dir data/118-hr5860 --type cr_substitution --format json | \
  jq 'map(. + {delta: (.dollars - .old_dollars)}) | sort_by(.delta) | .[] |
    "\(.delta)\t\(.account_name // "unnamed")"'

Match CR Substitutions to Omnibus Provisions

A natural follow-up question is: “This CR cut Rural Water from $325M to $60M. What did the full-year omnibus give it?”

Using –similar

If embeddings are available, use --similar to find the omnibus counterpart. First, find the CR substitution’s provision index:

congress-approp search --dir data/118-hr5860 --type cr_substitution --format json | \
  jq '.[] | select(.account_name | test("Rural.*Water"; "i")) | .provision_index'

Then find similar provisions across all bills:

congress-approp search --dir data --similar hr5860:<INDEX> --type appropriation --top 3

Even though the CR names accounts differently than the omnibus (e.g., “Rural Utilities Service—Rural Water and Waste Disposal Program Account” vs. “Rural Water and Waste Disposal Program Account”), the embedding similarity is typically in the 0.75–0.80 range — well above the threshold for confident matching.

Using –account

If the names are close enough, a substring search works:

congress-approp search --dir data/118-hr4366 --account "Rural Water" --type appropriation

This will find the omnibus appropriation for the same program, letting you compare the CR anomaly level to the full-year funding.

Understanding the CR Structure

Not all provisions in a CR are substitutions. The full structure of H.R. 5860 includes:

The continuing_resolution_baseline provision (usually SEC. 101) establishes the default rule: fund everything at the prior fiscal year’s rate. The CR substitutions are exceptions to that rule. Everything else — riders, mandatory extensions, limitations — modifies or supplements the baseline.

To see the full picture:

# All provisions in the CR
congress-approp search --dir data/118-hr5860

# The baseline mechanism
congress-approp search --dir data/118-hr5860 --type continuing_resolution_baseline

# Mandatory programs extended
congress-approp search --dir data/118-hr5860 --type mandatory_spending_extension

# Standalone appropriations (FEMA, etc.)
congress-approp search --dir data/118-hr5860 --type appropriation

Verify CR Substitution Amounts

Both dollar amounts in each CR substitution are independently verified. You can confirm this in the audit:

congress-approp audit --dir data/118-hr5860

The audit shows NotFound = 0 for H.R. 5860, meaning every dollar string — including both the “new” and “old” amounts in all 13 CR substitutions — was found in the source bill text.

To verify a specific pair manually:

# Check that both amounts from the Migration and Refugee Assistance anomaly exist
grep '915,048,000' data/118-hr5860/BILLS-118hr5860enr.xml
grep '1,535,048,000' data/118-hr5860/BILLS-118hr5860enr.xml

Both should return matches. The source text will show them adjacent to each other in a “substituting X for Y” pattern.

Tips for CR Analysis

1. CRs don’t show the full funding picture. Programs not mentioned in CR substitutions are funded at the prior-year rate. The CR itself doesn’t state what that rate is — you need the prior year’s appropriations bill to know the baseline.

2. Watch for paired substitutions. The two FAA provisions at the bottom of the table (SEC. 137) have opposite deltas: +$47M for Facilities and Equipment and -$47M for the same agency’s account. This is a reallocation within the same agency — not a net change in FAA funding.

3. Some substitutions reference statute sections, not accounts. When the bill says “section 521(d)(1) shall be applied by substituting X for Y,” the tool captures both amounts but may not identify the account name. Check the raw_text field for the statutory reference and look it up in the U.S. Code.

4. Export and sort by delta for the narrative. The story is always “which programs got cut, which got more, and by how much.” Export to CSV, sort by delta, and you have the outline for a briefing or article.

5. Use --similar to find the regular appropriation. Every CR anomaly corresponds to a regular appropriation in an omnibus or annual bill. The --similar command finds that correspondence even when naming conventions differ between bills.

Quick Reference

# Find all CR substitutions
congress-approp search --dir data/118-hr5860 --type cr_substitution

# Export to CSV
congress-approp search --dir data/118-hr5860 --type cr_substitution --format csv > cr_subs.csv

# Export to JSON
congress-approp search --dir data/118-hr5860 --type cr_substitution --format json

# Find the full-year omnibus equivalent of a CR account
congress-approp search --dir data --similar hr5860:<INDEX> --type appropriation --top 3

# See all CR provisions (not just substitutions)
congress-approp search --dir data/118-hr5860

# Audit CR verification
congress-approp audit --dir data/118-hr5860

Next Steps

• Compare Two Bills — account-level comparison between a CR and an omnibus
• Track a Program Across Bills — use --similar to match CR accounts to their omnibus counterparts
• The Provision Type System — detailed documentation of all 11 provision types including cr_substitution

Use the Library API from Rust

You will need: A Rust project with congress-appropriations as a dependency.

You will learn: How to load extracted bill data, query it programmatically using the library API, and build custom analysis tools on top of the structured provision data.

congress-appropriations is both a CLI tool and a Rust library. The library exposes the same query functions the CLI uses — summarize, search, compare, audit, rollup_by_department, and build_embedding_text — as pure functions that take loaded bill data and return plain data structs. No I/O, no formatting, no side effects.

This guide shows you how to use the library in your own Rust projects.

Add the Dependency

Add congress-appropriations to your Cargo.toml:

[dependencies]
congress-appropriations = "3.0"

The crate re-exports the key types you need:

#![allow(unused)]
fn main() {
use congress_appropriations::{load_bills, query, LoadedBill};
use congress_appropriations::approp::query::SearchFilter;
}

Load Bills

The entry point is load_bills(), which recursively walks a directory to find all extraction.json files and loads them along with their sibling verification and metadata files:

use congress_appropriations::load_bills;
use std::path::Path;

fn main() -> anyhow::Result<()> {
    let bills = load_bills(Path::new("data"))?;
    println!("Loaded {} bills", bills.len());

    for bill in &bills {
        println!(
            "  {} ({}) — {} provisions",
            bill.extraction.bill.identifier,
            bill.extraction.bill.classification,
            bill.extraction.provisions.len()
        );
    }

    Ok(())
}

Expected output with the included example data:

Loaded 3 bills
  H.R. 4366 (Omnibus) — 2364 provisions
  H.R. 5860 (Continuing Resolution) — 130 provisions
  H.R. 9468 (Supplemental) — 7 provisions

What LoadedBill Contains

Each LoadedBill has three fields:

#![allow(unused)]
fn main() {
pub struct LoadedBill {
    /// Path to the bill directory on disk
    pub dir: PathBuf,
    /// The extraction output: bill info, provisions array, summary
    pub extraction: BillExtraction,
    /// Verification report (if verification.json exists)
    pub verification: Option<VerificationReport>,
    /// Extraction metadata (if metadata.json exists)
    pub metadata: Option<ExtractionMetadata>,
}
}

Only extraction is required — verification and metadata are loaded if their files exist but are None otherwise. This means you can use the library on data that was only partially extracted.

Summarize Bills

The summarize function computes per-bill budget authority, rescissions, and net BA:

use congress_appropriations::{load_bills, query};
use std::path::Path;

fn main() -> anyhow::Result<()> {
    let bills = load_bills(Path::new("data"))?;
    let summaries = query::summarize(&bills);

    for s in &summaries {
        println!(
            "{}: ${:>15} BA, ${:>13} rescissions, ${:>15} net",
            s.identifier,
            format_dollars(s.budget_authority),
            format_dollars(s.rescissions),
            format_dollars(s.net_ba),
        );
    }

    Ok(())
}

fn format_dollars(n: i64) -> String {
    // Simple comma formatting for display
    let s = n.to_string();
    let mut result = String::new();
    for (i, c) in s.chars().rev().enumerate() {
        if i > 0 && i % 3 == 0 && c != '-' {
            result.push(',');
        }
        result.push(c);
    }
    result.chars().rev().collect()
}

BillSummary Fields

#![allow(unused)]
fn main() {
pub struct BillSummary {
    pub identifier: String,      // e.g., "H.R. 4366"
    pub classification: String,  // e.g., "Omnibus"
    pub provisions: usize,       // total provision count
    pub budget_authority: i64,   // sum of new_budget_authority provisions
    pub rescissions: i64,        // sum of rescission provisions (absolute)
    pub net_ba: i64,             // budget_authority - rescissions
    pub completeness_pct: Option<f64>, // from verification, if available
}
}

Budget authority is computed from the actual provisions — it sums all Appropriation provisions where semantics == NewBudgetAuthority and detail_level is not sub_allocation or proviso_amount. The LLM’s self-reported totals are never used.

Search Provisions

The search function takes a SearchFilter and returns matching provisions:

#![allow(unused)]
fn main() {
use congress_appropriations::approp::query::{SearchFilter, SearchResult};

let results = query::search(&bills, &SearchFilter {
    provision_type: Some("appropriation"),
    agency: Some("Veterans"),
    min_dollars: Some(1_000_000_000),
    ..Default::default()
});

for r in &results {
    println!(
        "[{}] {} — ${:?}",
        r.bill_identifier, r.account_name, r.dollars
    );
}
}

SearchFilter Fields

All fields are optional and use AND logic — every field that is Some must match:

#![allow(unused)]
fn main() {
pub struct SearchFilter<'a> {
    pub provision_type: Option<&'a str>,  // e.g., "appropriation"
    pub agency: Option<&'a str>,          // case-insensitive substring
    pub account: Option<&'a str>,         // case-insensitive substring
    pub keyword: Option<&'a str>,         // search in raw_text
    pub bill: Option<&'a str>,            // exact bill identifier
    pub division: Option<&'a str>,        // division letter, e.g., "A"
    pub min_dollars: Option<i64>,         // minimum absolute dollar amount
    pub max_dollars: Option<i64>,         // maximum absolute dollar amount
}
}

You can construct a filter with defaults for all fields and override just the ones you care about:

#![allow(unused)]
fn main() {
let filter = SearchFilter {
    provision_type: Some("rescission"),
    min_dollars: Some(100_000_000),
    ..Default::default()
};
}

Compare Bills

The compare function computes account-level deltas between two sets of bills:

#![allow(unused)]
fn main() {
let base_bills = load_bills(Path::new("data/118-hr4366"))?;
let current_bills = load_bills(Path::new("data/118-hr9468"))?;

let deltas = query::compare(&base_bills, &current_bills, None);

for d in &deltas {
    println!(
        "{}: base=${}, current=${}, delta={} ({})",
        d.account_name,
        d.base_dollars,
        d.current_dollars,
        d.delta,
        d.status,
    );
}
}

The optional third parameter is an agency filter (Option<&str>) that restricts the comparison to accounts from a specific agency.

AccountDelta Fields

#![allow(unused)]
fn main() {
pub struct AccountDelta {
    pub agency: String,
    pub account_name: String,
    pub base_dollars: i64,
    pub current_dollars: i64,
    pub delta: i64,
    pub delta_pct: f64,
    pub status: String,  // "changed", "unchanged", "only in base", "only in current"
}
}

Results are sorted by the absolute value of delta, largest changes first.

Audit Bills

The audit function returns per-bill verification metrics:

#![allow(unused)]
fn main() {
let audit_rows = query::audit(&bills);

for row in &audit_rows {
    println!(
        "{}: {} provisions, {} verified, {} not found, {:.1}% coverage",
        row.identifier,
        row.provisions,
        row.verified,
        row.not_found,
        row.completeness_pct.unwrap_or(0.0),
    );
}
}

AuditRow Fields

#![allow(unused)]
fn main() {
pub struct AuditRow {
    pub identifier: String,
    pub provisions: usize,
    pub verified: usize,       // dollar amounts found at unique position
    pub not_found: usize,      // dollar amounts NOT found in source
    pub ambiguous: usize,      // dollar amounts found at multiple positions
    pub exact: usize,          // raw_text byte-identical match
    pub normalized: usize,     // raw_text normalized match
    pub spaceless: usize,      // raw_text spaceless match
    pub no_match: usize,       // raw_text not found
    pub completeness_pct: Option<f64>,
}
}

The critical metric is not_found — it should be 0 for every bill. Across the included example data, it is.

Roll Up by Department

The rollup_by_department function aggregates budget authority by parent department. This is a query-time computation — it never modifies stored data:

#![allow(unused)]
fn main() {
let agencies = query::rollup_by_department(&bills);

for a in &agencies {
    println!(
        "{}: ${} BA, ${} rescissions, {} provisions",
        a.department,
        a.budget_authority,
        a.rescissions,
        a.provision_count,
    );
}
}

Agency names are split at the first comma to extract the parent department (e.g., “Salaries and Expenses, Federal Bureau of Investigation” → “Federal Bureau of Investigation”). The exception is “Office of Inspector General, …” which takes the text after the comma.

Results are sorted by budget authority descending.

Build Embedding Text

The build_embedding_text function constructs the deterministic text representation used for embedding a provision. This is useful if you want to use your own embedding model instead of OpenAI:

#![allow(unused)]
fn main() {
use congress_appropriations::approp::ontology::Provision;

for provision in &bills[0].extraction.provisions[..3] {
    let text = query::build_embedding_text(provision);
    println!("Embedding text ({} chars): {}...",
        text.len(),
        &text[..text.len().min(100)]
    );
}
}

The text concatenates the provision’s meaningful fields (account name, agency, program, raw text) in a consistent format. The same provision always produces the same text, regardless of when or where you call the function.

Access Provision Fields Directly

The Provision enum has 11 variants. Accessor methods provide a uniform interface across all variants:

#![allow(unused)]
fn main() {
use congress_appropriations::approp::ontology::{Provision, AmountSemantics};

for bill in &bills {
    for p in &bill.extraction.provisions {
        // These methods work on all provision variants:
        let ptype = p.provision_type_str();   // e.g., "appropriation"
        let account = p.account_name();       // "" if not applicable
        let agency = p.agency();              // "" if not applicable
        let section = p.section();            // e.g., "SEC. 101"
        let division = p.division();          // Some("A") or None
        let raw_text = p.raw_text();          // bill text excerpt
        let confidence = p.confidence();      // 0.0-1.0

        // Amount access returns Option<&DollarAmount>
        if let Some(amt) = p.amount() {
            if matches!(amt.semantics, AmountSemantics::NewBudgetAuthority) {
                if let Some(dollars) = amt.dollars() {
                    println!("{}: ${}", account, dollars);
                }
            }
        }
    }
}
}

Key accessor methods

Pattern matching for type-specific fields

When you need fields specific to a provision type, use pattern matching:

#![allow(unused)]
fn main() {
match p {
    Provision::Appropriation {
        account_name,
        agency,
        amount,
        detail_level,
        parent_account,
        fiscal_year,
        availability,
        ..
    } => {
        println!("Appropriation: {} (detail: {})", account_name, detail_level);
        if let Some(parent) = parent_account {
            println!("  Sub-allocation of: {}", parent);
        }
    }
    Provision::CrSubstitution {
        account_name,
        new_amount,
        old_amount,
        ..
    } => {
        let new_d = new_amount.dollars().unwrap_or(0);
        let old_d = old_amount.dollars().unwrap_or(0);
        println!("CR Sub: {} — ${} → ${} (delta: ${})",
            account_name.as_deref().unwrap_or("unnamed"),
            old_d, new_d, new_d - old_d);
    }
    Provision::Rescission {
        account_name,
        amount,
        reference_law,
        ..
    } => {
        println!("Rescission: {} — ${}", account_name, amount.dollars().unwrap_or(0));
        if let Some(law) = reference_law {
            println!("  From: {}", law);
        }
    }
    _ => {
        // Handle other provision types generically
        println!("{}: {}", p.provision_type_str(), p.description());
    }
}
}

Compute Budget Authority Manually

The BillExtraction struct has a compute_totals() method that returns (budget_authority, rescissions):

#![allow(unused)]
fn main() {
for bill in &bills {
    let (ba, rescissions) = bill.extraction.compute_totals();
    let net = ba - rescissions;
    println!("{}: BA=${}, Rescissions=${}, Net=${}",
        bill.extraction.bill.identifier, ba, rescissions, net);
}
}

This uses the same logic as the summary command: it sums Appropriation provisions where semantics == NewBudgetAuthority and detail_level is not sub_allocation or proviso_amount.

Full Working Example

Here’s a complete program that loads all example bills, finds the top 10 appropriations by dollar amount, and prints them:

use congress_appropriations::{load_bills, query};
use congress_appropriations::approp::query::SearchFilter;
use std::path::Path;

fn main() -> anyhow::Result<()> {
    // Load all bills under data/
    let bills = load_bills(Path::new("data"))?;
    println!("Loaded {} bills with {} total provisions\n",
        bills.len(),
        bills.iter().map(|b| b.extraction.provisions.len()).sum::<usize>()
    );

    // Search for all appropriations
    let results = query::search(&bills, &SearchFilter {
        provision_type: Some("appropriation"),
        ..Default::default()
    });

    // Sort by dollars descending, take top 10
    let mut with_dollars: Vec<_> = results.iter()
        .filter(|r| r.dollars.is_some())
        .collect();
    with_dollars.sort_by(|a, b| b.dollars.unwrap().abs().cmp(&a.dollars.unwrap().abs()));

    println!("Top 10 appropriations by dollar amount:");
    println!("{:<50} {:>20} {}", "Account", "Amount", "Bill");
    println!("{}", "-".repeat(85));

    for r in with_dollars.iter().take(10) {
        println!("{:<50} ${:>18} {}",
            &r.account_name[..r.account_name.len().min(48)],
            r.dollars.unwrap(),
            r.bill_identifier,
        );
    }

    // Budget summary
    println!("\nBudget Summary:");
    for s in query::summarize(&bills) {
        println!("  {}: ${} BA, ${} rescissions",
            s.identifier, s.budget_authority, s.rescissions);
    }

    Ok(())
}

Design Principles

The library API follows these conventions:

1. All query functions are pure. They take &[LoadedBill] and return data. No side effects, no I/O, no API calls, no formatting.

2. The CLI formats; the library computes. main.rs handles table/JSON/CSV/JSONL rendering. The library returns structs that derive Serialize for easy JSON output.

3. Semantic search is separate. Embedding loading and cosine similarity live in embeddings.rs, not query.rs. This keeps the library usable without OpenAI. The CLI wires them together for --semantic and --similar searches.

4. Error handling uses anyhow. All fallible functions return anyhow::Result<T>. For library consumers who prefer typed errors, the underlying error types from thiserror are also available.

5. Serde for everything. All data types derive Serialize and Deserialize. You can serialize any query result to JSON with serde_json::to_string(&results)?.

Working with Embeddings

The embeddings module is separate from the query module. If you want to work with embedding vectors directly:

#![allow(unused)]
fn main() {
use congress_appropriations::approp::embeddings;
use std::path::Path;

// Load embeddings for a bill
if let Some(loaded) = embeddings::load(Path::new("data/118-hr9468"))? {
    println!("Loaded {} vectors of {} dimensions",
        loaded.count(), loaded.dimensions());

    // Get the vector for provision 0
    let vec0 = loaded.vector(0);
    println!("First 5 dimensions: {:?}", &vec0[..5]);

    // Compute cosine similarity between two provisions
    let sim = embeddings::cosine_similarity(loaded.vector(0), loaded.vector(1));
    println!("Similarity between provisions 0 and 1: {:.4}", sim);
}
}

Key embedding functions

Tips

1. Load once, query many times. load_bills() does all the file I/O. After that, all query functions work on in-memory data and are extremely fast.

2. Use SearchFilter::default() as a base. Override only the fields you need — all None fields are unrestricted.

3. Check provision_type_str() instead of pattern matching when you just need the type name as a string.

4. The amount() accessor returns None for provisions without dollar amounts. Riders, directives, and some other types don’t carry amounts. Always handle the None case.

5. Budget authority totals should match the CLI. If compute_totals() returns different numbers than congress-approp summary, something is wrong. The included example data produces these exact totals: H.R. 4366 = $846,137,099,554 BA; H.R. 5860 = $16,000,000,000 BA; H.R. 9468 = $2,882,482,000 BA.

Next Steps

• Architecture Overview — understand how the crate is structured internally
• extraction.json Fields — complete field reference for the data structures
• Adding a New Provision Type — extend the library with new provision types

Upgrade Extraction Data

You will need: congress-approp installed, existing extracted bill data (with extraction.json).

You will learn: How to use the upgrade command to migrate extraction data to the latest schema version, re-verify against current code, and update files — all without making any LLM API calls.

The upgrade command is your tool for keeping extraction data current without re-extracting. When the tool’s schema evolves — new fields, renamed fields, new verification checks, or updated deserialization logic — upgrade applies those changes to your existing data. It re-deserializes each bill’s extraction.json through the current code’s parsing logic, re-runs deterministic verification against the source XML, and writes updated files.

No LLM API calls are made. Upgrade is fast, free, and safe.

When to Use Upgrade

Use upgrade when:

• You’ve updated congress-approp to a new version that includes schema changes, new provision type handling, or improved verification logic. The upgrade command applies those improvements to your existing extractions.
• You want to re-verify without re-extracting. Maybe you suspect the verification logic has been improved, or you want to check data integrity after moving files between systems.
• You see schema version warnings. If your data was extracted with an older schema version and the tool detects this, it may suggest running upgrade.
• You want to normalize data. Upgrade re-serializes through the current schema, which normalizes field names, fills in defaults for new fields, and standardizes enum values.

Do NOT use upgrade when:

• You want a fresh extraction with a different model. Use extract instead — that makes new LLM API calls.
• Your source XML has changed. If you re-downloaded the bill, you need to re-extract, not upgrade.

Preview Before Upgrading

Always start with a dry run:

congress-approp upgrade --dir data --dry-run

This shows what would change for each bill without writing any files:

• Which bills would be upgraded
• Whether the schema version would change
• How many provisions would be re-parsed
• Whether verification results would differ

No files are modified during a dry run.

Run the Upgrade

Upgrade all bills in a directory

congress-approp upgrade --dir data

The tool walks recursively from the specified directory, finds every extraction.json, and upgrades each one. For each bill:

1. Load the existing extraction.json
2. Re-deserialize every provision through the current from_value.rs parsing logic, which handles missing fields, type coercions, and unknown provision types
3. Re-compute the schema_version field
4. Re-run verification against the source XML (if BILLS-*.xml is present in the same directory)
5. Write updated extraction.json and verification.json

Upgrade a single bill

congress-approp upgrade --dir data/118/hr/9468

Verbose output

Add -v for detailed logging:

congress-approp upgrade --dir data -v

This shows per-provision details: which fields were defaulted, which types were coerced, and any warnings from the deserialization process.

What Upgrade Changes

extraction.json

• schema_version is set to the current version
• New fields added in recent versions get their default values (e.g., a new Option<String> field defaults to null)
• Renamed fields are mapped from old names to new names
• Type coercions are applied — for example, if a dollar amount was stored as a string "$10,000,000" in an old extraction, upgrade converts it to the integer 10000000
• Unknown provision types that have since been added to the schema are re-parsed into their proper variant instead of falling back to Other

The provision data itself is not re-generated — upgrade works with whatever the LLM originally produced. It only normalizes the representation, not the content.

verification.json

Verification is fully re-run against the source XML:

• Amount checks — Every text_as_written dollar string is searched for in the source text
• Raw text checks — Every raw_text excerpt is checked as a substring of the source (exact → normalized → spaceless → no match)
• Completeness — The percentage of dollar strings in the source text matched to extracted provisions is recomputed

If the source XML (BILLS-*.xml) is not present in the bill directory, verification is skipped and the existing verification.json is left unchanged.

metadata.json

The source_xml_sha256 field is added or updated if the source XML is present. This is part of the hash chain that enables staleness detection for downstream artifacts (embeddings).

What is NOT changed

• The provisions themselves — the LLM’s original extraction is preserved. Upgrade doesn’t re-classify provisions, change account names, or modify dollar amounts.
• tokens.json — Token usage records from the original extraction are untouched.
• chunks/ — Per-chunk LLM artifacts are not modified.
• embeddings.json / vectors.bin — Embeddings are not regenerated. If the upgrade changes extraction.json, the embeddings become stale. The tool will warn you about this, and you can run embed to regenerate.

Handling the SuchSums Fix

One specific issue that upgrade addresses: in early versions, SuchSums amount variants (for “such sums as may be necessary” provisions) could serialize incorrectly. The upgrade command detects and fixes this, converting them to the proper tagged enum format. This is transparent — you don’t need to do anything special.

After Upgrading

Check the audit

Run audit to see whether verification metrics improved:

congress-approp audit --dir data

If the upgrade applied new verification logic, you may see changes in the Exact/NormText/TextMiss columns. The NotFound column should remain at 0 (it would only increase if the upgrade somehow corrupted dollar amount strings, which it doesn’t).

Check for stale embeddings

If upgrade modified extraction.json, the hash chain detects that embeddings are stale:

⚠ H.R. 4366: embeddings are stale (extraction.json has changed)

Regenerate embeddings if you use semantic search:

congress-approp embed --dir data

Verify budget authority totals

As a sanity check, confirm that budget authority totals haven’t changed:

congress-approp summary --dir data --format json

Upgrade should never change the dollar amounts in provisions, so budget authority totals should be identical before and after. If they differ, something unexpected happened — file a bug report.

For the included example data, the expected totals are:

Upgrade vs. Re-Extract: Decision Guide

Key principle: upgrade preserves the LLM’s work and improves how it’s stored and verified. extract discards the LLM’s work and starts over.

Troubleshooting

“No extraction.json found”

The upgrade command only processes directories that already contain extraction.json. If you haven’t extracted a bill yet, use extract first.

“No source XML found — skipping verification”

Upgrade re-runs verification against the source XML. If the BILLS-*.xml file isn’t in the bill directory (maybe you moved files around), verification is skipped. The extraction data is still upgraded, but verification.json won’t be updated.

To fix, make sure the source XML is in the same directory as extraction.json:

ls data/118/hr/9468/
# Should show both BILLS-118hr9468enr.xml and extraction.json

Budget authority totals changed after upgrade

This should not happen. If it does:

1. Compare the pre-upgrade and post-upgrade extraction.json using diff or a JSON diff tool
2. Look for provisions whose detail_level or semantics changed — these fields affect the budget authority calculation
3. File a bug report with the before/after data

Quick Reference

# Preview what would change (no files modified)
congress-approp upgrade --dir data --dry-run

# Upgrade all bills under a directory
congress-approp upgrade --dir data

# Upgrade a single bill
congress-approp upgrade --dir data/118/hr/9468

# Upgrade with verbose logging
congress-approp upgrade --dir data -v

# Verify after upgrading
congress-approp audit --dir data

# Regenerate stale embeddings after upgrade
congress-approp embed --dir data

Full Command Reference

congress-approp upgrade [OPTIONS]

Options:
    --dir <DIR>  Data directory to upgrade [default: ./data]
    --dry-run    Show what would change without writing files

Next Steps

• Verify Extraction Accuracy — run a full audit after upgrading
• Extract Provisions from a Bill — when upgrade isn’t enough and you need a fresh extraction
• Data Integrity and the Hash Chain — understand how the hash chain detects stale artifacts

Enrich Bills with Metadata

The enrich command generates bill_meta.json for each bill directory, enabling fiscal year filtering, subcommittee scoping, and advance appropriation classification. Unlike extraction (which requires an Anthropic API key) or embedding (which requires an OpenAI API key), enrichment runs entirely offline.

Quick Start

# Enrich all bills in the data directory
congress-approp enrich --dir data

This creates a bill_meta.json file in each bill directory. You only need to run it once per bill — the tool skips bills that already have metadata unless you pass --force.

What It Enables

After enriching, you can use these filtering options on summary, search, and compare:

# See only FY2026 bills
congress-approp summary --dir data --fy 2026

# Search within a specific subcommittee
congress-approp search --dir data --type appropriation --fy 2026 --subcommittee thud

# Combine semantic search with FY and subcommittee filtering
congress-approp search --dir data --semantic "housing assistance" --fy 2026 --subcommittee thud --top 5

# Compare THUD funding across fiscal years
congress-approp compare --base-fy 2024 --current-fy 2026 --subcommittee thud --dir data

Note: The --fy flag works without enrich — it uses the fiscal year data already in extraction.json. But --subcommittee requires the division-to-jurisdiction mapping that only enrich provides.

Note on embeddings: Semantic search (the --semantic flag) requires embedding vectors. If you cloned the git repository, pre-generated vectors.bin files are included for all example bills. If you installed via cargo install, the embedding files are not included (they exceed the crates.io size limit) — run congress-approp embed --dir data to generate them (~30 seconds per bill, requires OPENAI_API_KEY). The enrich command itself does not require embeddings and does not use any API keys.

What It Generates

The enrich command creates a bill_meta.json file in each bill directory containing five categories of metadata:

Subcommittee Mappings

Each division in an omnibus or minibus bill gets mapped to a canonical jurisdiction. The tool parses division titles directly from the enrolled bill XML and classifies them using pattern matching:

This solves the problem where Division A means Defense in one bill but CJS in another — the --subcommittee flag uses the canonical jurisdiction, not the letter.

Available subcommittee slugs for --subcommittee:

Advance Appropriation Classification

Each budget authority provision is classified as:

• current_year — money available in the fiscal year the bill funds
• advance — money enacted now but available in a future fiscal year
• supplemental — additional emergency or supplemental funding
• unknown — a future fiscal year is referenced but no known pattern was matched

The classification uses a fiscal-year-aware algorithm:

1. Extract “October 1, YYYY” from the provision’s availability text — this means funds available starting fiscal year YYYY+1
2. Extract “first quarter of fiscal year YYYY” — this means funds for FY YYYY
3. Compare the availability year to the bill’s fiscal year
4. If the availability year is later than the bill’s fiscal year → advance
5. If the availability year equals the bill’s fiscal year → current_year (start of the funded FY)
6. Check provision notes for “supplemental” → supplemental
7. Default to current_year

This correctly handles cases like:

• H.R. 4366 (FY2024): VA Compensation and Pensions “available October 1, 2024” → advance for FY2025 ($182 billion)
• H.R. 7148 (FY2026): Medicaid “for the first quarter of fiscal year 2027” → advance for FY2027 ($316 billion)
• H.R. 7148 (FY2026): Tenant-Based Rental Assistance “available October 1, 2026” → advance for FY2027 ($4 billion)

Across the 13-bill dataset, the algorithm identifies $1.49 trillion in advance appropriations — approximately 24% of total budget authority. Failing to separate advance from current-year can cause year-over-year comparisons to be off by hundreds of billions of dollars.

Bill Nature

The enriched bill classification provides finer distinctions than the original LLM classification:

The classification uses provision type distribution and subcommittee count: 5+ real subcommittees = omnibus, 2-4 = minibus, CR baseline + many appropriations without multiple subcommittees = full-year CR with appropriations.

Canonical Account Names

Every account name is normalized for cross-bill matching:

Normalization lowercases, strips em-dash and en-dash prefixes, and trims whitespace. This eliminates false orphans in compare caused by capitalization differences and hierarchical naming conventions.

Classification Provenance

Every classification in bill_meta.json records how it was determined:

{
  "timing": "advance",
  "available_fy": 2027,
  "source": {
    "type": "fiscal_year_comparison",
    "availability_fy": 2027,
    "bill_fy": 2026
  }
}

This means: “classified as advance because the money becomes available in FY2027 but the bill covers FY2026.” Provenance types include xml_structure, pattern_match, fiscal_year_comparison, note_text, and default_rule.

When to Re-Enrich

The tool automatically detects when bill_meta.json is stale — when extraction.json has changed since enrichment. You will see a warning:

⚠ H.R. 7148: bill metadata is stale (extraction.json has changed). Run `enrich --force`.

Run enrich --force to regenerate metadata for all bills.

Flags

Previewing Before Writing

Use --dry-run to see what the enrich command would produce without writing any files:

congress-approp enrich --dir data --dry-run

  would enrich H.R. 1968: nature=FullYearCrWithAppropriations, 3 divisions, 192 BA provisions (8 advance, 3 supplemental)
  would enrich H.R. 4366: nature=Omnibus, 7 divisions, 511 BA provisions (11 advance, 4 supplemental)
  would enrich H.R. 7148: nature=Omnibus, 9 divisions, 505 BA provisions (11 advance, 4 supplemental)
  ...

Using with Compare

The compare command benefits most from enrichment. Without enrich, comparing two omnibus bills that cover different subcommittees produces hundreds of false orphans. With enrichment and --subcommittee scoping:

# Before: 759 orphans (mixing Defense with Agriculture)
congress-approp compare --base data/118-hr4366 --current data/118-hr7148

# After: 43 meaningful changes, 12 unchanged
congress-approp compare --base-fy 2024 --current-fy 2026 --subcommittee thud --dir data

The --base-fy and --current-fy flags automatically select the right bills for each fiscal year and the --subcommittee flag scopes to the correct division in each bill.

Known Limitations

• Sub-agency mismatches — the LLM sometimes uses sub-agency names (e.g., “Maritime Administration”) in one bill and parent department names (e.g., “Department of Transportation”) in another. The compare command includes a 35-entry sub-agency-to-parent-department lookup table that resolves most of these, but some agency naming inconsistencies (~5-15 orphans per subcommittee) may remain for agencies not in the table.
• 17 supplemental policy division titles (e.g., “FEND Off Fentanyl Act”, “Protecting Americans from Foreign Adversary Controlled Applications Act”) are classified as other jurisdiction by default. These are from just two bills (H.R. 815 and S. 870) and don’t affect regular appropriations bill analysis.
• Advance detection patterns cover “October 1, YYYY” and “first quarter of fiscal year YYYY.” If Congress uses novel phrasing in future bills, those provisions would default to current_year. The tool logs a warning when it detects a provision referencing a future fiscal year but not matching any known advance pattern.

Related

• The Extraction Pipeline — where enrich fits in the overall pipeline
• Data Integrity and the Hash Chain — how staleness detection works for bill_meta.json
• CLI Command Reference — complete flag reference for enrich and other commands
• Data Directory Layout — where bill_meta.json lives in the directory structure

Adjust for Inflation

When comparing appropriations across fiscal years, nominal dollar changes can be misleading. A program that received $100M in FY2024 and $104M in FY2026 looks like it got a 4% increase — but if inflation over that period was 3.9%, the real increase is only 0.1%. The program’s purchasing power barely changed.

The --real flag on compare adds inflation-adjusted context to every row, showing you which programs received real increases and which ones lost ground to inflation.

Quick Start

# Compare THUD FY2024 → FY2026 with inflation adjustment
congress-approp compare --base-fy 2024 --current-fy 2026 --subcommittee thud --dir data --real

The output adds two columns: Real Δ %* (the inflation-adjusted percentage change) and a directional indicator:

• ▲ — real increase (nominal change exceeded inflation)
• ▼ — real cut or inflation erosion (purchasing power decreased)
• — — unchanged in both nominal and real terms

The asterisk on “Real Δ %*” reminds you this is a computed value based on an external price index, not a number verified against bill text.

A summary line at the bottom counts how many programs beat inflation and how many fell behind.

What It Shows

Account                   Base ($)       Current ($)   Δ %    Real Δ %*  
TBRA                   28,386,831,000  34,438,557,000  +21.3%  +16.7%  ▲
Project-Based Rental   16,010,000,000  18,543,000,000  +15.8%  +11.4%  ▲
Operations (FAA)       12,729,627,000  13,710,000,000   +7.7%   +3.6%  ▲
Public Housing Fund     8,810,784,000   8,319,393,000   -5.6%   -9.1%  ▼
Capital Inv Grants      2,205,000,000   1,700,000,000  -22.9%  -25.8%  ▼
Payment to NRC            158,000,000     158,000,000    0.0%   -3.9%  ▼

45 beat inflation, 17 fell behind | CPI-U FY2024→FY2026: 3.9% (2 months of FY2026 data)

Key insight: “Payment to NRC” got the exact same dollar amount both years. Nominally that’s “unchanged.” But after adjusting for 3.9% inflation, it’s effectively a 3.9% cut in purchasing power. The ▼ flag makes this visible at a glance.

How It Works

The tool ships with a bundled CPI data file containing monthly Consumer Price Index values from the Bureau of Labor Statistics (CPI-U All Items, series CUUR0000SA0). When you pass --real:

1. The tool identifies the base and current fiscal years from the comparison
2. It computes fiscal-year-weighted CPI averages (October through September) from the monthly data
3. The inflation rate is the ratio: current_fy_cpi / base_fy_cpi - 1
4. For each row, the real percentage change is: (current / (base × (1 + inflation))) - 1
5. The inflation flag compares the nominal change to the inflation rate

The bundled CPI data is compiled into the binary — no network access is needed. It’s updated with each tool release.

Using Your Own Price Index

The default deflator is CPI-U (Consumer Price Index for All Urban Consumers), which is the standard measure used in journalism and public policy discussion. However, different analyses may call for different deflators:

• GDP Deflator — used by CBO for aggregate budget analysis; broader than CPI
• PCE Price Index — the Federal Reserve’s preferred measure; typically 0.3–0.5% below CPI
• Sector-specific deflators — DoD procurement indices, medical care CPI, construction cost indices

To use a different deflator, provide your own data file:

congress-approp compare --base-fy 2024 --current-fy 2026 --subcommittee thud --dir data \
  --real --cpi-file my_gdp_deflator.json

The file must follow this JSON schema:

{
  "source": "GDP Deflator (BEA NIPA Table 1.1.4)",
  "retrieved": "2026-03-15",
  "note": "Quarterly values interpolated to monthly",
  "monthly": {
    "2023-10": 118.432,
    "2023-11": 118.576,
    "2023-12": 118.701,
    "2024-01": 118.823,
    "...": "..."
  }
}

The tool reads the monthly values and computes fiscal-year averages (Oct–Sep) from them. The source and note fields are displayed in the output footer, so the reader knows exactly which deflator was used.

If you provide calendar-year annual averages instead of monthly data, you can use:

{
  "source": "My custom deflator",
  "retrieved": "2026-03-15",
  "annual_averages": {
    "2024": 118.9,
    "2025": 121.3,
    "2026": 123.1
  },
  "partial_years": {
    "2026": { "months": 2, "through": "2026-02" }
  }
}

The tool prefers monthly data for precise fiscal year computation, falling back to annual_averages (calendar year proxy) when monthly data is not available.

Understanding the Output

Nominal vs. Real

The nominal number answers: “What did Congress decide?” The real number answers: “Did the program’s purchasing power go up or down?”

Inflation Flags

The most important insight is inflation erosion: programs that received a nominal increase but still lost purchasing power. These are politically described as “increases” but economically function as cuts. The --real flag makes this visible.

The Footer

Every inflation-adjusted output includes a footer showing:

• The deflator used (CPI-U by default, or whatever --cpi-file specifies)
• The base and current fiscal year CPI values
• The inflation rate between them
• How many months of data are available for partial years
• A count of programs that beat or fell behind inflation

This metadata ensures the analysis is reproducible and the methodology is transparent.

CSV and JSON Output

CSV

With --real --format csv, the CSV output adds three columns:

account_name,agency,base_dollars,current_dollars,delta,delta_pct,status,real_delta_pct,inflation_flag

The inflation_flag values are: real_increase, real_cut, inflation_erosion, or unchanged. These are designed for filtering in spreadsheets — sort or filter on inflation_flag to find all programs that lost ground.

JSON

With --real --format json, the output includes an inflation metadata object:

{
  "inflation": {
    "source": "Bureau of Labor Statistics, CPI-U All Items (CUUR0000SA0)",
    "base_fy": 2024,
    "current_fy": 2026,
    "base_cpi": 311.6,
    "current_cpi": 325.1,
    "rate": 0.0434,
    "current_fy_months": 4,
    "note": "FY2026 based on 4 months of data (Oct 2025 – Jan 2026)"
  },
  "rows": [
    {
      "account_name": "Tenant-Based Rental Assistance",
      "base_dollars": 28386831000,
      "current_dollars": 34438557000,
      "delta": 6051726000,
      "delta_pct": 21.3,
      "real_delta_pct": 16.7,
      "inflation_flag": "real_increase",
      "status": "changed"
    }
  ],
  "summary": {
    "beat_inflation": 45,
    "fell_behind": 17,
    "inflation_rate_pct": 4.34
  }
}

Important Caveats

CPI-U is a consumer measure

CPI-U measures the cost of goods and services purchased by urban consumers — groceries, rent, gasoline, healthcare. Government spending has a different cost structure: federal employee salaries, military procurement, construction, transfer payments. CPI-U is the standard deflator for public-facing analysis but may not precisely reflect the cost pressures facing a specific government program.

For sector-specific analysis, consider using --cpi-file with a deflator appropriate to the spending category (medical care CPI for VA health, construction cost index for infrastructure, etc.).

Partial-year data

For the most recent fiscal year, CPI data may be incomplete. The output always notes how many months of data are available. The inflation rate may shift as more months are published — typically by 0.1–0.3 percentage points.

This is analysis, not extraction

Nominal dollar amounts in this tool are verified against the enrolled bill text — every number traces to a specific position in the source XML. Inflation-adjusted numbers are computed values that depend on an external data source (BLS) and methodology choices (CPI-U, fiscal year weighting). The asterisk on “Real Δ %*” marks this distinction. When citing inflation-adjusted figures, note the deflator used.

Updating the Bundled CPI Data

The tool includes CPI-U data current as of its release date. To use more recent data:

1. Download fresh monthly CPI from the BLS Public Data API or FRED
2. Format as the JSON schema shown above
3. Pass via --cpi-file

Alternatively, wait for the next tool release — each version bundles the latest available CPI data.

Related

• Compare Two Bills — the base comparison workflow that --real extends
• Budget Authority Calculation — how nominal budget authority is computed
• Why the Numbers Might Not Match Headlines — context for interpreting appropriations figures
• CLI Command Reference — full flag reference for compare

Resolving Agency and Account Name Differences Across Bills

When comparing appropriations across fiscal years, the same program sometimes appears under different agency names. The Army’s research budget might be listed under “Department of Defense—Army” in one bill and “Department of Defense—Department of the Army” in another. These are the same program, but the tool can’t tell without your help.

The dataset.json file at the root of your data directory is where you record these equivalences. Once recorded, every command — compare, relate, link suggest — uses them automatically.

The Problem

Run a Defense comparison and you’ll likely see orphan pairs:

congress-approp compare --base-fy 2024 --current-fy 2026 \
    --subcommittee defense --dir data

only in base    "RDT&E, Army"  agency="Department of Defense—Army"         $17.1B
only in current "RDT&E, Army"  agency="Department of Defense—Dept of Army" $16.7B

Same account name. Same program. Different agency string. The tool treats them as different accounts.

Two Ways to Discover Naming Variants

normalize suggest-text-match — Local analysis

congress-approp normalize suggest-text-match --dir data

Scans your data for orphan pairs (same account name on both sides of a cross-FY comparison, different agency name) and structural patterns (preposition variants like “of” vs “for”, prefix expansion like “Defense—Army” vs “Defense—Department of the Army”).

Runs entirely offline. No API calls. Instant.

Found 94 suggested agency groups (252 orphan pairs resolvable):

  1. [064847a5] [orphan-pair] "Department of Health and Human Services"
     = "National Institutes of Health"
     Evidence: 27 shared accounts (e.g., national cancer institute, ...)

  2. [3dec4083] [orphan-pair] "Centers for Disease Control and Prevention"
     = "Department of Health and Human Services"
     Evidence: 13 shared accounts (e.g., environmental health, ...)

Each suggestion has an 8-character hash for use with normalize accept.

Use --format hashes to output just the hashes (one per line) for scripting:

congress-approp normalize suggest-text-match --dir data --format hashes

Use --min-accounts N to only show pairs sharing N or more account names (higher = stronger evidence):

congress-approp normalize suggest-text-match --dir data --min-accounts 3

normalize suggest-llm — LLM-assisted classification

congress-approp normalize suggest-llm --dir data

Sends unresolved ambiguous accounts to Claude along with the XML heading context from each bill. The LLM sees the full organizational structure surrounding each provision — the [MAJOR] and [SUBHEADING] headings from the enrolled bill XML — and classifies agency pairs as SAME or DIFFERENT.

Requires ANTHROPIC_API_KEY. Uses Claude Opus.

The LLM uses three types of evidence:

• XML heading hierarchy — which department/agency heading the provision appears under in the bill structure
• Dollar amounts — similar amounts across years suggest the same program
• Institutional knowledge — understanding organizational relationships (e.g., Space Force is under Department of the Air Force)

Both suggest commands cache their results. Neither writes to dataset.json directly — use normalize accept to review and persist.

Accepting Suggestions

After running either suggest command, accept specific suggestions by hash:

congress-approp normalize accept 064847a5 3dec4083 --dir data

Or accept all cached suggestions at once:

congress-approp normalize accept --auto --dir data

The accept command reads from the suggestion cache (~/.congress-approp/cache/), matches hashes, and writes the accepted groups to dataset.json. If dataset.json already exists, new groups are merged with existing ones.

What dataset.json Looks Like

Open data/dataset.json in any text editor:

{
  "schema_version": "1.0",
  "entities": {
    "agency_groups": [
      {
        "canonical": "Department of Health and Human Services",
        "members": [
          "National Institutes of Health",
          "Centers for Disease Control and Prevention"
        ]
      }
    ],
    "account_aliases": [
      {
        "canonical": "Office for Civil Rights",
        "aliases": ["Office of Civil Rights"]
      }
    ]
  }
}

Each agency group says: when matching, treat all these agency names as equivalent. The canonical name is what appears in compare output. The members are variants that get mapped to it.

Each account alias maps variant spellings of an account name to a preferred form.

This file contains only user knowledge — decisions that cannot be derived from scanning bill files. There is no cached or derived data.

How Matching Works

When you run compare, relate, or link suggest, the tool matches provisions by (agency, account name). Here’s exactly what happens:

1. Both agency and account name are lowercased
2. Account name em-dash prefixes are stripped (“Dept—Account” → “account”)
3. If dataset.json exists, agency names are mapped through the agency groups
4. If dataset.json exists, account names are mapped through account aliases
5. Provisions with the same (mapped agency, normalized account) are matched

No other normalization happens. The tool does not silently rename agencies or merge accounts. If two provisions don’t match, they appear as orphans — and you can decide whether to add a group.

When normalization is applied, the compare output marks it:

Account                          Base ($)        Current ($)    Status
RDT&E, Army                      $17,115,037,000 $16,705,760,000 changed (normalized)
Tenant-Based Rental Assistance   $32,386,831,000 $38,438,557,000 changed

The (normalized) marker tells you this match used an agency group from dataset.json. Matches without the marker are exact. In CSV output, normalized is a separate true/false column rather than a status suffix.

Using –exact to Disable Normalization

congress-approp compare --exact --base-fy 2024 --current-fy 2026 --dir data

Ignores dataset.json entirely. Every match is exact lowercased strings only. Use this to see the raw matching results without any entity resolution applied.

When dataset.json Doesn’t Exist

The tool uses exact matching only. No implicit normalization. This is the default behavior — explicit and predictable. To create a dataset.json:

congress-approp normalize suggest-text-match --dir data
congress-approp normalize accept --auto --dir data

Viewing Current Rules

congress-approp normalize list --dir data

Displays all agency groups and account aliases currently in dataset.json.

Editing by Hand

You can edit dataset.json directly in any text editor. The format is simple JSON with two sections:

• agency_groups — each group has a canonical name and a list of members that should be treated as equivalent
• account_aliases — each alias has a canonical name and a list of alternative spellings

Typical Workflow

1. Run compare, notice orphan pairs in the output
2. Run normalize suggest-text-match to discover obvious naming variants
3. Review suggestions — check the hashes, evidence, and shared accounts
4. Accept the ones you trust: normalize accept HASH1 HASH2 --dir data
5. Re-run compare — orphans are now matched, marked (normalized)
6. For remaining ambiguous pairs, run normalize suggest-llm for LLM-assisted classification with XML evidence
7. Accept LLM suggestions the same way: normalize accept HASH --dir data

Tips

• Start with suggest-text-match. It finds the obvious pairs for free. Run suggest-llm only for the remaining ambiguous cases.
• Use --min-accounts 3 to focus on the strongest suggestions first — pairs sharing 3+ account names are very likely the same agency.
• Review every suggestion. Especially from the LLM. Check the reasoning.
• Verify merges. After accepting groups, re-run compare and check that the merged numbers make sense. If a merged amount looks too high, you may have grouped agencies that should be separate.
• One file per dataset. The dataset.json file is specific to the data directory it lives in. Different data directories can have different normalization rules.
• Version control it. If your data directory is in git, commit dataset.json alongside your bill data. It records the decisions you made about entity identity.
• Use --exact to verify. At any time, run compare --exact to see the raw matching results without normalization. This is your ground truth.