How we prepare our data

What we check before publication

“Clean” needs a specific meaning for each product. The checks follow the table’s purpose and documented rules:

• Identifiers and row meaning. Required keys are present and unique at the declared level of detail. Identifiers remain text, with leading zeros and source punctuation preserved or transformations explained.
• Relationships and calculations. A classification split or a ZCTA crossing county lines remains a real relationship. Derived fields, such as land-area shares and the primary-county flag, have defined formulas and selection rules.
• Coverage and exclusions. We distinguish source records, delivered relationships and excluded or unresolved records. Product-specific outcome tables and quality reports explain the applicable gaps.
• The delivered files. We check schemas, row counts, keys and file hashes. Our public guide examples are also checked against the complete purchased files, so the samples demonstrate the actual product.

These checks establish whether our preparation follows its rules. Upstream errors and gaps can still exist; each product page describes the source coverage and limits that affect its use.

Source editions and processing revisions

A dataset can change because its publisher supplied new information, because we improved our processing, or both. We record those changes separately.

Source edition

The publisher’s release, classification year or reference period. A product combining several sources records each source separately. Missing version or retrieval information stays unknown.

ACD processing revision

The extraction and preparation rules we used. Recorded code, settings and runtime information let us identify changes to those rules.

Release identity

The particular prepared output, with its parent inputs and processing records. Changed sources or changed processing can each produce a new release; previous artifacts are retained.

Production date

When we built the package. A new build date alone does not establish a new source edition or a reason to charge again.

Provenance records where the inputs came from and what versions they represent. Lineage follows those inputs through extraction and processing into a release. Together they explain both the origin of the data and our work on it.

Reprocessing an edition you already bought

For example, suppose we improve a crosswalk’s matching logic after you purchase it. We can run the improved logic on the saved source edition you bought, producing a new release without substituting newer source data.

Both are versions of the dataset. Your right to the improvement is a separate question: corrections and processing improvements to your purchased source edition are free. A genuinely new publisher edition may be offered as an optional separate purchase. An upstream correction or a new retrieval date alone is not a new paid edition.

If we change the sources and our processing together, we provide the processing improvement for the purchased edition, or give affected buyers the combined release free. Our update policy explains access and notification arrangements.

Comparisons hold one thing fixed

• Source changes: compare source editions using the same processing.
• Processing changes: compare processing revisions using the same saved inputs.
• Mixed changes: label the comparison mixed and explain what cannot be attributed to either change alone.

For a published example, our October 13 revision note describes stronger processing records and documentation with identical dataset rows. The earlier March–October NDC → ATC comparison combines source and method changes and is labeled accordingly.

What you can inspect in a download

The product page lists the files included in its ZIP. Across our current products, the supporting records cover:

Fields and usage

A field dictionary and README explain identifiers, the meaning of each row, joining rules and limitations. Product guides provide runnable examples with free samples.

Sources and processing

Metadata and provenance records identify the retained inputs and preparation. You can inspect a public guide’s provenance record before buying.

Changes and coverage

Revision records explain the delivered update. Depending on the product, change summaries, lookup outcomes or quality reports provide the relevant counts and exceptions.

File integrity

SHA-256 checksums identify file contents. A matching checksum confirms that you have the expected bytes; it does not independently prove the source data is correct.

Purchase access is emailed after payment, and Find my downloads can recover a lost link. Material processing improvements call for a service notice to affected buyers under our update policy; those revision notices currently require manual handling. Automatic revision emails are not yet implemented.