Founding release · v0.1.0 · Python 3.12
YY Dataset Preflight
Know what is in a dataset before it enters your pipeline.
One command turns a local directory into a deterministic review packet for inventory, duplicates, provenance coverage, secret-risk signals, and CI policy.
- Scan
- Review
- Decide
- Enforce
One-time founding price for one organization, including all 0.x updates.
Inspect before purchase
A real report from the released v0.1.0 wheel
These are the real, unedited outputs generated from a tiny synthetic six-file, 149-byte fixture. They demonstrate one exact duplicate, one narrowly normalized duplicate, one filename-only secret-risk signal, four intentionally uncovered files, no scan issues, and a passing explicit policy.
The manifest authenticates the JSON and HTML reports.
The sample shows the output contract and review experience. It is not customer data or a performance benchmark, and it does not prove that another dataset is safe, lawful, suitable for training, or able to pass the same policy.
Local by construction
Review the data without sending it anywhere
Dataset Preflight runs on your machine. The runtime has no dependencies, makes no network requests, performs no telemetry or update checks, and reports paths relative to the scanned root.
The dataset stays local.
Files are read to build the inventory and findings. Outputs are written only to the new local directory you choose. The report excludes absolute source paths, hostnames, usernames, timestamps, network state, and output paths.
yy-preflight scan C:\path\to\dataset --output C:\path\to\preflight-reportThe evidence packet
Three files, built for review and automation
report.json
Canonical, machine-readable inventory and findings with sorted keys and stable output bytes.
report.html
A standalone offline review generated from the same report object, with inline CSS and no JavaScript or external resources.
manifest.sha256
SHA-256 checksums for the JSON and HTML reports, ordered by filename.
Measured scope
Concrete findings before a pipeline run
The report states observable facts and bounded signals. Optional declarations and policies let your team add its own evidence and acceptance limits without changing scan findings.
Deterministic inventory
Records normalized relative paths, fixed local file types, byte sizes, and SHA-256 content hashes in a stable order.
Exact duplicates
Groups files with the same raw SHA-256 and reports excess-file counts you can threshold in CI.
Narrowly normalized duplicates
Finds strict UTF-8 text made identical only by a documented normalization of Unicode, newlines, trailing spaces and tabs, and outer blank lines.
Provenance coverage
Checks optional customer-authored declarations against file scopes and reports uncovered paths, conflicts, and whether named evidence files exist.
Secret-risk signals
Surfaces conservative filename and eligible-text signal codes without copying matched text, secret values, offsets, or line numbers into the report.
CI thresholds
Turns selected counts into stable exit codes for duplicate excess, uncovered files, secret-risk files, and scan issues.
Sensitive outputs
Keep the reports with the dataset
Reports intentionally contain filenames and content hashes. Treat them as sensitive operational artifacts even though secret values and matched text are never included.
Strict boundary
What this release does not claim
A preflight packet supports a human decision; it does not replace one. Secret-risk signals are conservative heuristics with false positives and false negatives.
- Not a security scanner, security audit, certification, or proof that a dataset is safe
- Not legal advice, license clearance, or a determination that data is lawful to use
- Not a determination that data is suitable or ready for model training
- Not semantic, fuzzy, or embedding-based near-duplicate detection
- Does not validate the meaning or legal sufficiency of provenance declarations
Founding release
$39 once
Licensed for internal use by one organization, including all 0.x updates. The package includes the installable Python wheel, customer documentation, example inputs, and an offline example workflow.
Support and future features are not promised by this founding release.