Stage 5 — How do we build a useful test set?

Separate examples used for development from the holdout set. The estimating lead prepares the reference register before the supplier sees the holdout outputs.

Include:

• Clean native PDFs and poor scans representative of actual work.
• Merged headers, footnotes and repeated tables.
• Blank quantities, explicit zero values and unreadable values.
• Multiple units and a deliberately conflicting capacity.
• Duplicate tags, missing pages and an equipment schedule referenced but not supplied.
• A changed equipment item and a deleted item in a revision.
• Scope located outside the schedule.
• A stale supplier offer or uncertain model match if pricing is in scope.

Set acceptance by consequence. An omitted controls obligation should be reviewed separately from punctuation in a description. Define what the system must flag, not only what it must extract. The NIST framework is a voluntary reference for risk assessment, not a substitute for these contractor-specific decisions.

Exit evidence: frozen case list, expected outcomes and signed acceptance criteria.