How we validate checks
Validation connects an expected decision to an actual source-to-CLI run in a recorded profile. A count of tests or a reference to a scientific paper is not a proof of the whole implementation.
What an example establishes#
A useful validation set includes a positive case, a control that removes the relevant witness, and an incomplete or unsupported case. Expectations are written independently of generated output. A check must retain unknown where its prerequisites are missing; it must not obtain a passing result by suppressing a reason.
Reproduced public examples#
The first-analysis packages were run at product 524fa07e0ae4153c84ac12ae9eb6559df89ec2be on Apple Swift 6.3.3, macOS arm64, on September 11, 2026.
| Case | Observed outcome | What it establishes |
|---|---|---|
| Positive | 2 findings, 2 admitted sources, degraded, exit 3 | Recognized detachment and guard-suspension witnesses in these files |
| Edited control | 0 findings, 2 admitted sources, degraded, exit 3 | Those findings disappear; remaining unknowns prevent a global negative claim |
| Unsupported input | 0 admitted sources, unavailable, exit 3 | The unsupported resource-processing input is refused rather than called clean |
The executable was built from the pinned source in a separate scratch directory. Each authored file is hashed with the internal replay inputs. Reports record actual status and process exits. The public manifest carries the corresponding summary; timings and snapshot IDs are not used as stable expectations.
Separate historical qualification#
The catalog retains 33 accepted bounded conditions out of 141 at historical revision 4558458d. This is a dated evidence record. It has not been inherited or requalified for the newer product snapshot by this documentation work. Invocation availability is independently extracted from the source-scan registration: 9 default and 2 opt-in conditions.
Limits of the measurement#
The public tutorial contains three small packages. It is not a representative benchmark of Swift applications and supplies no portfolio-wide precision, recall, false-positive rate, performance distribution or formal soundness proof. No such percentage is claimed here.
Repeat and challenge the result#
Run the downloaded examples, keep their source unchanged for comparison, and record compiler, product revision, command, coverage and exit status. For a mismatch, preserve the smallest reproducer and report it through the owner of your checkout. A new supported form or changed claim needs source evidence, an independent expectation and an updated documentation impact review.