{"schemaVersion":"2.0.0","locale":"en","title":"How we validate checks","description":"Validation connects an expected decision to an actual source-to-CLI run in a recorded profile. A count of tests or a reference to a scientific paper is not a proof of the whole imp","htmlUrl":"https://docs.swift-analyzer.com/concepts/validation/","markdownUrl":"https://docs.swift-analyzer.com/concepts/validation/index.md","productRevision":"524fa07e0ae4153c84ac12ae9eb6559df89ec2be","catalogRevision":"4558458d","docsRevision":"1f17ba58ab08627e0de898f1442067b09e960b48cd5605d6529e9275f8930d68","channel":"pre-prod","maturity":"experimental","access":"owner-provided-checkout","markdown":"# How we validate checks\n\nValidation connects an expected decision to an actual source-to-CLI run in a recorded profile. A count of tests or a reference to a scientific paper is not a proof of the whole implementation.\n\n## What an example establishes\n\nA useful validation set includes a positive case, a control that removes the relevant witness, and an incomplete or unsupported case. Expectations are written independently of generated output. A check must retain unknown where its prerequisites are missing; it must not obtain a passing result by suppressing a reason.\n\n## Reproduced public examples\n\nThe [first-analysis packages](https://docs.swift-analyzer.com/get-started/first-analysis/) were run at product `524fa07e0ae4153c84ac12ae9eb6559df89ec2be` on Apple Swift 6.3.3, macOS arm64, on September 11, 2026.\n\n| Case | Observed outcome | What it establishes |\n| --- | --- | --- |\n| Positive | 2 findings, 2 admitted sources, degraded, exit 3 | Recognized detachment and guard-suspension witnesses in these files |\n| Edited control | 0 findings, 2 admitted sources, degraded, exit 3 | Those findings disappear; remaining unknowns prevent a global negative claim |\n| Unsupported input | 0 admitted sources, unavailable, exit 3 | The unsupported resource-processing input is refused rather than called clean |\n\nThe executable was built from the pinned source in a separate scratch directory. Each authored file is hashed with the internal replay inputs. Reports record actual status and process exits. The public manifest carries the corresponding summary; timings and snapshot IDs are not used as stable expectations.\n\n## Separate historical qualification\n\nThe catalog retains 33 accepted bounded conditions out of 141 at historical revision 4558458d. This is a dated evidence record. It has not been inherited or requalified for the newer product snapshot by this documentation work. Invocation availability is independently extracted from the source-scan registration: 9 default and 2 opt-in conditions.\n\n## Limits of the measurement\n\nThe public tutorial contains three small packages. It is not a representative benchmark of Swift applications and supplies no portfolio-wide precision, recall, false-positive rate, performance distribution or formal soundness proof. No such percentage is claimed here.\n\n## Repeat and challenge the result\n\nRun the downloaded examples, keep their source unchanged for comparison, and record compiler, product revision, command, coverage and exit status. For a mismatch, preserve the smallest reproducer and report it through the owner of your checkout. A new supported form or changed claim needs source evidence, an independent expectation and an updated documentation impact review.\n","anchors":["how-we-validate-checks","what-an-example-establishes","reproduced-public-examples","separate-historical-qualification","limits-of-the-measurement","repeat-and-challenge-the-result"]}