Tool Confidence Level Assessment
ISO 26262-8:2018 Clause 11.4
Tool Identification
| Field | Value |
|---|---|
| Tool Name | Provadyne |
| Version | 0.1.0 |
| Purpose | MISRA C:2023 / CERT C / CWE static analysis overlay + test stub generation |
| Supplier | — |
| Date of Assessment | 2026-05-03 |
Tool Impact (TI)
TI2 — Tool output may fail to detect safety-relevant defects without direct indication of failure.
Rationale: The tool performs static analysis on source code. A false negative (missed violation) would not itself introduce a fault, but could allow an existing violation to go undetected. The user is responsible for reviewing all tool output via the integrated diff viewer before accepting any suggested fix.
Tool Error Detection (TD)
TD2 — User reviews all tool output via diff viewer before application.
Rationale: - All fix suggestions are presented as diffs and require explicit user acceptance - No automated code modification is performed without user confirmation - The tool does not integrate with build pipelines without user-configured CI hooks
Resulting TCL
TCL2 — Increased confidence from use required.
Per ISO 26262-8:2018 Table 4 (TI2 × TD2 → TCL2), the following confidence measures shall be applied.
Required Confidence Measures (TCL2)
| Measure | Method | Status |
|---|---|---|
| Evaluation of the tool (11.4.6) | Tool validation suite with known-violation fixtures | Implemented |
| Increased confidence from use (11.4.7) | CI gate with 60% coverage threshold | Implemented |
| Development in accordance with a safety standard | — | Not applicable (COTS tool) |
Validation Methods
- Golden test suite — fixture C files with known MISRA violations; expected violations are asserted in CI
- cppcheck reference comparison — output compared against cppcheck
--addon=misrareference - False positive / false negative rate tracking — logged per rule per release
- CI pipeline — 60% code coverage gate on daemon (Go) and 80% on engine (Python)
Test Results
| Metric | Target | Actual |
|---|---|---|
| Test fixtures | ≥5 | 5 (4 violation + 1 clean) |
| Known violations covered | 100% | 100% (4/4 rules) |
| False positive rate | <5% | 0% (clean_code.c verified) |
| False negative rate | 0% | 0% (all fixture violations detected) |
| Code coverage (Go) | ≥60% | 60.7% (core packages) |
| Code coverage (Python) | ≥80% | ~85% (62 test cases pass) |
| Python test cases | — | 62 (5+7+40+10) |
| Go test cases | — | 93 |
| Cross-platform builds | 4 | 4 (darwin-arm64/amd64, linux-amd64, windows-amd64) |
Note: Go coverage measured across internal packages (api, config, git, license, updater). Main entry points (cmd/server, cmd/nativehost) are excluded from the coverage gate as they have no unit-testable logic paths.
Backend Tools and Their Qualification
Provadyne does not perform analysis itself; it dispatches to one of three
external analyzers depending on source language (see engine/backends/) and
normalizes their output. Each underlying tool is a separate program invoked at
arm's length (subprocess), installed and version-managed by the integrator. The
TCL assessment of each underlying tool is the responsibility of the integrator;
the table below records the version pin and the standards Provadyne maps from
each, so that drift can be detected.
| Backend | Tool | Version pin | Standards mapped | Confidence basis |
|---|---|---|---|---|
| C | cppcheck (+ misra.py) |
2.20.0 | MISRA C:2023, CERT C (subset), CWE | Golden fixtures + cppcheck --errorlist ID validation; runtime version mismatch emits a ::warning::. Cppcheck Premium offers a TÜV SÜD ISO 26262 qualification kit for the commercialization path. |
| C++ | clang-tidy (LLVM) | major 18 | CERT C++ (cert-*, mechanically derived), C++ Core Guidelines (cppcoreguidelines-*) |
Check IDs mapped only where the tool's own check name maps deterministically; MISRA C++ is not asserted (commercial scope). |
| Rust | clippy | 0.1.x (tolerant) | clippy lints | lint→CWE map seeded empty; only verified correspondences are added. |
Mapping discipline (all backends): a rule/check is mapped only where the correspondence is verified. Unmapped findings pass through unmapped rather than being guessed — "unmapped beats guessed". This keeps the standards-coverage claims honest and avoids false confidence in the safety mapping.
Limitations and Assumptions
- The primary qualified scope is C: MISRA C:2023, SEI CERT C (a mapped subset), and CWE. C++ (CERT C++ / C++ Core Guidelines via clang-tidy) and Rust (clippy lints) backends are available as Pro features but carry lighter qualification — they are validated against canned/e2e tool output rather than a full golden-fixture suite, and MISRA C++:2023 / AUTOSAR C++ are not claimed.
- The underlying static analysis engines are
cppcheck(C),clang-tidy(C++), andclippy(Rust). Each is invoked at arm's length; its own TCL assessment is the responsibility of the integrator. See the Backend Tools table above. - AI-generated fix suggestions are informational only and do not constitute certified outputs.
- Air-gapped (offline) operation is fully supported; no data leaves the local machine in local LLM mode.