Skip to content

ADR-026: Advisory CI Vulnerability Scanning

Status

accepted

Date

2026-05-09

Context

APTL already runs a single GitHub Actions Checks workflow for quality and security feedback. The workflow includes pre-commit, Python tests, MCP and web vitest coverage, an advisory dependency audit, and advisory SonarCloud analysis.

Issue #278 adds Trivy and OSV-scanner coverage:

  • Trivy for lab container images under web/ and containers/, plus filesystem and IaC scanning where relevant.
  • OSV-scanner for the root uv.lock dependency graph.
  • SARIF, JSON, or equivalent scan artifacts, with advisory behavior.

The lab intentionally contains vulnerable targets and research tooling. Scanner findings are therefore signal for triage, not an automatic release or merge gate.

Decision

Add Trivy and OSV-scanner as advisory CI scanners in the existing GitHub Actions checks surface. They extend the current CI security feedback loop; they do not introduce a new release gate, runtime security service, policy engine, or lab configuration schema.

Boundaries

  • Keep scanner jobs in the existing CI workflow unless there is a concrete operational reason to split workflows.
  • Preserve advisory semantics with explicit non-failing behavior (--exit-code 0, continue-on-error: true, or the scanner's equivalent).
  • Publish scan output as short-retention GitHub Actions artifacts, consistent with existing coverage artifacts.
  • Keep GitHub token permissions minimal. Do not add security-events: write unless the implementation intentionally uploads SARIF to GitHub code scanning, not merely as an artifact.
  • Treat OSV-scanner as the authoritative check for uv.lock; do not duplicate a second Python lockfile parser in scripts.
  • Treat Trivy as the authoritative scanner for container images and filesystem/IaC checks; do not add custom Dockerfile, Compose, or Terraform parsing logic.
  • Do not rebuild and scan the Kali image on every PR or protected-branch push. The Kali image is intentionally noisy because it is a rolling security-tooling distribution; scan it when files under containers/kali/ change, or when the workflow itself changes.

Consequences

Positive

  • Container image, filesystem, IaC, and Python lockfile vulnerability feedback is visible on every PR and protected-branch push.
  • Advisory behavior matches SonarCloud and existing dependency-audit semantics, avoiding false merge blockers from intentionally vulnerable lab assets.
  • Scanner output is retained as CI evidence without adding repo-local state.

Negative

  • CI time and network dependency increase because images and vulnerability databases must be built or fetched.
  • Advisory scans can be ignored unless maintainers actively review artifacts and job summaries.

Risks

  • Trivy image scans can become expensive if every Dockerfile is built independently without cache-aware job structure.
  • The Kali image can dominate scan noise because most findings come from the upstream tool distribution rather than APTL-owned code. Findings should be triaged when APTL changes the Kali image, exposes host/operator data, weakens lab isolation, or adds an owned package with a practical update path.
  • Scanning built images, filesystems, and IaC are related but distinct concerns; conflating them can hide coverage gaps.
  • SARIF upload to GitHub code scanning changes the permission and visibility model. Artifact-only SARIF does not require that broader permission.
  • The existing dependency-audit job already runs pip-audit and npm audit. OSV-scanner should complement that job for uv.lock, not be wired as a replacement for all ecosystem audits unless that is decided separately.

Amendment: OpenSSF Scorecard and dependency pinning (issue #847)

Issue #847 adds OpenSSF Scorecard. Two parts of this ADR need qualifying.

Scorecard runs in its own workflow

The boundary above says to keep scanner jobs in the existing Checks workflow "unless there is a concrete operational reason to split workflows." Scorecard is that concrete reason, and the only one so far.

Publishing a result with publish_results: true subjects the job to Scorecard's own publisher validation, which rejects workflow- or job-level env and defaults, any workflow-level write permission, non-Ubuntu runners, and actions outside its approved set. checks.yml violates several of those by design. Folding the publisher into it would make a fragile contract that breaks the next time an unrelated job needs an env block. The publisher also needs id-token: write, which must not sit at workflow scope where every other job would inherit it.

.github/workflows/scorecard.yml therefore runs alone, triggered only by pushes to main and a weekly schedule.

Scorecard is advisory, and its score is not a vulnerability count

Scorecard is a repository and supply chain posture assessment. It does not replace Trivy, OSV-scanner, pip-audit, or npm audit, which remain the authoritative scanners for their surfaces. Its aggregate score is not a merge gate and must not become a required pull-request check: the workflow runs on the default branch and cannot reliably produce a pull-request status.

Findings are triaged per check, at the detail level, never from the aggregate score. A check that describes a governance or process fact (Contributors, Code-Review, CII-Best-Practices, Branch-Protection) is not a vulnerability and is not remediated by editing a checked-in file to look compliant. In particular, .github/branch-protection-baseline.json documents intent; the live GitHub repository rules are authoritative, and editing the JSON is not remediation.

Dependencies are pinned by content, not by label

security-events: write is requested for the Scorecard job specifically because that job intentionally uploads SARIF to GitHub Code Scanning, which is the exception this ADR's boundary already contemplated.

Beyond the scanners, #847 makes the pinning itself an invariant:

  • Container base images are pinned by digest, with Dependabot docker entries so the digests are refreshed rather than frozen.
  • Node installs use npm ci, which honors the lockfile and fails on drift.
  • Python installs use pip install --require-hashes -r requirements/<x>.txt. Those exports are generated from uv.lock by astral-sh/uv-pre-commit hooks, never by hand, and the required "Pre-commit hooks" job re-runs them on every pull request, so a stale export fails a required check.
  • Release artifacts carry a SLSA build-provenance attestation, and the bundle is attached to the GitHub release so a downloaded artifact can be verified.

tests/test_supply_chain_pinning.py and tests/test_hashed_requirements.py enforce these as structural gates, so the properties survive without depending on a reviewer noticing.

Two pip invocations are deliberately not hash-pinned, each for a stated reason rather than convenience:

  • appliance/guest/provision-offline.sh installs with --no-index from a staged wheelhouse inside a content-identified payload, at an exact == version. No index is contacted, so there is no mutable upstream for a hash to protect. The test asserts --no-index and the version pin, so removing either fails rather than silently reopening a network fetch.
  • scenarios/fixtures/techvault-content/.../deploy.sh is fake corporate content a red-team agent discovers on a compromised host. It is never executed by a build. Hash-pinning it would make the prop read as machine-generated and degrade the scenario, which is the product. Per this ADR's existing guidance, a designed lab weakness is preserved while real supply chain components are repaired.

Amendment (2026-09-20): Required platform dependency audit (issue #969)

The advisory decision above continues to govern the mixed target, image, filesystem, IaC, and OSV scanner jobs. The existing Dependency vulnerability scan job now has a narrower blocking duty for the platform that handles operator credentials and host resources. It audits the generated, hash-pinned CLI/runtime, API/web, and CI/build Python exports directly, plus production dependencies in every MCP package and the web control plane. Python known vulnerabilities and high or critical npm findings fail the job. Lock or scanner failures also fail it. The job is a required dev pull-request context.

Intentionally vulnerable lab targets remain outside that blocking dependency surface and retain advisory scanner feedback. A target's teaching purpose does not exempt a dependency in the CLI, API, web control plane, MCP broker, or build path. There are no active platform vulnerability exceptions. Any temporary exception must sit next to its scanner command or affected manifest and name the advisory, dependency and version, affected APTL component, concrete reason, owner, and removal condition. It must not suppress an entire package family, lower the global threshold, or turn scanner failure into success.

The checked-in branch-protection baseline records the required context names; the live GitHub dev branch settings enforce them. The bounded installed-wheel smoke and the profile it supports are described in platform-pr-gates.md.