Secure Development at Scale experimental

Secure Development at Scale Security that survives a thousand applications, disciplined by the Common Criteria Threat modelling. STRIDE, attack trees, data-flow diagrams and trust boundaries, producing a disciplined security problem definition.. Build a data-flow diagram and mark its trust boundaries.. Identify threats with STRIDE and structure them as attack trees.. Turn a threat model into a security problem definition (threats, assumptions, policies).. Data-flow diagrams and trust boundaries, STRIDE, Attack trees, The security problem definition Threat modelling Threat modelling turns a vague sense of 'we should be secure' into a disciplined list of what could go wrong, where, and why it matters. At Level 3 the goal is not a brainstorm but an artefact: a diagram, a threat list, and a written problem definition that downstream design and testing can be traced back to. Data-flow diagrams and trust boundaries A data-flow diagram (DFD) models a system as external entities, processes, data stores, and the data flows between them. Drawing one forces you to be explicit about what talks to what. The single most valuable line you add to a DFD is the trust boundary: a boundary marking where data crosses between elements operating at different levels of trust or privilege, for example between an anonymous internet user and an authenticated internal service, or between a third-party vendor connection and a payment network. The 2013 Target breach happened because attackers entered through a Heating, Ventilation, and Air Conditioning (HVAC) vendor's remote-access credentials and reached the cardholder environment: the trust boundary that should have separated vendor access from payment systems was never enforced. Every trust boundary on the diagram is a place demanding its own authentication, authorization, and validation controls. In practice, a DFD level 0 diagram (the context diagram) shows the system as a single process against its external entities, and successive levels decompose that process into its constituent parts only as far as the analysis needs; over-decomposing early wastes effort on components that turn out to carry no trust boundary at all. The most common omission is not drawing a boundary at all around a third-party integration that a team treats as 'internal' simply because it has been in production for years; age is not evidence of trust, and every external dependency should cross a boundary on the diagram regardless of how long it has been relied upon. STRIDE Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, and Elevation of privilege (STRIDE) is a mnemonic for six threat categories applied per element of the DFD, each named for the security property it violates: Spoofing (violates authentication), Tampering (violates integrity), Repudiation (violates non-repudiation), Information disclosure (violates confidentiality), Denial of service (violates availability), and Elevation of privilege (violates authorization). Working through STRIDE systematically for each process, data store, and flow, rather than waiting for inspiration, is what makes threat modelling repeatable and auditable rather than a matter of luck. Applying STRIDE per element rather than per system is what keeps the exercise from collapsing into a single generic list; a data store, for example, is naturally exposed to Tampering and Information disclosure but rarely to Spoofing in the same direct sense a process is, so working element by element surfaces threats a system-level pass would miss. Teams under time pressure tend to skip categories that feel implausible for their domain, commonly Repudiation for internal-only systems, but the categories that feel unlikely are frequently the ones an attacker who has already read the same threat model will target precisely because the defender assumed they could be skipped. Attack trees An attack tree structures threats hierarchically: the root node is the attacker's goal, and child nodes are the steps that achieve it. Children can be connected by OR (any one child suffices) or AND (all children are required). Attack trees are useful once STRIDE has produced a long flat list of threats, because they let you see which threats chain together into a realistic path to compromise, and which single control, placed correctly, breaks multiple paths at once. Assigning a rough cost or likelihood to each leaf node, even an informal low, medium or high estimate, turns an attack tree from a diagram into a decision tool: it lets you identify the cheapest path to the attacker's goal rather than the most dramatic one, and defenders who fixate on the most technically interesting path while ignoring a cheap OR-branch elsewhere in the tree are optimising for the wrong threat. Because OR nodes only need one child to succeed, a single overlooked leaf can undo the value of hardening every other branch, which is why attack trees reward completeness over depth on any one path. The security problem definition The output of threat modelling is a security problem definition: a structured statement of threats, assumptions about the operational environment, and organisational security policies the system must satisfy. It deliberately excludes solution mechanisms and design choices, which come later, once the problem is fixed. If an assumption cannot actually hold in a given deployment, every requirement resting on it loses its justification there, so assumptions must be stated precisely enough to be checked, not asserted as comforting background. A security problem definition is a living artefact, not a one-time deliverable: it should be versioned alongside the system it describes and revisited whenever the system's trust boundaries, deployment environment, or assumptions change, since a definition that is accurate at design time but never revisited silently drifts out of sync with what is actually running. This document is also the direct input to the next module's Security Target, so imprecision here, an assumption stated too vaguely to check or a threat left implicit rather than enumerated, propagates forward into every security objective and requirement derived from it. Related CCI capabilities Cambridge Cyber International's Global Cyber Incident Database (GCIDB) is the kind of documented incident record this module argues a threat model should be checked against: real trust-boundary failures, like the HVAC-vendor path into the Target cardholder environment, are exactly what a data-flow diagram exercise is meant to anticipate before the fact rather than explain after it. See cambridgecyberinternational.com for current coverage. Security Targets and Protection Profiles. The Target of Evaluation, security objectives, deriving Security Functional Requirements, and reading a real Protection Profile.. Define the TOE and its security objectives.. Derive Security Functional Requirements from objectives.. Read a Protection Profile and explain how it drives requirements.. The Target of Evaluation, Security objectives, Deriving SFRs, Reading a Protection Profile Security Targets and Protection Profiles Once a security problem is defined, the Common Criteria (CC) give it a formal home: the Security Target (ST). This lesson connects the threat-modelling work to the vocabulary evaluators, procurement teams, and certification bodies actually use. The Target of Evaluation The Target of Evaluation (TOE) is the part of the product, together with its guidance documentation, that is actually subject to evaluation. It is not 'the whole company' or even the whole product; it is a scoped boundary, and everything inside that boundary must be described precisely enough that an evaluator can test claims against it. Getting the TOE boundary wrong, too broad or too narrow, undermines every claim made afterward. Drawing the TOE boundary too broadly is just as damaging as drawing it too narrowly: claiming the whole company as the TOE invites an evaluator to demand evidence for organisational controls that have nothing to do with the product, while claiming too little lets a genuinely relevant component, an authentication library the product depends on but does not itself implement, escape scrutiny entirely. A useful discipline is to draw the boundary around exactly what the vendor controls and can make binding claims about, and to describe everything outside it as part of the operational environment instead. Security objectives Security objectives translate the security problem definition into statements of what must be achieved, split between objectives for the TOE itself and objectives for the operational environment, things the environment must provide because the TOE cannot. An ST must show, explicitly, that its objectives answer every threat, assumption, and policy in the problem definition; this traceability is the rationale a reviewer checks first. A common error is writing an objective that is really a requirement in disguise, for example 'the TOE shall use Advanced Encryption Standard (AES) with a 256-bit key', which forecloses design choices an SFR should instead express as a testable requirement derived from a more general objective such as 'the TOE shall protect data confidentiality in transit'. Objectives for the operational environment matter as much as objectives for the TOE itself: a TOE that assumes 'the operating system enforces process isolation' is only as sound as that assumption, and if the environment cannot actually guarantee it, the objective, and everything built on it, fails silently in deployment. Deriving SFRs Security Functional Requirements (SFRs) are drawn, generally from the standard catalogue in CC Part 2, to satisfy the security objectives. Common classes include identification and authentication (FIA), cryptographic support (FCS), and user data protection (FDP). Each SFR must trace back to an objective, and each objective must trace back to the problem definition; a security objective with no supporting SFR is a traceability gap, not a stylistic choice, and an evaluator will record it as a failure to demonstrate the objective is met. Beyond FIA, FCS, and FDP, the CC Part 2 catalogue includes classes such as security audit (FAU) and security management (FMT), and a mature ST typically draws from several classes because a single mechanism, a login screen, say, usually implicates identification and authentication, audit logging of the attempt, and management of the account state all at once. Selecting SFRs is not simply picking every class that sounds relevant; each selected component should trace to a specific objective, and a reviewer checking the rationale table will treat an SFR with no supporting objective as unjustified scope, not as thoroughness. Reading a Protection Profile A Protection Profile (PP) states reusable, implementation-independent security requirements for a whole product class, for example firewalls or smartcards, so that competing products can be measured against one baseline. An ST can claim conformance to a PP, meaning it satisfies the PP's requirements; procurement teams favour PP-conformant products precisely because it lets them compare vendors on a common yardstick rather than reading each vendor's bespoke claims. Certification, including PP conformance, attests conformance to the claimed requirements at evaluation time; it is not a guarantee of the absence of every vulnerability, as the 2017 Return of Coppersmith's Attack (ROCA) flaw in evaluated smartcards and Trusted Platform Modules (TPMs) showed. Reading a real PP well means checking that its objectives, SFRs, and rationale form one continuous chain from threat to requirement, with no unsupported leaps. Because a PP is implementation-independent, two vendors can both claim conformance while making very different engineering choices underneath, which is precisely the point: procurement teams compare the claims, not the implementations, and the PP is what makes those claims commensurable across vendors. The 2017 ROCA flaw is worth returning to here because the affected smartcards had been evaluated against a PP and certified before the flaw was found; certification narrows the space of plausible vulnerabilities by forcing rigor into the development and evaluation process, but it evaluates conformance to stated requirements at a point in time, not the permanent absence of unknown weaknesses. Related CCI capabilities Cambridge Cyber International's EviGen offering supports exactly the evidence discipline a Security Target rationale demands: automated, dated technical evidence collection that lets a claim of conformance be checked against artefacts rather than taken on trust. See cambridgecyberinternational.com for current availability. Secure SDLC and the assurance life cycle. Requirements to design to build to verify to release, mapped onto the Common Criteria life-cycle support class (ALC).. Model the phases of a secure SDLC onto the CC life-cycle class.. Set up configuration management and a flaw-remediation policy.. Explain development security as an ALC concern.. The secure SDLC, Configuration management (ALC_CMC, ALC_CMS), Delivery and development security (ALC_DEL, ALC_DVS), Flaw remediation (ALC_FLR) Secure SDLC and the assurance life cycle Security requirements and a Security Target mean little unless the process that builds the product is itself trustworthy. The Common Criteria capture this as the life-cycle support (ALC) class; NIST's Special Publication (SP) 800-218 Secure Software Development Framework (SSDF) offers a practice catalogue for the same territory that any organisation can adopt without pursuing formal certification. The secure SDLC A secure SDLC threads security through every phase: security requirements alongside functional ones, threat-modelled design, secure coding and code review during build, independent verification, and a release-and-response phase that keeps monitoring the product after ship. Skipping a phase does not remove its risks, it just defers discovery of them to production, usually at higher cost. Mapping a secure SDLC onto a specific process model, waterfall, agile sprints, or continuous delivery, changes the cadence of each phase but not its necessity: a two-week sprint still needs threat-modelled design and code review before merge, it simply needs to happen inside that shorter cycle rather than being deferred to a separate hardening phase at the end, which is the pattern most likely to see it dropped entirely under deadline pressure. Configuration management (ALC_CMC, ALC_CMS) ALC_CMC and ALC_CMS require that everything which produces the shipped artefact, source, build tools, and third-party components, is tracked under configuration management (CM), not just first-party code. What ships is the product of the whole toolchain, and an untracked tool or dependency can taint the artefact, as the 2020 SolarWinds SUNBURST compromise demonstrated when malicious code entered through the build system itself. When engineers bypass configuration management, for example hotfixing production directly, the CM records no longer describe what is actually running, and every claim resting on those records fails. CM scope extends further than most teams initially assume: build scripts, Continuous Integration (CI) pipeline definitions, container base images, and even the compiler and toolchain versions used to produce the artefact all belong under configuration management, because any of them can silently change the artefact's behaviour or introduce a vulnerability without a single line of first-party source code changing. A CM system is only as trustworthy as its own integrity, so it needs the same tamper-evidence and access control expected of the artefacts it tracks, otherwise an attacker who can rewrite CM history can also rewrite the story of what actually shipped. Delivery and development security (ALC_DEL, ALC_DVS) ALC_DEL protects the channel between developer and consumer, because an artefact can be substituted or tampered with in transit unless delivery preserves integrity end to end. ALC_DVS covers the physical, procedural, and personnel controls protecting the development environment itself, recognising that a compromised build environment can undermine every other control regardless of how good the code is. ALC_DEL and ALC_DVS address two different attackers: ALC_DEL assumes the artefact is correct and asks whether it can be substituted or corrupted after it leaves the vendor, which is why signed releases and verified download channels matter even when the build itself was clean, while ALC_DVS assumes the artefact might be corrupted before it ever leaves the vendor, which is why physical access control, background-checked personnel, and monitored build infrastructure matter even when delivery is perfectly secure. Treating either as sufficient on its own leaves the other half of the pipeline unprotected. Flaw remediation (ALC_FLR) ALC_FLR requires a working process to receive, track, correct, and communicate fixes for security flaws discovered after release: an intake channel, triage and severity assignment, and distribution of fixes and advisories to affected users. When Heartbleed (CVE-2014-0160) was disclosed in 2014, every vendor embedding OpenSSL had to exercise exactly this discipline at speed. A sound flaw-remediation policy is judged by whether it actually closes that loop, not by whether flaws exist, since flaws are inevitable and the process, not their absence, is what assurance rests on. A flaw-remediation policy is judged in practice by three concrete numbers: how long from intake to acknowledgement, how long from acknowledgement to a fix, and how reliably affected users are actually notified, and an organisation that can state all three with evidence is demonstrating assurance in a way that a policy document alone cannot. When Heartbleed, tracked under the Common Vulnerabilities and Exposures (CVE) identifier CVE-2014-0160, was disclosed in 2014, vendor responses varied enormously along exactly these three axes; the difference was not the severity of the underlying flaw, which was identical for everyone, but the maturity of each vendor's ALC_FLR process. Related CCI capabilities Cambridge Cyber International's Secure SDLC at Scale offering operationalises this module's discipline directly: an on-premises secure software development lifecycle (SDLC) platform run jointly with CCI engineers for estates of the thousand-application order, rather than a policy document teams are expected to self-implement. See cambridgecyberinternational.com for current availability. Supply-chain integrity. Software bill of materials, provenance and signed artefacts, reproducible builds and dependency risk.. Generate and read a software bill of materials.. Explain and verify a signed artefact's provenance.. Assess dependency risk and the value of reproducible builds.. Software bill of materials, Provenance and signing, Reproducible builds, Dependency risk Supply-chain integrity Modern software is assembled, not written from scratch, and every assembled component is a potential entry point. This lesson covers the four controls that let you know what you shipped, prove where it came from, and limit what an attacker upstream can do to you. Software bill of materials A software bill of materials (SBOM) is a structured inventory of every component in a build and its version. Its value is answering one question fast: which of our systems contain the affected component and version. During the log4shell incident in December 2021, organisations holding current SBOMs scoped their exposure in hours; those without one spent days just enumerating what they ran. SBOMs also feed licence compliance and incident-response scoping, but vulnerability matching is the use case that pays for itself first. An SBOM's usefulness depends heavily on its depth: a shallow SBOM listing only direct dependencies misses the transitive dependencies several layers down that most vulnerabilities actually live in, so a format capable of expressing the full dependency graph, not merely a flat list of direct dependencies, and a generation process that walks it completely, matters more than simply having an SBOM at all. An SBOM generated once at release and never regenerated also drifts out of date the moment a dependency is patched in place, which is why SBOM generation belongs in the build pipeline itself rather than as a manual, occasional exercise. Provenance and signing A cryptographic signature proves integrity from the signing point onward: the artefact you have matches what was signed. It says nothing about whether the build that produced the artefact was clean. SolarWinds SUNBURST artefacts carried valid vendor signatures, because the malicious code was injected before signing, inside the build system. Provenance, in the Supply-chain Levels for Software Artifacts (SLSA) sense, closes that gap by recording the builder's identity, the input materials and their digests, and the build recipe, so a consumer can check not just 'is this signed' but 'was this built the way it claims to have been built, from the inputs it claims'. SLSA's provenance framework is deliberately staged into levels, because demanding the highest level immediately from every project is unrealistic; a project can start by simply recording who built an artefact and from what source, then progressively add hermetic, reproducible builds and two-party review as maturity allows. What distinguishes provenance from a plain signature is that provenance is a claim about the process, not just the output, so a consumer verifying it is really asking 'do I trust this build pipeline', not merely 'do I trust this signing key'. Reproducible builds A build is reproducible when the same source and toolchain produce a bit-identical artefact, letting an independent party verify that the recorded source really produces the shipped artefact. This is the strongest defence against attacks like the 2024 xz-utils backdoor, catalogued under the Common Vulnerabilities and Exposures (CVE) system as CVE-2024-3094, where the released tarball differed from the public source repository: reproducibility lets anyone rebuild from source and diff the result against what was shipped. Achieving reproducibility in practice usually means eliminating sources of nondeterminism that seem harmless in isolation: embedded timestamps, filesystem path strings baked into binaries, non-deterministic map or set iteration order in the build tooling, and even compiler optimisation choices that vary by build-machine architecture. Projects that have pursued reproducibility systematically, Debian's reproducible-builds initiative being the most visible example, report that most of the effort is in identifying and removing this class of accidental nondeterminism rather than in any single large architectural change. Dependency risk Dependencies bring transitive trust: the 2018 event-stream incident showed a maintainer handoff introducing a malicious transitive dependency, and dependency confusion (Birsan, 2021) showed resolvers preferring a higher-version public package over an internal one of the same name. A version string alone is a mutable pointer; binding resolution to a cryptographic digest in a lockfile, reserving internal namespaces, and pinning or vendoring critical dependencies (the left-pad lesson, 2016) are what convert 'we depend on this' into 'we know exactly what this is'. Verifying fetched build tooling before executing it, as the 2021 Codecov compromise underlines, extends the same discipline to the pipeline itself. Lockfiles alone are not sufficient if the lockfile itself can be silently regenerated against a different, compromised version during the build; the digest pinning has to be enforced by the build tooling, not merely recorded for human review. Dependency confusion in particular exploits a resolution ambiguity rather than a code flaw: nothing in the compromised package needs to be technically sophisticated if the resolver can simply be tricked into preferring it, which is why namespace reservation and explicit registry scoping close the gap that pinning alone does not. Related CCI capabilities Cambridge Cyber International's Mirage mediation bastion reduces the supply-chain trust this module is concerned with by design: a formally specified mediation point stands between an operator and the target system, so credential custody and direct system contact never pass through a third party the way a compromised dependency or build tool otherwise could. See cambridgecyberinternational.com for current status. Policy as code. Encoding security and compliance rules as enforceable, versioned checks that run in the pipeline.. Explain how to express a security rule as code.. Run a policy as a gate that blocks a non-compliant change.. Version and test policies like any other code.. Why policy as code, Writing an enforceable rule, Gating a change, Versioning and testing policy Policy as code A security rule written in a document is a suggestion; a security rule expressed as code that runs automatically is a control. Policy as code converts standards into enforceable, versioned, testable checks that fire at the same points as every other pipeline gate. Why policy as code Repeated cloud-storage exposures (Accenture 2017, Booz Allen 2017, and many since) show that manual review does not scale to thousands of resources; the durable fix is machine-checked configuration policy enforced at provisioning. A policy engine turns 'reviewers should catch this' into 'the system cannot produce this state', which is the only approach that holds as the number of services grows into the hundreds or thousands. The underlying argument for policy as code is a scaling one: a human reviewer can catch a misconfigured storage bucket reliably when reviewing ten resources a week, but the same reviewer's error rate rises sharply once the volume reaches hundreds of resources a day, not because the reviewer becomes careless but because sustained attention to a repetitive check is a poor use of human judgement regardless of skill. A policy engine does not tire, so the argument for automating a check is strongest precisely where the check is simple, repetitive, and high-volume, which describes most cloud misconfiguration classes. Writing an enforceable rule A policy takes the resource's configuration, the identity of the requester, and the target environment as inputs, and returns a deterministic allow or deny. In the 2019 Capital One breach, a server-side request forgery reached the cloud metadata service and abused an over-privileged Identity and Access Management (IAM) role; a policy-as-code gate checking the wildcard breadth of that role against a least-privilege rule at deployment time would have flagged it before the role was ever granted. Good rules are declarative rather than ad hoc scripts, because a declarative decision is a reproducible function of recorded inputs that can be audited and replayed later. A deterministic policy needs a precisely defined input contract, exactly which fields of the resource configuration and requester identity it reads, because a rule that behaves differently depending on fields it was never designed to consider is not actually deterministic even if it looks that way in testing. The Capital One case is instructive because the missing check was not exotic: a rule comparing the requested IAM role's permissions against a documented least-privilege baseline is a straightforward width check, and its absence, not a lack of sophisticated tooling, was the gap. Gating a change Policy gates can legitimately sit pre-merge as a pull-request check, pre-deploy as an admission decision, and at runtime as drift detection; shifting checks left, to commit or pull-request time, is preferred because fixing a violation there is cheapest. A well-designed gate fails closed: when the gate itself errors, the change is blocked rather than waved through, accepting an availability cost in exchange for not silently disabling the control. Shifting a check left changes who sees the failure and how expensive it is to fix, but it does not remove the need for the same check later in the pipeline, because a developer's local environment or a fast-tracked emergency change can bypass an earlier stage entirely; a mature policy-as-code programme runs the same or an equivalent rule at multiple stages precisely because no single stage can be trusted as the only enforcement point. Fail-closed is uncomfortable operationally because it can block a legitimate change during a policy-engine outage, but the alternative, fail-open, means the control silently stops enforcing anything for the duration of the outage without anyone necessarily noticing. Versioning and testing policy Policies are code, so they carry unit tests that assert allow and deny decisions against fixture inputs before the policy ships, and policy changes go through code review because a policy change is a security-posture change. When two rules match one request with conflicting outcomes, a documented precedence rule such as deny-overrides resolves it deterministically, never chance or rule age. Finally, exceptions must be scoped, justified, and expiring: a permanent 'emergency' bypass used weekly means the gate no longer enforces anything at all. Testing a policy against fixture inputs before it ships is what prevents policy changes from becoming their own source of outages: a policy update intended to tighten one rule can accidentally broaden another if the two interact in ways the author did not anticipate, and a fixture suite that exercises both the intended change and the existing rule set together catches that interaction before it reaches production. Recording precedence explicitly also matters for auditability after the fact: when an incident review asks why a particular resource was allowed despite an apparently applicable deny rule, a documented precedence order gives a definite, checkable answer rather than requiring the reviewer to reconstruct evaluation order from the engine's source code. Related CCI capabilities Cambridge Cyber International's Mirage offering includes an extended policy engine of the kind this module argues every estate needs: deterministic, versioned, testable rules enforced at the mediation point itself rather than left to reviewer diligence. See cambridgecyberinternational.com for current status. The pipeline and its gates. Mapping controls onto continuous-integration stages, and mapping the Common Criteria test and vulnerability classes onto review gates and the evidence each emits.. Place security controls at the right pipeline stages.. Map the ATE and AVA classes onto review gates.. Configure a gate to emit test and vulnerability evidence on every change.. Controls along the pipeline, ATE: tests as a gate, AVA: vulnerability analysis as a gate, Evidence the pipeline emits The pipeline and its gates A Continuous Integration (CI) and Continuous Delivery (CD) pipeline is where policy, testing, and vulnerability analysis actually get enforced on every change. This lesson maps the Common Criteria's tests (ATE) and vulnerability analysis (AVA) classes onto concrete pipeline stages, and asks what evidence each gate must leave behind. Controls along the pipeline Different controls belong at different stages: static analysis and secret scanning at commit or pull-request time, dependency and licence checks at build, dynamic testing and policy admission at pre-deploy, and vulnerability scanning and drift detection continuously in production. The January 2023 CircleCI breach, which forced customers to rotate every secret the CI system had held, is the sharp reminder that the pipeline is itself a high-value asset: its secrets and its evidence store need the same protection as production, not less. Placing a control at the wrong stage undermines it even when the control itself is sound: static analysis run only at pre-deploy, long after the vulnerable code was written, still finds the defect but at far higher cost to fix and far later feedback to the author than the same check run at commit time, while a check that genuinely needs deployed, running infrastructure to be meaningful, such as configuration drift detection, cannot usefully be shifted left at all. Matching each control to the earliest stage where it can actually produce a meaningful result, not simply the earliest stage available, is the design skill this module is teaching. ATE: tests as a gate Mapping the ATE class onto a pipeline means treating functional test results, coverage, and independent execution as machine-checked gate criteria, not as a developer's verbal assurance. ATE_IND, testing independent of the developer, maps to a separate stage and identity re-executing the suite from a clean checkout of the exact release artefact, rather than developers re-running their own tests on their own machine. Independent execution matters because a developer's local test run can pass against an uncommitted local change, a stale dependency version, or an environment variable set only on their machine, none of which will be true in production; ATE_IND closes exactly that gap by re-running the suite from a clean checkout against the exact artefact digest that will actually ship. Coverage figures reported alongside test results are only meaningful when they are measured on that same clean, independent run, since a coverage figure computed against a developer's locally modified checkout can overstate what the shipped artefact actually exercises. AVA: vulnerability analysis as a gate AVA_VAN grades findings by the attack potential required to exploit them, factoring in elapsed time, attacker expertise, and equipment and access required. In pipeline terms this becomes recurring vulnerability analysis whose triaged findings feed a release gate with a defined severity threshold, not a single penetration test performed once at launch and never revisited. Attack potential in AVA_VAN terms is a composite of several factors, elapsed time, specialist expertise required, knowledge of the Target of Evaluation (TOE) needed, window of opportunity, and equipment required, and a finding that scores low on one factor, say it requires very little time, can still score high overall if it also requires deep specialist expertise and privileged access, which is why a severity score collapsed to a single number tends to lose exactly the information a triage decision needs. Recurring analysis matters because attack potential is not static: a technique that required rare specialist expertise last year can become a commodity tool this year, silently lowering the attack potential required to exploit an unpatched finding that was previously deprioritised as low risk. Evidence the pipeline emits Evidence must prove the control actually ran, not merely that nothing red appeared: a scanner that crashes and exits zero must not be read as 'no findings'; a gate has to require positive proof that the scan completed. Every result should be bound to the exact commit and artefact digest it was produced against, so a stale-branch scan cannot masquerade as a clean release. As the tampered Codecov uploader in 2021 showed, running for months before an external checksum comparison caught it, a gate must also verify the integrity of its own tooling, because evidence is only as trustworthy as the tool that emitted it. Evidence bundles belong in an append-only, tamper-evident store bound to artefact digests, never a mutable branch that force-push can rewrite. A gate taking four hours per change will accumulate exemptions until it enforces nothing, so gate latency is itself a security property. Binding evidence to an artefact digest rather than a branch name closes a specific and common gap: a branch name is a mutable pointer that can be force-pushed to point at different commits over time, so evidence recorded against 'main passed on this date' can no longer be trusted to describe what 'main' points to by the time an auditor checks it, while a digest is immutable by construction and cannot be silently redirected. Gate latency becomes a security property precisely because a slow gate creates pressure to bypass it under delivery deadlines, and a bypass mechanism used often enough stops being an exception and becomes the actual process, quietly replacing the control it was meant to be an exception to. Related CCI capabilities Cambridge Cyber International's PenTeva offering is a virtual machine that executes penetration tests and simulates vulnerabilities from a domain-specific language, the kind of recurring, automatable AVA_VAN-style analysis this module argues should run continuously rather than as a once-a-year exercise. See cambridgecyberinternational.com for current status. Verification at scale. Static and dynamic analysis, dependency and secret scanning, fuzzing, triage and the economics of false positives, and attack potential.. Run a scan suite and triage its findings against a gate.. Explain false-positive economics at scale.. Relate vulnerability findings to attack potential and AVA_VAN levels.. Static and dynamic analysis, Dependency and secret scanning, Fuzzing, Triage and false positives Verification at scale Scanning tools only produce security when their findings reach a system and a person who will act on them. Equifax, 2017, is the cautionary tale: a Struts patch, catalogued under the Common Vulnerabilities and Exposures (CVE) system as CVE-2017-5638, existed for months, but the vulnerable system was missed by scans and unowned. Verification at scale is as much about inventory, ownership, and triage discipline as it is about the scanners themselves. Static analysis trades soundness against noise: a tool tuned to find every possible instance of an injection pattern will also flag many code paths that are not actually reachable with attacker-controlled input, while a tool tuned to minimise false positives will miss some real defects, and most production tools let teams choose a point on that tradeoff rather than offering a single correct setting. Dynamic analysis avoids that particular tradeoff by observing real execution, but at the cost of only ever seeing the code paths actually exercised during the run, so its coverage is bounded by the quality of the test suite or traffic driving it. Static and dynamic analysis The core distinction is that static analysis reasons about code without executing it, catching classes of defect like injection patterns or unsafe API use across an entire codebase quickly, while dynamic analysis observes the running program, catching defects that only manifest at runtime, such as authentication bypass paths or memory corruption under real input. Neither subsumes the other; a mature pipeline runs both, at different stages, against different confidence levels. Dependency and secret scanning Software composition analysis matches the component inventory, ideally a software bill of materials (SBOM), against vulnerability intelligence feeds, turning 'we use library X' into 'library X version Y has CVE Z'. Secret scanning is most valuable at pre-receive or pull-request time, because a secret that never lands in history needs no rotation, while one discovered later in git history must be treated as compromised regardless of whether it was ever used. Software composition analysis is only as current as its vulnerability feed, and a component with a newly disclosed vulnerability will not be flagged by a scan run before the disclosure regardless of how current the SBOM itself is, which is why continuous rescanning against a live feed matters more than the freshness of the inventory alone. For secrets, rotation cost rises sharply the longer a leaked credential sits undetected, because every system that credential could have touched, not just the repository it was found in, now needs to be considered potentially compromised, which is the practical argument for scanning at commit time over scanning on a periodic schedule. Fuzzing Fuzzing feeds mutated, often malformed, inputs to a program to trigger memory-safety and input-parsing defects that manual review misses; Heartbleed was found in part through fuzzing-adjacent techniques. Effective fuzzing depends on coverage feedback to guide mutation toward unexplored code, sanitizers or oracles that make failures observable rather than silently corrupting state, and quality seed corpora to start from. Google's OSS-Fuzz, whose name denotes continuous fuzzing for Open-Source Software (OSS), demonstrates that continuous fuzzing of widely used open-source dependencies finds real vulnerabilities at scale, ahead of attackers. Coverage-guided fuzzing works by instrumenting the target to report which code paths a given input exercised, then using that feedback to bias future mutations toward inputs that reach new, previously unexplored code, which is what separates it from purely random input generation and is why it finds deeper defects with far fewer executions. A fuzzer without a sanitizer or oracle can run for months and report zero crashes even against a genuinely vulnerable target, because a memory-safety violation that corrupts state without immediately crashing the process is invisible to a harness that only checks for a crash, which is precisely the failure mode sanitizers are built to catch. Triage and false positives Sound triage prioritises findings by exploitability in attack-potential terms, the exposure of the affected asset, and the cost and blast radius of the fix, not alphabetical order or gut feeling. Every result must bind to the exact commit and artefact digest scanned, so a stale-branch 'clean' report cannot masquerade as coverage of the real release. False positives at scale are dangerous precisely because they consume scarce review attention and erode trust until the gate is ignored or disabled; the fix is to tune and suppress per finding class with a recorded justification, never a blanket severity cut that can silently drop a real high-severity finding, as happened when a team filtered to critical-only and missed a finding that later chained into a compromise. Rising AVA_VAN levels, in Common Criteria terms, correspond to demonstrated resistance to attackers of increasing attack potential, which is what disciplined verification is ultimately building toward. Attack-potential-based prioritisation and Common Vulnerability Scoring System (CVSS) base scores frequently disagree, because CVSS describes the vulnerability in the abstract while attack potential accounts for this specific asset's exposure, so a critical CVSS finding on an internal, network-isolated system can legitimately triage below a medium CVSS finding on an internet-facing one; a triage process that mechanically sorts by CVSS alone will systematically misallocate the limited review attention a team actually has. Recording the justification for every suppression, not just the high-severity ones, is what lets a later reviewer distinguish a deliberately accepted false positive from a finding that was simply never looked at, which look identical in a dashboard that only shows current status. Related CCI capabilities Cambridge Cyber International's EviGen offering addresses the triage-and-ownership gap this module raises: automated technical evidence collection that binds a finding to a system and a named owner, rather than leaving a scan result to be discovered later in a dashboard nobody is assigned to read. See cambridgecyberinternational.com for current availability. Security across a thousand applications. Paved roads and golden paths, secure defaults, inventory and drift, exceptions and risk acceptance, and governance at scale.. Design a paved road with secure defaults.. Identify when a service has drifted from the paved road.. Handle exceptions and risk acceptance without eroding the baseline.. Paved roads and golden paths, Secure defaults, Inventory and drift, Exceptions and risk acceptance Security across a thousand applications Controls that work for one application do not automatically work for a thousand. At estate scale, the security team's job shifts from reviewing individual systems to shaping the defaults and paths that most teams will follow without ever reading a policy document. Paved roads and golden paths A paved road, or golden path, is a supported default path that makes the secure choice the easiest choice for product teams: a pre-built service template with authentication, logging, and dependency scanning already wired in, so adopting it is less work than doing it yourself insecurely. A mandate issued without a paved implementation predictably produces divergence: when security demands a new control with no reference implementation, forty teams will hand-roll forty variants, and the mandate has effectively designed its own failure. A paved road only stays paved if it is actively maintained against the same threats the rest of the estate faces; a service template wired up once and never updated will eventually embed an outdated authentication library or a deprecated logging format, at which point teams who adopted it early are running the vulnerability the template shipped with, and the team owning the paved road carries an ongoing obligation, not a one-time build. Adoption also has to be genuinely easier than the insecure alternative, not merely officially sanctioned, because a paved road that requires more setup effort than hand-rolling a service will lose to the hand-rolled version regardless of how strongly it is recommended. Secure defaults Most services run whatever the default is, so improving the default fixes the estate wholesale in a way that training or documentation never will, because engineers reliably do not read manuals under delivery pressure. Investment in the default is investment with estate-wide leverage; investment in exhortation is not. Changing a default typically requires touching a single configuration point, while training changes require reaching every engineer individually and hoping the training is remembered under pressure months later, which is why the leverage of a default fix is structurally different in kind, not just in degree, from the leverage of a policy document or a training session. A default that is merely 'secure enough to pass a checklist' but inconvenient in normal use invites engineers to override it locally the first time it gets in the way, at which point the estate-wide leverage the default was supposed to provide is lost service by service. Inventory and drift During log4shell, organisations without an application inventory spent their first days merely enumerating exposure, which is why a continuously maintained inventory of what runs where, with named owners and dependency data, is the first estate-scale control, prior to any scanner. NotPetya, 2017, spreading through the compromised update channel of M.E.Doc accounting software, is the reminder that software you did not choose, embedded updaters and vendor agents, is still part of your inventory and attack surface. The percentage of services on the paved road is a useful governance metric because it shows where investment in the paved path yields estate-wide return, and where drift concentrates. An inventory that records only which services exist, without also recording their dependency graph, embedded third-party components, and named owners, cannot answer the question that actually matters during an incident: not 'do we run library X' but 'which of our systems run library X, who owns each one, and can we reach them today'. The NotPetya spread through M.E.Doc's updater illustrates a category of inventory gap that is easy to miss deliberately: a component nobody chose or configured, bundled inside a vendor product, is still part of the attack surface and still needs an owner even though no engineer ever wrote a line of it. Exceptions and risk acceptance Risk acceptance, done properly, is a named owner recording acceptance of a residual risk with rationale and a review date, not the security team unilaterally waiving a rule. A disciplined exception process requires a defined scope, a time box with expiry, and a named owner accepting the risk in writing; a service that received a 'temporary' exception eighteen months ago and still holds it reveals a process defect, exceptions not time-boxed with automatic expiry and re-review, not a failure of the original decision. The count and age of outstanding exceptions is a leading indicator of baseline erosion. Governance at estate scale typically works best when a central team owns the baseline and paved roads while product teams own risk decisions within them, rather than either central approval of every change or every team setting its own baseline. A time-boxed exception with automatic expiry forces a periodic re-decision, keep the exception, fix the underlying issue, or escalate, while an exception with no expiry defaults to being kept forever simply because nobody is prompted to revisit it, which is the mechanism by which a genuinely temporary decision quietly becomes a permanent, undocumented gap in the baseline. Tracking the count and age of outstanding exceptions as a metric works because it makes baseline erosion visible before it becomes an incident, converting a qualitative worry, 'are we drifting', into a number a governance review can actually act on. Related CCI capabilities Cambridge Cyber International's Secure software development lifecycle (SDLC) at Scale offering is built for exactly the estate-scale problem this module describes: a platform operated jointly with CCI engineers for estates of the thousand-application order, so the paved road stays maintained rather than becoming the outdated template teams adopted early are still running. See cambridgecyberinternational.com for current availability. Capstone: a service through the disciplined pipeline. Take a sample service end to end: threat model, mini Security Target, life-cycle-instrumented repository, policy gates and test and vulnerability evidence, graded at the review gates.. Produce a threat model and a short Security Target for the service.. Identify the repository and pipeline touchpoints and instrument them with the controls and gates.. Pass each review gate with the evidence attached.. Threat model and Security Target, Instrumenting the repository, Wiring the gates, Defending the evidence Capstone: a service through the disciplined pipeline The capstone applies every earlier lesson to one sample service, in the disciplined order the course has argued for throughout: threat model, then mini Security Target, then instrumented repository and pipeline, then gates emitting evidence. Building the artefacts out of order, for example wiring gates before the threat model exists, produces controls that trace to nothing. Threat model and Security Target Start with a data-flow diagram and trust boundaries for the service, enumerate threats using Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, and Elevation of privilege (STRIDE) analysis, and write the security problem definition. From that, derive a mini Security Target: security objectives for the service, and selected Security Functional Requirements (SFRs) with a traceability rationale connecting each SFR back to an objective and each objective back to a threat. A gate that traces to no threat in the threat model and no requirement in the Security Target is an unjustified control, a candidate for removal or evidence that something is missing upstream, not a sign of extra diligence. Deriving the mini Security Target from the threat model, rather than writing both in parallel, is what keeps the traceability chain honest: writing them independently makes it easy to produce a Security Target (ST) that reads well on its own but does not actually answer every threat identified earlier, while deriving it strictly from the threat model forces every objective to point back to a specific threat, assumption, or policy that was actually written down. The capstone deliberately withholds credit for objectives or requirements that cannot be traced this way, because an untraceable requirement is indistinguishable from one chosen by habit rather than by analysis. Instrumenting the repository Instrumenting the repository means configuration-management discipline, protected branches and reviewed merges, so the configuration management (CM) record always matches what is actually running; signed commits and artefacts, so provenance can be checked later; and the pipeline definition itself treated as reviewed code, not an unaudited shell script. Enabling force-push for speed is precisely the shortcut that breaks this discipline and should not appear in the instrumented repository. Protected branches and reviewed merges matter less for catching a malicious insider, though they help with that too, than for ensuring the CM record actually reflects reality: a repository that allows direct pushes to the release branch means the recorded history of reviewed changes can diverge from what is actually running, silently invalidating every later claim that traces back to 'this was reviewed'. Treating the pipeline definition itself as reviewed code closes a specific gap the earlier modules raised: a pipeline file is just as capable of introducing a supply-chain compromise as application source code, and an unreviewed change to it bypasses every control the reviewed application code goes through. Wiring the gates Supply-chain integrity becomes a release gate that blocks unless the artefact digest matches its recorded provenance. Policy-as-code rules, such as an unsigned-artefact deny rule, must actually fire on the release commit, not merely exist in a repository. When an injected constraint arrives mid-capstone, for example a new evidence-retention obligation, the disciplined response is to re-derive the affected objectives and requirements, adjust the gates, and show the traceability still holds, not to bolt on an unrelated fix or ignore it until release. A gate that exists in the pipeline configuration but has never actually blocked a change, because every artefact so far has happened to be compliant, has not yet been proven to work; the capstone's expectation is that a deliberately non-compliant test case is run against each gate to confirm it fails closed as designed, not merely that it has never yet had a reason to fire. Defending the evidence Acceptable tests (ATE) evidence is results traced to requirements, a coverage threshold met, and execution from a clean checkout of the release artefact, not a chat message saying tests passed. Acceptable vulnerability-analysis (AVA) evidence is a triage record reasoning in attack-potential terms about every finding left open below the threshold, not an empty scanner screen. The evidence that a deny rule worked is the logged deny decision bound to the offending artefact's digest. The distinction between a scanner producing no output and a scanner that ran and found nothing is exactly the distinction the pipeline evidence discussion raised earlier in the course, and the capstone is where it becomes concrete: a pipeline step that silently fails to execute and a pipeline step that executes and passes cleanly can look identical in a dashboard showing only green checkmarks, and only evidence that positively demonstrates execution, not merely the absence of a failure signal, distinguishes the two. A pipeline that shows green while the evidence reveals a gate was actually skipped for the release commit fails, because a pass corresponds to the control demonstrably firing on this specific change, not to the absence of red or to an eloquent narrative with no machine-checkable evidence behind it. Related CCI capabilities Cambridge Cyber International's Global Cyber Incident Database (GCIDB) is a natural source of the kind of real capstone scenario this module asks for: a documented incident with a genuine threat, timeline, and consequence to threat-model, target, and gate against, rather than an invented exercise with no external accountability. See cambridgecyberinternational.com for current coverage.