SA-15 — Development Process, Standards, and Tools

Organizations use quality metrics to establish acceptable levels of system quality. Metrics can include quality gates, which are collections of completion criteria or sufficiency standards that represent the satisfactory execution of specific phases of the system development project. For example, a quality gate may require the elimination of all compiler warnings or a determination that such warnings have no impact on the effectiveness of required security or privacy capabilities. During the execution phases of development projects, quality gates provide clear, unambiguous indications of progress. Other metrics apply to the entire development project. Metrics can include defining the severity thresholds of vulnerabilities in accordance with organizational risk tolerance, such as requiring no known vulnerabilities in the delivered system with a Common Vulnerability Scoring System (CVSS) severity of medium or high.

1. SA-15(01)(a)the developer of the system, system component, or system service is required to define quality metrics at the beginning of the development process;
2. SA-15(01)(b)the developer of the system, system component, or system service is required to provide evidence of meeting the quality metrics [Organization-defined: sa-15.01_odp.01].

System development teams select and deploy security and privacy tracking tools, including vulnerability or work item tracking systems that facilitate assignment, sorting, filtering, and tracking of completed work items or tasks associated with development processes.

1. SA-15(02)[01]the developer of the system, system component, or system service is required to select and employ security tracking tools for use during the development process;
2. SA-15(02)[02]the developer of the system, system component, or system service is required to select and employ privacy tracking tools for use during the development process.

Criticality analysis performed by the developer provides input to the criticality analysis performed by organizations. Developer input is essential to organizational criticality analysis because organizations may not have access to detailed design documentation for system components that are developed as commercial off-the-shelf products. Such design documentation includes functional specifications, high-level designs, low-level designs, source code, and hardware schematics. Criticality analysis is important for organizational systems that are designated as high value assets. High value assets can be moderate- or high-impact systems due to heightened adversarial interest or potential adverse effects on the federal enterprise. Developer input is especially important when organizations conduct supply chain criticality analyses.

1. SA-15(03)(a)the developer of the system, system component, or system service is required to perform a criticality analysis at [Organization-defined: decision points] in the system development life cycle;
2. SA-15(03)(b)
   1. SA-15(03)(b)[01]the developer of the system, system component, or system service is required to perform a criticality analysis at the following rigor level: [Organization-defined: breadth];
   2. SA-15(03)(b)[02]the developer of the system, system component, or system service is required to perform a criticality analysis at the following rigor level: [Organization-defined: depth] .

Attack surface reduction is closely aligned with threat and vulnerability analyses and system architecture and design. Attack surface reduction is a means of reducing risk to organizations by giving attackers less opportunity to exploit weaknesses or deficiencies (i.e., potential vulnerabilities) within systems, system components, and system services. Attack surface reduction includes implementing the concept of layered defenses, applying the principles of least privilege and least functionality, applying secure software development practices, deprecating unsafe functions, reducing entry points available to unauthorized users, reducing the amount of code that executes, and eliminating application programming interfaces (APIs) that are vulnerable to attacks.

the developer of the system, system component, or system service is required to reduce attack surfaces to [Organization-defined: thresholds].

Developers of systems, system components, and system services consider the effectiveness and efficiency of their development processes for meeting quality objectives and addressing the security and privacy capabilities in current threat environments.

the developer of the system, system component, or system service is required to implement an explicit process to continuously improve the development process.

Automated tools can be more effective at analyzing exploitable weaknesses or deficiencies in large and complex systems, prioritizing vulnerabilities by severity, and providing recommendations for risk mitigations.

1. SA-15(07)(a)the developer of the system, system component, or system service is required to perform automated vulnerability analysis [Organization-defined: frequency] using [Organization-defined: tools];
2. SA-15(07)(b)the developer of the system, system component, or system service is required to determine the exploitation potential for discovered vulnerabilities [Organization-defined: frequency];
3. SA-15(07)(c)the developer of the system, system component, or system service is required to determine potential risk mitigations [Organization-defined: frequency] for delivered vulnerabilities;
4. SA-15(07)(d)the developer of the system, system component, or system service is required to deliver the outputs of the tools and results of the analysis [Organization-defined: frequency] to [Organization-defined: personnel or roles].

Analysis of vulnerabilities found in similar software applications can inform potential design and implementation issues for systems under development. Similar systems or system components may exist within developer organizations. Vulnerability information is available from a variety of public and private sector sources, including the NIST National Vulnerability Database.

1. SA-15(08)[01]the developer of the system, system component, or system service is required to use threat modeling from similar systems, components, or services to inform the current development process;
2. SA-15(08)[02]the developer of the system, system component, or system service is required to use vulnerability analyses from similar systems, components, or services to inform the current development process.

The incident response plan provided by developers may provide information not readily available to organizations and be incorporated into organizational incident response plans. Developer information may also be extremely helpful, such as when organizations respond to vulnerabilities in commercial off-the-shelf products.

1. SA-15(10)[01]the developer of the system, system component, or system service is required to provide an incident response plan;
2. SA-15(10)[02]the developer of the system, system component, or system service is required to implement an incident response plan;
3. SA-15(10)[03]the developer of the system, system component, or system service is required to test an incident response plan.

Archiving system or system components requires the developer to retain key development artifacts, including hardware specifications, source code, object code, and relevant documentation from the development process that can provide a readily available configuration baseline for system and component upgrades or modifications.

the developer of the system or system component is required to archive the system or component to be released or delivered together with the corresponding evidence supporting the final security and privacy review.

Organizations can minimize the risk to an individual’s privacy by using techniques such as de-identification or synthetic data. Limiting the use of personally identifiable information in development and test environments helps reduce the level of privacy risk created by a system.

the developer of the system or system component is required to minimize the use of personally identifiable information in development and test environments.

In support of better incident response and the ability to more quickly reconstruct security-related actions, identifying specific requirements for secure logging facilitates the ability to connect application-produced audit event logs with operational data. Event types are consistent with the event types defined in [AU-02](#au-2).

Determine if: the developer of the system, system component, or system service uses [Organization-defined: secure logging format(s)] to log [Organization-defined: events types to log] at [Organization-defined: level of detail to log].