Penetration Testing Level 1
[PT1.1: 82] Use external penetration testers to find problems.
Many organizations aren’t willing to address software security until there’s unmistakable evidence that the organization isn’t somehow magically immune to the problem. If security has not been a priority, external penetration testers can demonstrate that the organization’s code needs help. Penetration testers could be brought in to break a high-profile application to make the point. Over time, the focus of penetration testing moves from “I told you our stuff was broken” to a smoke test and sanity check done before shipping. External penetration testers bring a new set of eyes to the problem.
[PT1.2: 58] Feed results to the defect management and mitigation system.
Penetration testing results are fed back to development through established defect management or mitigation channels, and development responds using their defect management and release process. The exercise demonstrates the organization’s ability to improve the state of security. Many firms are beginning to emphasize the critical importance of not just identifying but, more importantly, fixing security problems. One way to ensure attention is to add a security flag to the bug tracking and defect management system. Evolving DevOps and integrated team structures do not eliminate the need for formalized defect management systems.
[PT 1.3: 54] Use penetration testing tools internally.
The organization creates an internal penetration testing capability that uses tools. This capability can be part of the SSG or part of a specialized and trained team elsewhere in the organization. The tools improve efficiency and repeatability of the testing process and are often necessary in agile environments. Tools can include off-the-shelf products, standard issue network penetration tools that understand the application layer, and hand-written scripts.
Penetration Testing Level 2
[PT2.2: 21] Provide penetration testers with all available information.
Penetration testers, whether internal or external, use all available information about their target. Penetration testers can do deeper analysis and find more interesting problems when they have source code, design documents, architecture analysis results, and code review results. Give penetration testers everything you have created throughout the SSDL and ensure they use it. If your penetration tester doesn’t ask for the code, you need a new penetration tester.
[PT2.3: 16] Schedule periodic penetration tests for application coverage.
Periodically test all applications in the SSG’s purview according to an established schedule, which could be tied to the calendar or to the release cycle. The testing serves as a sanity check and helps ensure yesterday’s software isn’t vulnerable to today’s attacks. This also helps maintain the security of software configurations and environments, especially in the cloud. High-profile applications might get a penetration test at least once a year. One important aspect of periodic testing is to make sure that the problems identified in a penetration test are actually fixed and they don’t creep back into the build.
Penetration Testing Level 3
[PT3.1: 10] Use external penetration testers to perform deep-dive analysis.
The organization uses external penetration testers to do deep-dive analysis for critical projects and to introduce fresh thinking into the SSG. These testers are experts and specialists; they keep the organization up to speed with the latest version of the attacker’s perspective and have a track record for breaking the type of software being tested. Skilled penetration testers will always break a system. The question is whether they demonstrate new kinds of thinking about attacks that can be useful when designing, implementing, and hardening new systems. Creating new types of attacks from threat intelligence and abuse cases prevents checklist-driven approaches that only look for known types of problems and is pretty much essential when it comes to new technology.
[PT3.2: 6] Have the SSG customize penetration testing tools and scripts.
The SSG either creates penetration testing tools or adapts publicly-available tools so they can more efficiently and comprehensively attack the organization’s systems. Tools improve the efficiency of the penetration testing process without sacrificing the depth of problems the SSG can identify. Automation can be particularly valuable under agile methodologies because it helps teams go faster. Tools that can be tailored are always preferable to generic tools. This activity considers both the depth of tests and their scope.