Netsparker. Vulnerability Assessments and Penetration Tests – What’s the Difference? (Part I)

Confusion between the terms ‘penetration testing’ and ‘vulnerability assessments’ often begins at the level of language. Those who are not full-time professionals in web security, such as journalists reporting on a big story that affects consumers, use the terms interchangeably, as if referring to the same process.

Experienced professionals in the industry know the difference, but those new to it can be easily confused. Why? Even professionals sometimes use terms in fuzzy or inexact ways, when they should distinguish between things that differ. Let’s be clear on the difference between the two.

What Are Vulnerability Assessments?

A vulnerability assessment involves running a series of multiple tests, against defined websites, web applications, IP addresses and ranges, using a known list of vulnerabilities, such as the OWASP Top 10 list. Assessors may also run tests against systems they know to be incorrectly configured or unpatched. Often, automated security scanning tools are used. Commercially licensed, subscription-based tools are regarded as coming with less risk – regular updates, release notes bring less chance of the inclusion of malicious code. (Their open source equivalents, however, have the significant advantage of being the exact same tools that malicious hackers prefer.)

Vulnerability assessments tend to include the following stages:

  • Identifying all resources, and connected resources, within an organisation’s IT systems
  • Assigning a value or priority to each one
  • Conducting an assessment of lists of known vulnerabilities across a large number of attack surfaces (from login screens to URL parameters to mail servers)
  • Fixing the most critical vulnerabilities and making decisions about how to the deal with the rest

What is Penetration Testing?

Penetration testing (pen testing), on the other hand – while it may be considered to be a type of vulnerability assessment – involves replicating a specific type of attack that might be carried out by a hacker. A pen tester will often explore the systems until they find a vulnerability. They may even employ a vulnerability assessment tool to uncover a vulnerability. Once they find something, they will then try to exploit it, to determine whether it would be possible for a hacker to achieve a certain objective (access, change or delete data, for example). Often, while doing this, they may accidentally encounter other vulnerabilities, and follow where they lead. The pen tester may use an automated tool at this point to run a series of exploits against the vulnerability.

Some penetration tests are referred to as ‘white box’ to indicate that the penetration tester has been given detailed information about the environment, such as a list of assets belonging to the organization, source codes, employee names and email addresses etc. When they are referred to as ‘black box’, this indicates tests that are conducted without any prior information about the internal structure, access to source code etc.  This kind of pen test of course, can more closely resemble the activities of a malicious hacker, but may also lead to less thorough coverage of the companies potentially vulnerable assets.


What Results Can I Expect From Each Approach?

The answer to this question might best be asked by thinking backwards: What results do you want?

Vulnerability Assessments Report Across All Vulnerabilities

The results are collated in an automated, lengthy report, with a comprehensive list of detected vulnerabilities arranged by priority, determined by how by severe and business-critical they are. As time goes on, this list can reveal changes since the last report. One of the criticisms of the results achieved is that, unlike in penetration testing, they can contain false positives or false negatives. Naturally, this is not the case if you use Netsparker web application vulnerability scanner to conduct your vulnerability testing. It is one of our key features – automatically verifying identified vulnerabilities with Proof-Based Scanning.

Reports should include guidance on how to remediate the detected vulnerabilities, and tools sometimes come with patches subscribers can use. In most cases, results are then allocated to dedicated development teams who conduct fixes, remove the most serious vulnerabilities, and otherwise address the less serious ones. In an ideal world, this activity is ongoing, scheduled regularly, and built into the organisation’s SDLC.

Penetration Testing Reports Deep Into Each Vulnerability

With pen testing, there is no lengthy public report, though some record and publish their actions and anonymized findings, blog about their experiments, or live hack at conferences. If you hire a pen tester, however, they should deliver a (pen test) report, but it tends to be focused on the attack method or exploit, and exactly what data can be compromised. It will generally be accompanied by suggestions on what a hacker might be able to do to, or with, it. This helps business analysts and non-technical professionals, who may not understand all of the technology behind such tests, grasp business process impacts quickly.

Sometimes reports also incorporate remediation advice. However, not all pen tests incorporate exploitation of vulnerabilities in the way that Netsparker does. It may be sufficient simply to illustrate that an attack is possible. In some cases the pen test report may simply report theoretical vulnerabilities because attempting to exploit them may result in a catastrophic denial of service (DoS). And, finally, there is no assessment of vulnerabilities, since the goal is simply to do one thing, or least to determine whether it can be done.

Which Approach Should My Organisation Adopt?

The main question to ask is: What is your current security posture?

To be continued…