When most think of a security test, it conjures images of Hollywood-style hacking. For those who work in the software industry, it still conjures images of an employee or contractor attempting to hack “for good” so that the flaws can be fixed before a bad actor finds them. I want to talk about the other stuff. The rest of the work that goes into testing for security that isn’t a tool-assisted investigation of a live (or live-like) deployment of the software.
The Application Under Test
Let’s start by describing a deployment that we’d like to test.
|--------------------------------|
| Application Container |
| |
| |--------------------------| |
| | Application | |
| | |------------| | |
| | | |------------| | |
| | |-| |------------| | |
| | |-| |------------| | |
| | |-| Dependency | | |
| | |------------| | |
| | | |
| | |--------------------| | |
| | | Application Code | | |
| | |--------------------| | |
| |--------------------------| |
| |
| |--------------------------| |
| | Operating System | |
| |--------------------------| |
|--------------------------------|
This is a single application container, based on an upstream base OS container plus our application under test. The application is bespoke code that exposes and API, and has dependencies. Here’s an example repository: https://github.com/EBTPT/spring-boot-hello-world.
Knowing the Risks
Looking at the architecture above, there are a few obvious risks:
- The application code is vulnerable
- A dependency of the application is vulnerable
- The operating system of the container is vulnerable
Note that this precludes the operating system of the container runner, or the version of the container runtime - those are part of the production environment, and whilst they could and should be considered before production deployment, are only truly testable in deployment.
Each of these identified risks are independently security testable and the risks addressable as pre-deployment activities. This doesn’t guarantee that a pentest won’t find problems, but it does mean it won’t find anything stupid.
Testing Application Dependencies
OWASP Dependency Check
Plenty of tooling exists to test dependencies. One of the most popular of these is OWASP Dependency-Check.
This can either be downloaded from their webpage, or installed via a package manager (e.g. brew install dependency-check).
For our example repository, we can run:
git clone [email protected]:EBTPT/spring-boot-hello-world.git
cd spring-boot-hello-world
mvn package
dependency-check -s .
This will generate a file named dependency-check-report.html which lists all of the vulnerable dependencies.
OWASP Dependency Check in Maven
Instead of relying on a local tool being downloaded or installed, you can add the dependency check to be run via maven.
Add the following to the build plugins within the POM:
<plugin>
<groupId>org.owasp</groupId>
<artifactId>dependency-check-maven</artifactId>
<version>6.1.6</version>
<executions>
<execution>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
Then run mvn verify to see the same page generated in the “target” directory. There are further configuration options in the Dependency-Check Documentation.
GitHub Dependabot Alerts
Rather than keep dependencies local, moving the scanning into CI has value. You don’t need active development or humans running commands. CI means that every build will include a dependency scan. In the previous step we added dependency scanning to Maven, and we could easily have added the mvn verify command as an additional step (or in lieu of an imaginary mvn package).
GitHub offers a further level of ease to this, where instead of requiring either active development or scheduled builds to enable the scan to occur, it constantly monitors dependencies in the background and offers alerts and solutions.
To switch it on, visit the Security settings of a project and enable it. To try this on the example project, you’ll need to fork it first.
Enable all three to get alerts and automated fixes.
Remediations for dependency problems
Except when using Dependabot in GitHub to generate PRs for updates, remediation for these is often just a case of bumping numbers in the POM or other dependency management file in your source code. The trickiest part can be finding the correct dependency to bump, since the vulnerability may be in a dependency of a dependency of a dependency.
It’s worth noting that just because a vulnerability is known, it might not always have a fix. Perhaps the fix is still in progress, or the software could be abandoned. It’s also possible that the vulnerability is “by design”, for example “eval” style tools intentionally allow execution of arbitrary strings, and it’s how they’re used that lead to whether it’s a genuine vulnerability.
Testing the Container
Given that the container image contains a lot of components that are seen in a full operating system (e.g. bash, curl), it has a lot of components, any of which could introduce its own vulnerabilities to the running application, allowing an attacker to exert some control.
Like with dependencies, there are a number of tools and methods available to scan your container for vulnerabilities.
Anchore Grype
Anchore has made a name for itself of as one of the de-facto container scanning tools. You can install and run this locally with our image like so:
git clone [email protected]:EBTPT/spring-boot-hello-world.git
cd spring-boot-hello-world
mvn package
docker build . -t spring-boot-hello-world
brew tap anchore/grype
brew install grype
grype spring-boot-hello-worldThis will print a table of known vulnerabilities with the components of this image.
$ grype ebtpt/hello-world-spring-boot
✔ Vulnerability DB [updated]
✔ Loaded image
✔ Parsed image
✔ Cataloged packages [176 packages]
✔ Scanned image [149 vulnerabilities]
NAME INSTALLED FIXED-IN VULNERABILITY SEVERITY
h2 1.4.197 CVE-2018-10054 High
h2 1.4.197 CVE-2018-14335 Medium
hibernate-validator 6.0.11.Final 6.0.18 GHSA-m8p2-495h-ccmh Medium
hibernate-validator 6.0.11.Final CVE-2020-10693 Medium
jackson-databind 2.9.6 2.9.7 GHSA-645p-88qh-w398 High
jackson-databind 2.9.6 2.9.8 GHSA-f9hv-mg5h-xcw9 High
...etc
Take a look at the Grype docs to see other output formats, especially if you’re using this in CI.
Docker Scan command
Little known fact: Docker includes container vulnerability scanning out of the box.
Instead of installing additional tooling, you can use the docker scan command (backed by the Snyk vulnerability database) to get a similar list of vulnerabilities as provided by Anchore.
Similar to above:
git clone [email protected]:EBTPT/spring-boot-hello-world.git
cd spring-boot-hello-world
mvn package
docker build . -t spring-boot-hello-world
docker scan spring-boot-hello-worldWhich outputs:
$ docker scan ebtpt/hello-world-spring-boot
Testing ebtpt/hello-world-spring-boot...
✗ Low severity vulnerability found in openssl/libcrypto1.1
Description: Inadequate Encryption Strength
Info: https://snyk.io/vuln/SNYK-ALPINE39-OPENSSL-1089236
Introduced through: openssl/[email protected], openssl/[email protected], apk-tools/[email protected], libtls-standalone/ [email protected], ca-certificates/ca-certificates@20190108-r0, krb5-conf/[email protected]
From: openssl/[email protected]
From: openssl/[email protected] > openssl/[email protected]
From: apk-tools/[email protected] > openssl/[email protected]
and 7 more...
Fixed in: 1.1.1j-r0
<...lots more here...>
✗ High severity vulnerability found in bzip2/libbz2
Description: Out-of-bounds Write
Info: https://snyk.io/vuln/SNYK-ALPINE39-BZIP2-452847
Introduced through: bzip2/[email protected], freetype/[email protected]
From: bzip2/[email protected]
From: freetype/[email protected] > bzip2/[email protected]
Fixed in: 1.0.6-r7
Package manager: apk
Project name: docker-image|ebtpt/hello-world-spring-boot
Docker image: ebtpt/hello-world-spring-boot
Platform: linux/amd64
Tested 55 dependencies for known vulnerabilities, found 87 vulnerabilities.
Alpine 3.9.4 is no longer supported by the Alpine maintainers. Vulnerability detection may be affected by a lack of security updates.
For more free scans that keep your images secure, sign up to Snyk at https://dockr.ly/3ePqVcp
Docker Hub (Paid plans)
Those with a paid Docker Hub plan can enable container scanning as images are pushed into the repository. This surfaces the same information as the docker scan command, and uses the same Snyk datasource. For some, this might be to late in the flow, but that’s entirely dependent on context.
GitHub Actions
Where the Docker Image is being built as part of the CI pipeline, adding scanning to the pipeline is easy, especially in GitHub where the GitHub Actions marketplace has many 3rd party tools packaged for easy inclusion in pipelines. GitHub Actions appears to have innovated upon the CircleCI concept of Orbs, and brought an enormous ecosystem of tools to bear for easy inclusion in CI, largely through its position of “being GitHub”. For example, here’s Anchore Grype. Results can be emitted as a file, as well as configured to fail the build.
Remediations for container problems
The vulnerabilities might be introduced by something in the local Dockerfile, one of the application dependencies, or something from the upstream image. The Dockerfile lists the upstream image as openjdk:8-jdk-alpine. Depending on the environment in which you’re developing, there’s a trade-off here between silent updates and reproducible builds. Docker tags are not immutable. This tag will have been updated many times through its lifetime. Each time a downstream image like ours was built, the latest image associated with the tag will have been fetched, including any new fixes in the upstream image. This has the downside of not being able to guarantee the same behaviour between two container image builds with no application code changes. This isn’t just a problem of reproducible builds, but a problem of unrecorded unintentional changes. This can be resolved by pinning the SHA256 of the image, e.g. changing FROM openjdk:8-jdk-alpine to FROM openjdk:8-jdk-alpine@sha256:a3562aa0b991a80cfe8172847c8be6dbf6e46340b759c2b782f8b8be45342717. Once this is done, you can treat this is as a dependency like any other, cognisantly upgrading when needed. If you use Dependabot for automated updates, this supports updates of Dockerfiles too.
The reason there are so many vulnerabilities on this particular tag is that it’s no longer maintained. Alpine 3.9 was end of life some time ago, and this image was last updated in May 2019.
To solve this instance, we could switch the base image to openjdk:8-jdk-oracle which is still maintained, and at time of writing lists 0 vulnerabilities. We could also switch to openjdk:8-jdk-slim but whilst this is still maintained, at time of writing it boasts more vulnerabilities than the unmaintained alpine image (not that “count” is necessary a subjective measure of quality or security).
Testing the Application Code
Given the information we’ve surfaced so far, we might now trust the container and trust our dependencies (insofar as they contain no known vulnerabilities), so we now need to turn our focus to our own code.
These examples are somewhat lighter on content, as at the time of writing, the repo I’ve picked contains no detectable vulnerabilities. If I updated the repo to introduce some, the examples would still be very contextual to the code and its issues, and wouldn’t likely aid you fair readers.
GitHub CodeQL
Turn it on to get static analysis as one of your CI pipeline checks on your PRs, and to get enhanced alerting in the Security tab in GitHub.
- Go to Security -> Code scanning, and click the ‘Configure scanning tool’ button.
- Next to CodeQL analysis, pick Set up, then Default
- Tick
Java / Kotlin - Press the
Enable CodeQLbutton
This will kick off a Github Action run, which will be slow on its first run, as it needs to scan the entire repository, and deliver results to Security -> Code scanning, ready for your review.
CodeQL alternatives in GitHub Actions
If you’d rather not rely more heavily on GitHub and its internal tooling (perhaps because you want to be ready to migrate away at a moment’s notice), the GitHub Actions marketplace absolutely has off-the-shelf alternatives that will be portable to other CI systems. Take a look at https://github.com/marketplace?query=sast
Snyk in the IDE
Snyk has a hosted option (which makes sense, given the number of times we’ve mentioned the rich Snyk data throughout this article) but did you know that multiple IDEs have Snyk plugins to enhance the static analysis that happens as you type or open files?
https://snyk.io/platform/ide-plugins/
As well as highlighting as you go, local full scans are possible to get a full set of problems, not just in code, but in dependencies and Infrastructure-as-Code as well.
SpotBugs and FindSecBugs
This is only for Java, but alternatives for other languages are listed in the OWASP Wiki.
Full System Solutions
Rather than apply effort layer-by-layer, some solutions want to reduce that burden by offering all layers wrapped in a single package.
Snyk
Mentioned above as an IDE plugin, Snyk offer a hosted platform that will cover every layer of the application. https://snyk.io
They’ll tackle container vulnerabilities, dependencies, code, and even look at code quality and peek at your IaC to check your deployment security.
GitLab
GitLab have had a proper Security Center in their offering since before GitHub offered Actions. They’ll do all the scans, including more advanced application security scanning like DAST. All vulnerabilities generate a detection which can be triaged (issues created, dismissed as false positives, etc), which is lovely auditor fodder, if you care about such things.
Disclaimer
I’ve mentioned a bunch of tools, some of which would love to have some of your money. I’ve mentioned tools that I’ve used and like, but I am in no way affiliated, remunerated, compensated or related to any of them.