One of the biggest challenges in the industry as a whole and in the DevOps world today in particular is vulnerability management.
As we in the industry are trying to implement better security practices and to evolve the secure software supply chain whether by choice or by necessity due to governmental or industry regulations and certifications, we run into a key issue which is the difficulty in handling vulnerabilities at scale.
Vulnerability Exploitability eXchange (VEX) documents have become a critical part of modern software security practices. As organizations increasingly rely on Software Bill of Materials (SBOMs) to gain transparency into the components of their software, managing vulnerabilities has grown more complex. SBOMs provide a detailed list of all open-source libraries, dependencies, and third-party components within an application. However, while SBOMs are essential for identifying potential vulnerabilities, they often result in overwhelming lists of issues that may not be directly exploitable. This is where VEX documents come in—they act as a filter, providing actionable information about whether a vulnerability in a component is actually exploitable in a specific context.
The shift from simply identifying vulnerabilities to assessing their real-world impact has driven the rise of VEX. Without VEX documents, security teams would be forced to investigate each vulnerability individually, regardless of its exploitability, leading to resource drain and inefficiencies. VEX helps narrow the focus to only the vulnerabilities that present genuine risks, enabling better prioritization and more effective mitigation strategies. In the evolving landscape of software security, the combination of SBOMs and VEX has become a powerful duo, helping organizations shift from broad awareness to targeted action.
While the idea of VEX documents is great, generating them can be a very challenging task, but it doesn’t have to be!
Trying to figure out what is actually exploitable is a challenging task, but with some of the innovations in the industry especially eBPF, we can make this task much more of a realistic problem to solve.
KubeScape which is an amazing Open Source project which is a CNCF Sandbox project, has an amazing feature which allows for auto generation of VEX documents for all of our applications running in our clusters!
The core functionality is using another CNCF Sandbox project called Inspektor Gadget which is used by KubeScape as a library. Inspektor Gadget is an Open source eBPF debugging and data collection tool for Kubernetes and Linux which is used by KubeScape within its node-agent daemonset to collect the needed data for generation of the VEX documents.
The way this works is that the node-agent using eBPF probes, looks at the file activity of every running container. When a pod starts up on a node, the node-agent watches all of its containers for a “learning period” and saves the data in an activity log. In addition to this container images used within pods are automatically scanned by KubeScape using the Grype image scanner which also outputs an SBOM. KubeScape then uses the SBOM and the activity log of what is actually being used within the container as the inputs to generate automatically a VEX document for us which is then saved to a Custom Resource we can use for any security measures we need.
There is also a great example showing how this can be used in CI using GitHub actions which can be found in the following repo. While the example is just an example, which can definitely be improved by for example adding in other elements like signing of the vex document or adding it to your OCI Registry using the new OCI 1.1 referrers API specification, it offers an amazing starting point!
Currently the VEX generation feature in KubeScape is experimental and therefore is not enabled by default, but to add it is extremely easy. We simply need to add the following flag to our helm install command of the operator “–set capabilities.vexGeneration=enable”
For more detailed information i recommend checking out the official docs on this feature, and trying it out in your own environments!
VEX Auto generation is a perfect example of implementing a Shift Down approach which is a crucial element of any successful Internal Developer Platform.