In TAP 1.4, VMware added an additional optional package to TAP for the External Secrets Operator, and in TAP 1.5 they have enhanced the integration.
What Is External Secrets Operator (ESO)
The External Secrets Operator is a Kubernetes operator that integrates with external secret management systems, for example, AWS Secrets Manager, Google Secrets Manager and Hashicorp Vault. It reads information from external APIs and automatically injects the values into Kubernetes secrets.
ESO In TAP
ESO is available as a package in the TAP package repository, but is not included in any of the profiles.
ESO can be utilized in TAP for many use cases, including for TAP installation itself, using the new GitOps installation method!
You can also utilize ESO for simplifying the management of secrets which are needed for Service Bindings, Git credentials or registry credentials just to mention a few common cases. It can also be very helpful when working with a scanner other than vault within TAP such as Snyk, Carbon Black, Prisma or Aqua where we need to provide credentials for running the scans in every developer namespace.
ESO is not integral to TAP at this point, and there is no dependency on it, but any environment that already has a secret management system can gain huge value from this integration.
What’s New In TAP 1.5
In TAP 1.5, we now have 3 key update to ESO:
- ESO is now a Beta component (previously alpha) in TAP
- Integration with TAP Installation flow using a GitOps methodology
- External Secrets plugin for Tanzu CLI
Let’s see what these updates add to the picture
ESO is now a Beta component (previously alpha) in TAP
This is a great sign, showing that the direction towards deeper integration, and full on GA status is moving in the right direction. ESO can bring a lot to the table for a platform like TAP, and it is great to see VMware integrating with ESO more and more with each release.
Integration with TAP Installation flow using a GitOps methodology
I have written a dedicated post regarding the GitOps installation option, and what that entails, so for here what I will state in short is that the new installation method allows us to split our TAP Values into 2 separate files, one with non sensitive values, and the other, with sensitive values.
The sensitive values are then stored as a secret in your external secrets management system, and at deployment time, The ESO package will be deployed and configured and the values will be populated into a kubernetes secret, which is in turn used, alongside the non sensitive values file which is pulled down from Git, as the values for the TAP installation.
This method offers a great UX, and a simple lifecycle management story for TAP, utilizing industry standard approaches like GitOps, while seamlessly integrating security at the forefront, utilizing ESO, as an agnostic integration layer, with nearly all common secret management solutions on the market today.
External Secrets plugin for Tanzu CLI
One of the main things VMware have done with Tanzu, is to build a full end to end cloud native portfolio, and not just a single tool. As such, a unified entrypoint is needed, and for Tanzu this can be found in the Tanzu CLI.
While not all Tanzu products are integrated yet into the Tanzu CLI, much work is happening to add more and more of the products into this model, where we can have a single CLI entry point to the entire Tanzu ecosystem, and for each tool, a set of dedicated or shared plugins will be available in order to support the relevant use cases.
This is already the case with TKG and TAP, and TMC have already announced that soon they will be moving to a Tanzu CLI based interface instead of the standalone TMC CLI which currently exists.
As part of this, TAP has since the beginning had multiple Tanzu CLI plugins which are dedicated for TAP, as well as TAP utilizing some of the core shared Tanzu CLI plugins for its needs.
In TAP 1.5, a new CLI plugin was added for ESO which can be accessed simple via the Tanzu CLI sub-command "external-secrets" or for those like me that hate typing long words 2 aliases have been added for the command "es" and "eso".
With the current CLI we can interact with 3 resource types, which are part of ESO installation:
Let’s see what we can do with this new CLI plugin.
We can create resources using the Tanzu CLI very easily.
For example, to create a secret store targeting vault, using the kubernetes authentication method, one could run a command like:
cat <<EOF | tanzu external-secrets stores create -f - apiVersion: external-secrets.io/v1beta1 kind: SecretStore metadata: name: vault-secret-store spec: provider: vault: server: "https://vault.vrabbi.cloud" path: secret version: v2 auth: kubernetes: mountPath: kubernetes role: demo-vault-role serviceAccountRef: name: my-sa EOF
And to create an external secret we can follow the same approach:
cat <<EOF | tanzu external-secrets secret create -f - apiVersion: external-secrets.io/v1beta1 kind: ExternalSecret metadata: name: vault-example spec: refreshInterval: "15s" secretStoreRef: name: vault-secret-store kind: SecretStore target: name: example-sync data: - secretKey: foobar remoteRef: key: secret/foo property: my-value EOF
We can also list all the relevant ESO resources and see their status for example:
Getting all Secret Stores:
$ tanzu external-secrets store list -A NAMESPACE NAME PROVIDER STATUS default vault-secret-store Hashicorp Vault Valid demo-app vault-secret-store Hashicorp Vault Valid scott-dev vault-secret-store Hashicorp Vault Valid
We can all list all external secrets:
$ tanzu external-secrets secret list -A NAMESPACE NAME SECRET NAME STORE REFRESH INTERVAL STATUS LAST UPDATED LAST REFRESH default git-creds registry-secret vault-secret-store 15m SecretSynced 18s 10m default registry-creds registry-secret vault-secret-store 15m SecretSynced 18s 10m demo-app git-creds registry-secret vault-secret-store 15m SecretSynced 7s demo-app registry-creds registry-secret vault-secret-store 15m SecretSynced 7s 10m scott-dev aqua-creds registry-secret vault-secret-store 15m SecretSynced 35s 10m scott-dev git-creds registry-secret vault-secret-store 15m SecretSynced 2s 10m scott-dev registry-creds registry-secret vault-secret-store 15m SecretSynced 63s 10m
This is all very nice, but what i think is probably the most useful command so far, is the ability to trigger a sync of a secret manually. While ESO has a sync interval at which it will check if there is an updated value in the configured secret store, and if there is, will update the generated secret, sometimes we want to trigger a sync on demand.
To trigger the sync of a secret manually, one can simply run for example:
$ tanzu external-secrets secret sync aqua-creds -n scott-dev
ESO is a great open source project, and it is great to see that once again, the VMware team, are leveraging the tremendous ecosystem out there in the kubernetes space, and enhancing the UX around these tools by seamlessly integrating them into TAP, without falling down the common pitfall of forking the project. In this case, like nearly all other key components in TAP, VMware wrap the OSS solutions into a cohesive end to end solution, abstracting away the complexities and intricacies of the individual tools for end users, but still enabling advanced users to utilize the full capabilities of these capabilities and to so called "break glass" when needed.
I am truly excited to see the deeper integration with ESO in TAP 1.5, and am excited to see what comes next!