What has changed
The initial feedback regarding the Backstage plugins we built which can be found here has been really great especially for the Crossplane related plugins!
While the initial proof of concept work showed potential, it was heavily in need for better performance, enhanced functionality, and better integrations between the different parts of the stack.
We have been hard at work making some key needed changes and in this post I want to discuss some of the key changes we have made, and what these mean practically.
Performance Improvements
We have reevaluated and improved the pulling of data from Kubernetes clusters to now work at nearly 10x the speed due to more exact and targeted API calls against our clusters, making the UX much smoother. We have also added in partial rendering of data as it comes in instead of the previous behavior where we waited for all data before anything was shown. This gives a much more fluid user experience.
Additional Resource Data
While initially the plugin would show the managed resources, the Claim and the Composite resources, many times we also want to see the additional resources which are relevant to this claim. For this we have added an additional table which provides the same visibility into the relevant XRD, Composition, Providers, and Functions. We have also added the provider-config being used for each MR into the Managed resources table to give a better picture at a glance.
Crossplane Overview Card
While the Crossplane frontend plugin still provides the 2 tabs with tabular and graph based visualizations, sometimes a birds eye view is all that is needed. This is now available as well via a simple card with overview data which can be added to the components overview page with basic data around the claim, its status, and general information. With this overview card, we can make the UX much better and streamline the developer experience by providing upfront data about the status of there Crossplane resources without them needing to deep dive and switch tabs.
Kyverno Integration
One of the tools commonly used alongside Crossplane is Kyverno which is an amazing Policy Management tool for Kubernetes. We have built a dedicated Kyverno Policy Report plugin which can visualize Kyverno Policy Reports related to the Kubernetes resources of a component directly in backstage. We have also enhanced the plugin to have better and more streamlined integration with Crossplane Claim backed components, making the User experience much smoother and streamlined. Not only can a developer create Crossplane resources via Backstage using the auto generated Software Templates for each XRD using the Kubernetes Ingestor plugin, and also visualize their Crossplane Claims and underlying resources via the Crossplane Resources plugin, they can now also receive insights and clear visibility into policy violations and policy adherence data directly in Backstage via the Kyverno Policy Reports plugin! We have also added an Overview card for this plugin, making an even more full and cohesive birds eye view possible for Crossplane Claims for any component.
Day 2 Updates
While the Kubernetes Ingestor Plugin auto generates Software Templates for creation of Crossplane Claims, We all know that day2 operations and updates is the true challenge which needs to be tackled. Day0 and Day1 tasks can easily be streamlined but the Day2 maintenance in Backstage is still a story in the making. This is where the new Crossplane Claim Updater plugin comes into play. The new plugin is based on another plugin we released called the Entity Scaffolder Content plugin. The use of these plugins allows us to embed the Backstage Scaffolder in a tab on a component and provide contextual data as a starting point for filling out the forms of a software template. The Claim Updater, via the creation of a custom field extension for the scaffolder along with a provided Software Template, allows a user to request to update a claim manifest. When the user runs this flow, the plugin will retrieve the latest schema of the resources definition based on the XRDs OpenAPI Schema, it will then pull in the existing manifest from GitHub with the current values set by the user, generate a form based on the OpenAPI schema, apply the current values from git in the form, and then allow the user to make any changes needed. When the user submits the form, a new PR is created in the GitOps repo, with the new desired state. Once merged the traditional GitOps tools like Flux CD or Argo CD will pickup the changes and update the claims within your Kubernetes clusters!
Going Beyond Crossplane
While Crossplane is in my opinion the best mechanism for building custom APIs and to build service offerings within a platform, other tools exist in this area like KRO, Kratix, and KubeVela, as well as many purpose built Kubernetes Operators, which we also want to manage the lifecycle of those entities via Backstage. While full support for generic CRDs is not yet fully built out, and has many challenges in terms of design and possibilities, we have begun to extend the capabilities of the plugins to add support for Generic CRDs as well. The first part of this integration is now completed with the ability to now generate Software Templates for any CRD, and also the Claim Updater plugin has been extended to also support the day2 management of custom resources not created via Crossplane claims. Currently the visualization aspects that we have for Crossplane claims does not exist for generic CRDs, but we are looking into this area as well, and how this can be achieved. Because clusters typically have many CRDs, and we don’t want to offer all of our CRDs as Software Templates, we have added the ability to provide a static list of CRDs via the app-config.yaml as well as the ability to provide a label selector where any CRD with this label will have a Software Template auto-generated for it. These CRs will also be automatically added as components into the software catalog, just like claims, and other core Kubernetes resources.
Additional Plugins
While the main focus on this post is around Crossplane related plugins, I also want to call out the additional plugins we have released in the same repo with integration of DevPod, for easy launching of remote dev environments using DevPod in your IDE of choice directly from Backstage, as well as a plugin to bring in data from ScaleOps, the best workload rightsizing tool for Kubernetes on the market.
Final Thoughts
These plugins are still in there early stages, and still need polishing, and are missing features, but they already prove to be very valuable and are paving a path for being a great building block for building an amazing IDP! If you havent seen the video that Viktor Farcic did on these plugins I strongly recommend checking out this video and you can also checkout a demo I did as part an episode of You Choose, where it was a battle between Backstage and Port for the peoples choice for an Internal Developer Portal which can be seen at this link.