If you’ve ever had to evolve a Kubernetes CRD’s schema — rename a field, split one field into two, change a type — you’ve probably run into the same wall I did: the only supported way to do it safely is a conversion webhook, and the only supported way to build one is to write it yourself, typically in Go, as a standalone service you now own forever.
That’s true whether your CRD is a plain native CustomResourceDefinition or a Crossplane CompositeResourceDefinition (XRD). Multiple served versions are a first-class, well-supported feature. Converting between them is where the tooling stops and “write your own webhook” begins.
So I built declarative-conversion-operator — a Kubernetes operator that replaces that hand-written webhook with a declarative custom resource. You describe what changed between versions using a small vocabulary of named strategies; the operator validates it, compiles it, and serves it from a shared, horizontally-scalable webhook runtime it manages for you.
Status: alpha, under active development. APIs (the CRDs, the Helm chart’s values, CLI flags) may still change. I’m building this in the open and would genuinely like feedback — see the repo for how to reach out or file an issue.
The problem, concretely
Say your Crossplane XRD’s v1 has this:
spec:
storageGB: "100"
and after some API design regret, v2 looks like this instead:
spec:
storage:
size: "100Gi"
Both versions are served. Every client — kubectl, controllers, other automation — needs to be able to read and write the resource at whichever version it understands, and get back a correctly-shaped object regardless of which version is actually stored. That’s exactly what a conversion webhook is for, and exactly why Kubernetes lets a CRD point spec.conversion.webhook at one.
Writing that webhook by hand means: a new deployable service, a ConversionReview HTTP handler, TLS wired up correctly (cert-manager, usually), and — the part that actually matters — a bunch of field-by-field logic you have to get right in both directions, for every version pair, forever, as the schema keeps evolving. Get it subtly wrong (drop a field, mis-handle a rename) and the failure mode is silent data loss on real user resources, not a compile error.
That’s a lot of bespoke, easy-to-get-wrong code for a problem that’s almost always the same handful of shapes: a field got renamed, a scalar got wrapped in an object, an enum’s values changed, a list became a map. Those shapes are enumerable. They don’t need custom Go code every time — they need a name and a couple of parameters.
What it looks like instead
Here’s a real (trimmed) example — an XRDConversionConfig for the schema change above:
apiVersion: terasky.com/v1alpha1
kind: XRDConversionConfig
metadata:
name: xpostgresqlinstances-conversion
spec:
targetXRD:
name: xpostgresqlinstances.database.example.org
hubVersion: v2
spokes:
- version: v1
rules:
- strategy: FieldRename
fieldRename:
hubPath: spec.storageGB
spokePath: spec.storageSize
kubectl apply that, and once it validates cleanly the operator patches the real XRD’s spec.conversion to route through a webhook server it manages — no Go code, no separate deployment to babysit.
CRDConversionConfig does the exact same thing for plain native CRDs, with the exact same rule vocabulary. If you’re not using Crossplane at all, or using both native CRDs and Crossplane side by side, both are independently toggleable.
The strategy vocabulary
There are 16 built-in strategies covering the shapes that actually come up when a schema evolves:
fieldRename, scalarToObject / objectToScalar, singletonArrayToObject / objectToSingletonArray, fieldsToMap / mapToFields, toAnnotation / toLabel, enumRemap, defaultValue, constant, delete, jsonPatch (an escape hatch for anything else), forEach (per-array-element rules), typeCoerce, scalarToFields / fieldsToScalar, arrayToMapByKey / mapToArrayByKey, numericScale, listJoin / listSplit.
A couple worth calling out:
numericScale— rescale a number by a fixed factor (e.g. megabytes stored, gigabytes displayed):hubValue == spokeValue * factor.arrayToMapByKey/mapToArrayByKey— the classic “list-map vs. map” API-evolution pattern. Array→map is lossless; map→array is treated as lossy, since the reconstructed array is sorted by key rather than reproducing the original order.jsonPatch— an explicit escape hatch. When a conversion genuinely doesn’t fit any named strategy, you can drop to raw JSON Patch — the engine can’t statically verify what it does, so it requires you to explicitly acknowledge that.
Rules apply to status.* fields exactly the same way they apply to spec.* — a conversion webhook receives the whole stored object, status included, and this operator doesn’t treat it as a special case.
Fail-closed by default
The part I care about most isn’t the strategy list, it’s the default posture: any field the engine can’t prove is handled — either by an explicit rule or because it’s structurally identical on both sides — is a validation error, not a silent pass. And any rule the engine determines is lossy in either direction is rejected unless you explicitly set acknowledgeLossy: true (with an optional reason, so the intent is on record).
Concretely, that means:
- A field present in one version’s schema and absent from the other, with no rule mapping it, fails validation. It never gets to production as a quiet data-loss bug.
- A rule that can’t be proven lossless — say, an
enumRemapwhere two hub values map to the same spoke value — has to be acknowledged explicitly. You can still do it; you just can’t do it by accident. - The live XRD/CRD is never touched until the whole config validates, the target resource is healthy, and the assigned webhook server is confirmed ready. And deleting a config won’t silently revert to
strategy: Noneif that would strand clients on a non-storage version — that’s blocked behind an explicit break-glass annotation.
Testing it before it ever touches a cluster
This is the part that actually sold me on building this as an operator instead of a one-off script: the exact same conversion engine that runs in the admission-webhook hot path is also packaged as a standalone CLI, convctl, that runs entirely offline against local YAML files.
convctl validate --config config.yaml --xrd xrd.yaml
convctl analyze --xrd xrd.yaml --config config.yaml
convctl test --xrd xrd.yaml --config config.yaml --samples ./samples/
convctl test round-trips every sample object through every served-version conversion path (via the hub) and tells you exactly which field diverged, between which two versions, and whether that loss was acknowledged. Here’s real output, trimmed, from a config with a genuine unacknowledged bug in it:
XRD Conversion Test Report
XRD: xwidgets.e2e.example.org Config: xwidgets-e2e-conversion (hub: v3)
Samples: 3 Paths tested: 9 Total time: 21.6ms
SAMPLE PATH RESULT FIELDS TIME(µs) RULES MATCHED
cluster:default/e2e-widget-v1 v3→v3 PASS 63 9 (identity)
v3→v2 PASS 64 138 v2:rule[9]:TypeCoerce,v2:rule[11]:NumericScale,...
v3→v1 PASS 64 138 v1:rule[0]:SingletonArrayToObject,...
ISSUES (1)
SAMPLE FIELD FROM → TO TYPE DETAIL
cluster:default/e2e-widget-v1 spec.priority v3 → v2 unacknowledged-loss round-trip mismatch; no rule declares this field lossy
RULE COVERAGE
v2:rule[9]:TypeCoerce matched 3 sample(s)
v2:rule[11]:NumericScale matched 3 sample(s)
v1:rule[6]:ForEach NOT EXERCISED by any sample (warning)
SUMMARY: 3 samples, 9 paths — 8 PASS, 0 LOSS(acknowledged), 1 FAIL(unacknowledged loss), 0 ERROR
Exit code 0 if everything passed or only had acknowledged loss; 1 if there’s an unacknowledged loss or a failure; 2 for a usage error. That’s a clean gate for CI — fail the pipeline before a bad conversion config ever gets near a cluster.
--live: test against everything that already exists
--samples is for hand-written fixtures, which is fine for coverage of the shapes you thought of. The thing that actually catches surprises is --live:
convctl test --xrd xrd.yaml --config new-config.yaml --live
Instead of fixture files, this fetches every existing instance of the target resource type from a real cluster (at its hub/storage version — so it works even before any conversion webhook is wired up) and runs the exact same round-trip test against real, messy, production data. It only needs get/list on the target resource type — no write access, nothing related to this operator’s own CRDs. This is the check I’d run before ever applying a new or changed config against anything that matters: does this hold up against what’s actually out there, not just what I imagined?
How it fits together
┌─────────────────────────┐
kubectl apply ──► │ XRDConversionConfig │
└────────────┬─────────────┘
│ validated by
▼
┌─────────────────────────┐ ┌──────────────────────────┐
│ operator (cmd/manager) │──SSA──►│ target XRD │
│ pkg/engine.Analyze() │ patch │ spec.conversion.webhook │
└────────────┬─────────────┘ └────────────┬─────────────┘
│ resolves assignment │ ConversionReview
▼ ▼
┌─────────────────────────┐ ┌──────────────────────────┐
│ ConversionWebhookServer │───────►│ cmd/webhook-server pods │
│ (Deployment/Service/…) │ owns │ in-memory plan registry │
└─────────────────────────┘ └──────────────────────────┘
Three CRDs total: XRDConversionConfig and CRDConversionConfig (the declarative rule sets, one per target resource) and ConversionWebhookServer — a deployable, independently-scalable instance of the shared webhook runtime, with its own Deployment, cert-manager Certificate, Service, HorizontalPodAutoscaler, and PodDisruptionBudget. Create more than one for scale-out or tenant isolation; the Helm chart gives you a default one out of the box.
Every rule gets precompiled into a resolved-path execution plan at reconcile time, so the webhook’s hot path — every ConversionReview on the API server’s admission critical path — does zero YAML parsing or schema lookups at request time. It just runs the plan.
Try it
helm install declarative-conversion-operator charts/declarative-conversion-operator \
--namespace declarative-conversion-system --create-namespace
kubectl wait --for=condition=Available conversionwebhookserver/default --timeout=120s
kubectl apply -f config/samples/terasky_v1alpha1_xrdconversionconfig.yaml
kubectl get xrdconversionconfig xpostgresqlinstances-conversion -o yaml
There’s also a make dev-up target that stands up a full disposable local environment — a kind cluster, cert-manager, Crossplane, and this operator, all built from your local checkout — if you want to poke at it without touching a real cluster.
- Repo: github.com/terasky-oss/declarative-conversion-operator
- Docs:
docs/— architecture, CLI reference, per-strategy pages, an FAQ - License: Apache 2.0
It’s early. The CRDs, Helm values, and CLI flags can still change, and it hasn’t run in production anywhere yet — mine included. If you’re staring down a CRD version migration and hand-writing a webhook feels like the wrong amount of effort for the problem, I’d love for you to try it and tell me where it breaks.