Workload Cluster Operator - GitOps Deployment History & Predictive Failure Detection

Problem Statement

When a SylvaWorkloadCluster deployment fails, the operator only shows Ready = false. There is no information on what went wrong, which resource in the chain failed, or which Git commit caused it.

Investigating requires admin access to the management cluster - access that most workload teams do not have. And since failures are only visible after they happen, there is no way to catch a bad deployment early

Description

The project adds two capabilities on top of the existing SylvaWorkloadCluster operator:

  • Deployment History - records how the cluster state changed over time, with each change linked to the Git commit that triggered it. Workload teams get a clear, read-only view of what broke and when, without needing management cluster access.
  • Predictive Failure Detection - a lightweight model trained on past deployment history that scores a live deployment as Low, Medium, or High risk. The risk score appears on the SylvaWorkloadCluster status before Ready flips, giving teams an early warning.

Both run entirely in-cluster with no data leaving the environment.

Project Details

Leader: Orange (Priya Goyal)

List of people/organization interested to join:

*

Related repositories: * workload-cluster-operator * sylva-units-operator

Hackathon Objectives

  • Implement a controller that watches SylvaWorkloadCluster status changes and stores correlated snapshots in a WorkloadClusterHistory custom resource, linked to the triggering Git commit.
  • Define a minimal read-only RBAC profile so workload teams can view their cluster history without management cluster admin rights.
  • Build and bundle a synthetic training dataset representing healthy and failing deployment patterns to bootstrap the predictor from day one.
  • Train a classifier on the combined dataset and add a predictionRisk field (Low / Medium / High) on the SylvaWorkloadCluster status.
  • Validate end-to-end on a live management cluster: push a breaking commit, confirm it is recorded in history with the correct commit reference, and verify the risk score rises before Ready flips.