Enhancing GitOps with Keptn for Autonomous Operations

Problem Statement

To enable autonomous operations, the existing GitOps framework must expand to accommodate closed-loop automation without compromising core GitOps principles.

Currently, Cloud Native (CN) workloads and their operating environments lack the integrated metrics and control loops needed for self-management. Implementing autonomous operations capabilities necessitates extending and enhancing existing GitOps work-flows. Incorporating structured data flows following the pattern metrics input --> inferencing logic --> configuration action allows for adding closed control-loops to existing K8s-FluxCD-GIT environments, thusly enabling automomous operations of CN workloads.

Description

Develop a Kubernetes Operator that compiles the logic of suitable Keptn tasks from sets of largely predefined configuration items (metrics, baseline values, decision making logic, etc.) so as to create the 'closed control loops' facilitating the autonomous operations of CN workloads.

Project Details

Leader: Deutsche Telekom (M. Sewera)

List of people/organization interested to join:

  • n.n.

Consultancy relating to Keptn provided by:

Use Cases

Automate Post-Deployment Functional and Performance Verification for 5GC Software (suggested by Telekom Deutschland)

After deploying a new 5G Core (5GC) software release via the GitOps pipeline, we want to ensure the network functions are fully operational and handling traffic correctly. We want to accomplish this by using Keptn to orchestrate an automated validation process that bridges the gap between basic Kubernetes pod health and 5G protocol-level verification through the following three-step verification loop:

  • Triggering the Functional Tests (Pre-Conditions): Once the GitOps controller rolls out the updated AMF, SMF, or UPF pods, Keptn intercepts the deployment process before it is finalized. Keptn automatically executes localized task definitions to trigger specific testing tools (i.e. traffic simulators or probes), forcing the network to execute end-to-end control and user plane signaling, including AMF registration, PDN session establishment, and user plane traffic forwarding via the UPF.
  • Evaluating 5GC KPIs (Quality Gates): While the test traffic is running, Keptn serves as an automated quality gate by directly querying the monitoring system (such as Prometheus). It utilizes declarative metrics evaluation definitions to scrape actual 5G network performance indicators over the test window, focusing specifically on session establishment rates, the percentage of successful session setups, and packet delivery success.
  • Automated Judgment & GitOps Rollback: Keptn analyzes the gathered performance metrics against strict, pre-defined SLA thresholds. If all 5G network KPIs meet the required quality criteria, Keptn approves the deployment and allows the GitOps state to finish successfully; if any protocol procedures or success percentages fall below the baseline, Keptn fails the quality gate, flagging the environment to trigger an immediate, automated GitOps fallback to the previous stable software version.

Project Elements

Project Lab

The project will require a cloud environment as a test-lab featuring among others tools such as Keptn V.2, Prometheus or OpentTel, and KubeBuilder.

The lab environment shall be set up and configured before the start of the Hackathon.

Keptn Task Design

A good portiton of the time alloted to the Hackathon will be spent on conceptualizing the closed control loop patterns required for enabling the autonomous deployment of selected cloud-native workloads. This work will yield specifications for Keptn tasks and observability related objects (data scraping agents, PromSQL queries, etc.)

Use of AI Platforms to Accelerate Coding and Testing

To ensure the delivery of demonstrable artefacts as proof-of-concept for the agreed Keptn task designs and related objects, the project team shall rely on AI tools such as Claude Code or Gemini for the creation of program code and associated test cases.

Hackathon Objectives

  • Show-case the use of Keptn 'pre-deployment' and 'post-deployment' tasks for making CN workload deployments predictable
  • Demonstrate the use of Keptn 'evaluation' tasks for identifying and subsequently mitigating CN workload operating deviations
  • Highlight how the Keptn enabled enhancement integrate seamlessly into existing GitOps work-flows