Software Delivery Guideline

Abstract

This document provides comprehensive guidelines for Network Function vendors on how to deliver their software efficiently to Communication Service Providers (CSP). It covers the preparation, delivery, and integration of software artifacts, ensuring compliance with our standards and facilitating seamless integration into our systems.

Keywords: OCI, Helm, GitOps, Kubernetes, Registry, Docker, Continous Delivery, Continous Deployment, CI/CD

date: 2025-02-26

contributors:

  • name: Morgan Richomme (editor) / Orange
  • name: Josua Hiller / Swisscom
  • name: Alberto Garrote Montes / Deutsche Telekom
  • name: Jéremy Weber / Swisscom
  • name: Manisha De / Swisscom
  • name: Adrian K / Swisscom
  • name: David Rodriguez Pereira / Swisscom

Support:

  • Orange
  • Swisscom
  • Deutsche Telekom

Why guidelines for Network Function software delivery

  • Faster and more efficient deployments by reducing time, effort, and costs in software delivery.
  • Seamless integration with CSP systems, minimizing risks and ensuring smooth deployments.
  • Enhanced scalability to efficiently manage growing software volumes and deployments.
  • Reduced time-to-market (TTM) for new features and services.
  • Improved software quality through frequent and secure updates.
  • Early insights to refine current and future products.
  • Secure software delivery
  • Support for incremental releases, enabling continuous innovation.

Preparation of Software Artifacts

Format

Artifacts must be delivered via OCI (Open Container Initiative)-compliant repositories. This ensures compatibility and standardization across different platforms and tools

Sylva

Sylva Requirements:

  • OCI Registry Compliant: Ensure that the registry is OCI Spec compliant.
  • OCI Images: Follow the OCI image specifications.
  • Helm Charts: Use Helm charts for packaging and deploying applications.
  • Metadata: Include metadata as generic files in an OCI image.
  • Push and Pull Support: Support both push and pull mechanisms. Implement a notification mechanism for pull operations.
  • No Custom API: Avoid any required scripting on the consumer end.
  • Semantic Versioning: Align product versions with Helm chart versions using semantic versioning (e.g., Packet core).
  • GitOps-ready: FluxCD or similar tool has to be able to deploy the workload with the appropriate configuration.
  • Ensure that artifacts delivered with Cloud Native Functions (CNF) (e.g., Helm charts) are customizable for efficient multi-purpose deployments.

Security

Ensuring the security of OCI images and Helm Charts is paramount in maintaining a robust and reliable deployment environment. Vendors must take necessary measures to safeguard their OCI images and Helm Charts, emphasizing the importance of regular vulnerability scans, prompt remediation of critical issues, and adherence to best practices in container and Kubernetes security.

To reduce the risk of supply chain attacks, it must be possible to verify integrity by hash and trust OCI images and Helm Charts through a signature.

OCI Image

  • Vendors must ensure that OCI images are secure.
  • Vendors should perform regular scans to check for CVEs in existing OCI images.
  • No Critical Common Vulnerabilities (CVEs) should be present at the time of release.
  • New images must be generated promptly to address Critical and Major CVEs.
  • New images should be tagged appropriately. Using Semantic Release is recommended.

Helm Charts

  • Vendors should use the principle of least privilege when deploying CNFs.
  • Avoid node access, running containers as root, or using dangerous capabilities unless required by the CNF being deployed.
  • If special node access, capabilities, or deviations from Kubernetes or container security best practices are necessary, vendors must provide proper documentation to identify and justify such use cases.
  • Ensure that the Helm charts are customizable for efficient multi-purpose deployments.

Helm Format

Ambiguous Helm Charts are frequently detected. Depending on the custom values and the defined templates, the generated manifests may include duplications of headers or annotations, which can be problematic if such annotations are used in dynamic processing, such as scaling. While these duplications are ignored by the Helm YAML parser, they may trigger errors in GitOps engines supporting Kustomize v4. See the section dealing with Duplicate YAML keys for details.

Sylva

The Sylva community recommends the following to address these issues:

  • Create Helm chart validation pipelines on the vendor side using a YAML parser linter to harden the Helm charts and avoid recurrent duplication key issues.
  • Ensure the resulting manifests are YAML-compliant.
  • Use a recent version of Helm (approximately 6 months old) in the validation pipeline to perform these tasks.

The vendor charts must follow the Helm best practices.

Image Efficiency

Vendors shall pay attention to the image layering. Very heavy images are frequently observed. It triggers security issues and operational efficiency issues.

Tools like Dive shall be used in order to improve the efficiency of the images. An efficiency below 95% is not acceptable.

Delivery of Software Artifacts

Sylva

The Sylva community recommends to align the OCI artifact management with the IT industry.

  • Vendor shall manage an OCI registry accessible from the Internet and protected by a RBAC policy based on renewable tokens or revokable accounts.
  • GitOps users shall be able to setup proxy/cache registry connected to the vendor registry. It should allow us, consumers, to retrieve Container Image layers, blobs, and manifest files on an on-demand basis.
  • Proper documentation and inventory for these shall also be available for the consumer, including comprehensive documentation on how to deploy the software and update to a software version.
  • Provide notification solution to notify the target environment when new content or updates are available.
  • Provide solution that allows the target environment to fetch a document describing the entire release/bundle of artifacts.
  • Support for a trade check model for cloud-native software distribution and adherence to telecommunications laws is required.
  • Standard Signature mechanisms on the images shall be implemented.
  • Machine-readable (SPDX or CycloneDX format) up to date SBOM. These SBOM shall be compliant with NIS2 regulation
  • A list of vulnerabilities known when the images are published and shared

Integration with CSP-Internal Automation Pipeline

The CSP-internal automation pipeline should be allowed to hook into the vendor software delivery solution. This should enable automatic subscription to CNF releases and seamless integration.