|
| 1 | +# AGENTS.md |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +This is a Helm umbrella chart repository for **MLRun Community Edition (CE)** — an open-source MLOps stack. The main chart lives at `charts/mlrun-ce/` and bundles: Nuclio, MLRun, Jupyter, MPI Operator, SeaweedFS (S3-compatible storage), Spark Operator, Kubeflow Pipelines, Prometheus stack, TimescaleDB, and Strimzi Kafka Operator. |
| 6 | + |
| 7 | +## Commands |
| 8 | + |
| 9 | +```bash |
| 10 | +# Lint the helm chart (requires helm and chart-testing `ct` installed) |
| 11 | +make helm-lint |
| 12 | + |
| 13 | +# Update sub-chart dependencies (run before lint or packaging) |
| 14 | +make helm-update-dependencies |
| 15 | + |
| 16 | +# Add all required helm repos from requirements.yaml |
| 17 | +make helm-repo-add |
| 18 | + |
| 19 | +# Package the chart as a tarball |
| 20 | +make package |
| 21 | + |
| 22 | +# Run full local end-to-end test on a Kind cluster (requires docker, kind, kubectl, helm) |
| 23 | +./tests/kind-test.sh full # Create Kind cluster + install chart |
| 24 | +./tests/kind-test.sh create # Create cluster only |
| 25 | +./tests/kind-test.sh install # Install chart (assumes cluster exists) |
| 26 | +./tests/kind-test.sh verify # Verify installation |
| 27 | +./tests/kind-test.sh delete # Delete Kind cluster |
| 28 | +CLEANUP_ON_EXIT=true ./tests/kind-test.sh # Auto-cleanup after test |
| 29 | +``` |
| 30 | + |
| 31 | +## Architecture |
| 32 | + |
| 33 | +### Chart Structure |
| 34 | + |
| 35 | +`charts/mlrun-ce/` is an **umbrella chart** that: |
| 36 | +1. Declares sub-chart dependencies in `requirements.yaml` (pulled from external Helm repos) |
| 37 | +2. Provides additional Kubernetes resources via `templates/` that the sub-charts don't include |
| 38 | +3. Wires all components together through shared `values.yaml` |
| 39 | + |
| 40 | +### Template Organization (`charts/mlrun-ce/templates/`) |
| 41 | + |
| 42 | +- `config/` — ConfigMaps and Secrets shared across components: MLRun env config, Jupyter env config, S3 credentials secret, Pipelines config, Spark config, Grafana dashboards |
| 43 | +- `seaweedfs/` — SeaweedFS-specific resources: S3 IAM config secret, bucket init job, admin UI NodePort service, ingress |
| 44 | +- `kafka/` — Kafka Strimzi custom resources: KafkaNodePool, Kafka cluster CR, bootstrap alias Service, RBAC, NetworkPolicy |
| 45 | +- `timescaledb/` — TimescaleDB Deployment, Service, PVC |
| 46 | +- `jupyter-notebook/` — Jupyter Deployment and supporting resources |
| 47 | +- `pipelines/` — Kubeflow Pipelines resources |
| 48 | +- `persistency/` — PVC definitions |
| 49 | +- `aws/` — AWS-specific resources |
| 50 | + |
| 51 | +### Key Design Patterns |
| 52 | + |
| 53 | +**S3 credentials propagation**: The top-level `s3.accessKey`/`s3.secretKey`/`s3.bucket` values flow into a `s3-credentials` Secret (created by `templates/config/s3-credentials-secret.yaml`), which is then mounted via `envFrom` in MLRun API and Jupyter pods. SeaweedFS uses the same credentials via the `seaweedfs-s3-config` Secret. |
| 54 | + |
| 55 | +**Global registry anchor**: `global.registry: &userRegistry` in `values.yaml` uses YAML anchors to multiplex the same docker registry config to both `nuclio.global.registry` and `mlrun.global.registry`. |
| 56 | + |
| 57 | +**SeaweedFS as S3 backend**: SeaweedFS replaced MinIO. The helpers in `_helpers.tpl` (`mlrun-ce.s3.*`) generate the SeaweedFS service URL. Legacy `mlrun-ce.minio.*` helpers are kept as aliases pointing to the SeaweedFS helpers. |
| 58 | + |
| 59 | +**Component enable/disable**: Most components can be disabled via `<component>.enabled: false`. The Kafka setup requires the Strimzi operator (deployed as a sub-chart via `strimzi-kafka-operator`) and custom Strimzi CRs in `templates/kafka/`. |
| 60 | + |
| 61 | +### Values Files |
| 62 | + |
| 63 | +- `charts/mlrun-ce/admin_installation_values.yaml` — admin install |
| 64 | +- `charts/mlrun-ce/non_admin_installation_values.yaml` — non-admin install |
| 65 | +- `charts/mlrun-ce/non_admin_cluster_ip_installation_values.yaml` — non-admin with ClusterIP |
0 commit comments