Manage Grafana Dashboards with GitOps Using ArgoCD

This guide shows you how to set up a continuous deployment pipeline using ArgoCD to synchronize your Grafana dashboards with a Git repository. You'll use the Grafana Dashboard Custom Resource provided by the Grafana Operator to manage dashboard configurations declaratively.

Prerequisites

• An existing Grafana Cloud stack
• A Kubernetes cluster with Grafana Operator installed
• ArgoCD installed on your Kubernetes cluster
• A Git repository to store your dashboard configurations

Set Up Your Git Repository

Create a directory structure in your repository to organize your Grafana and dashboard configurations. For this tutorial, create a folder named grafana.

Set Up the Grafana Operator

The Grafana Operator allows you to authenticate with the Grafana instance using the Grafana Custom Resource (CR).

Create the Grafana API Token Secret

Store the Grafana API Token in a secret. Create a file named grafana-token.yml in the grafana folder:

apiVersion: v1
kind: Secret
metadata:
  name: grafana-cloud-credentials
  namespace: '<GRAFANA_OPERATOR_NAMESPACE>'
stringData:
  GRAFANA_CLOUD_INSTANCE_TOKEN: '<GRAFANA_API_KEY>'
type: Opaque

Replace <GRAFANA_API_KEY> with your API key and <GRAFANA_OPERATOR_NAMESPACE> with the namespace where the grafana-operator is deployed.

Configure the Grafana Custom Resource

Set up the connection to your Grafana Cloud instance. Create a file named grafana-cloud.yml:

apiVersion: grafana.integreatly.org/v1beta1
kind: Grafana
metadata:
  name: '<GRAFANA_CLOUD_STACK_NAME>'
  namespace: '<GRAFANA_OPERATOR_NAMESPACE>'
  labels:
    dashboards: '<GRAFANA_CLOUD_STACK_NAME>'
spec:
  external:
    url: https://<GRAFANA_CLOUD_STACK_NAME>.grafana.net/
    apiKey:
      name: grafana-cloud-credentials
      key: GRAFANA_CLOUD_INSTANCE_TOKEN

Add Dashboards to Your Git Repository

In your grafana directory, create a sub-folder called dashboards. Below are three approaches to defining dashboards.

1. Simple Dashboard (Inline JSON)

Create simple-dashboard.yaml:

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDashboard
metadata:
  name: grafanadashboard-sample
  namespace: '<GRAFANA_OPERATOR_NAMESPACE>'
spec:
  resyncPeriod: 30s
  instanceSelector:
    matchLabels:
      dashboards: '<GRAFANA_CLOUD_STACK_NAME>'
  json: >
    {
      "id": null,
      "title": "Simple Dashboard",
      "tags": [],
      "style": "dark",
      "timezone": "browser",
      "editable": true,
      "hideControls": false,
      "graphTooltip": 1,
      "panels": [],
      "time": { "from": "now-6h", "to": "now" },
      "timepicker": {
        "time_options": [],
        "refresh_intervals": []
      },
      "templating": { "list": [] },
      "annotations": { "list": [] },
      "refresh": "5s",
      "schemaVersion": 17,
      "version": 0,
      "links": []
    }

2. Dashboard from ConfigMap

Create dashboard-from-cm.yaml. This approach separates the dashboard JSON into a ConfigMap, which is useful for larger dashboards:

apiVersion: v1
kind: ConfigMap
metadata:
  name: dashboard-definition
  namespace: '<GRAFANA_OPERATOR_NAMESPACE>'
data:
  json: >
    {
      "id": null,
      "title": "Simple Dashboard from ConfigMap",
      "tags": [],
      "style": "dark",
      "timezone": "browser",
      "editable": true,
      "panels": [],
      "time": { "from": "now-6h", "to": "now" },
      "schemaVersion": 17,
      "version": 0
    }
---
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDashboard
metadata:
  name: grafanadashboard-from-configmap
  namespace: '<GRAFANA_OPERATOR_NAMESPACE>'
spec:
  instanceSelector:
    matchLabels:
      dashboards: '<GRAFANA_CLOUD_STACK_NAME>'
  configMapRef:
    name: dashboard-definition
    key: json

3. Dashboard from Grafana.com

Reference a community dashboard by ID. Create dashboard-from-id.yaml:

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDashboard
metadata:
  name: node-exporter-latest
  namespace: '<GRAFANA_OPERATOR_NAMESPACE>'
spec:
  instanceSelector:
    matchLabels:
      dashboards: '<GRAFANA_CLOUD_STACK_NAME>'
  grafanaCom:
    id: 1860

Configure ArgoCD to Sync the Git Repository

After you commit all changes to Git, create an ArgoCD application. Prepare argo-application.yaml:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: grafana
  namespace: '<ARGOCD_NAMESPACE>'
spec:
  destination:
    name: ''
    namespace: ''
    server: 'https://kubernetes.default.svc'
  source:
    path: '<PATH_TO_GRAFANA_FOLDER>'
    repoURL: '<GIT_REPO_URL>'
    targetRevision: HEAD
    directory:
      recurse: true
  project: default
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
    retry:
      limit: 2
      backoff:
        duration: 5s
        maxDuration: 3m0s
        factor: 2

Replace the placeholders with your Git repo URL, the path to the grafana folder, and your ArgoCD namespace. Then apply it:

kubectl apply -f argo-application.yaml

Verify & Update

Monitor the newly created ArgoCD application in the ArgoCD dashboard to ensure it successfully syncs your configuration. Once synced, your Grafana dashboards should be live and matching the state in Git.

To update a dashboard:

1. 1Make changes to the dashboard JSON in your Git repository
2. 2Commit and push the changes
3. 3ArgoCD detects the update and synchronizes the changes to your Custom Resource
4. 4Grafana Operator syncs changes to the Grafana instance

Next Steps

You've successfully set up a GitOps workflow to manage Grafana dashboards using ArgoCD and the Grafana Operator. Your dashboards are now version-controlled and can be consistently deployed across environments. This approach provides a reliable and auditable way to manage observability dashboards and scale your operations.

• You can install the Grafana Operator's Helm Chart using ArgoCD for full GitOps management of the operator itself
• Follow a similar setup for Grafana Folders and other resources managed by the Grafana Operator
• Consider adding a CI validation step to lint your dashboard JSON before it reaches ArgoCD