How to Install Grafana with K10 Dashboard and Data Sources Pre-provisioned
Purpose
Starting in Veeam Kasten for Kubernetes v7.5.0, Grafana will no longer be included by default.
This in-depth technical guide will demonstrate the process of installing Grafana on Kubernetes and populating it with Kasten Data. It will allow you to quickly access pre-provisioned Kasten Dashboards in Grafana, which can be customized to suit your business requirements.
Solution
Grafana Deployment Options on Kubernetes
A new instance of Grafana OSS can be installed on Kubernetes using the following three primary methods:
- Grafana Helm Chart (recommended)
Method Documented in this Article
This article will document the recommended method, installing Grafana using helm chart and customizing it using values.yaml so that it gets installed with the K10 Grafana dashboard and data source.
About Helm
Helm is a package manager for Kubernetes that simplifies the distribution and installation of applications on Kubernetes clusters. Application configurations can easily be changed/modified during installation by providing a custom configuration file using the '-f' or '--values' flag with the helm install command.
Preparing Helm values for Grafana Installation
To configure the new Grafana instance with the Kasten dashboard and appropriate data source, a values.yaml file must be created and later passed to the helm install command.
Please start by creating a values.yaml file where you will add custom configurations for the Grafana instance.
The steps below specify the Helm fields that can be used to configure specific configuration/settings of Grafana.
Persistent Storage
As mentioned in the Grafana docs, persistent storage is disabled by default in Grafana, which means it will use ephemeral storage, and all of its data will be stored within the container's file system. This data will be lost if the container is stopped, restarted, or if the container crashes. To prevent data loss, we must configure the installation to persist changes into a volume.
To configure this, add the below field to the values.yaml file:
persistence:
enabled: true K10 Grafana Dashboard
K10's Grafana dashboard is already published on the Grafana dashboard index https://grafana.com/grafana/dashboards/21065-k10-dashboard/ that can be used in values.yaml to configure the dashboard.
Ensure the K10 Grafana Dashboard is automatically created in the newly installed Grafana by configuring the dashboards field in the values.yaml file as shown here:
dashboardProviders:
dashboardproviders.yaml:
apiVersion: 1
providers:
- name: 'default'
orgId: 1
folder: ''
type: file
disableDeletion: false
editable: true
options:
path: /var/lib/grafana/dashboards/default
dashboards:
default:
k10-dashboard:
gnetId: 21065
revision: 1
datasource:
- name: DS_PROMETHEUS
value: Prometheus
- name: VAR_CLUSTER
value: "" Data Source
As we can see in the snippet above, the K10 Grafana dashboard relies on a Prometheus data source.
This data source can be configured using the datasources field of the values.yaml file as shown below:
Please replace <kasten-namespace> in the url attribute with the namespace where Kasten is installed.
datasources:
datasources.yaml:
apiVersion: 1
datasources:
- access: proxy
editable: false
isDefault: true
jsonData:
timeInterval: 1m
name: Prometheus
type: prometheus
url: http://prometheus-server.<kasten-namespace>/k10/prometheus/ SMTP Configuration (Optional)
If you need to configure SMTP settings to enable the email alerts, the settings (server, user, password) must be provided to the Grafana instance.
Below are the helm fields in the values.yaml that can be used to enable SMTP in Grafana:
grafana.ini:
smtp:
enabled: true
host: <SMTP.SERVER.COM:PORT>
user: <SMTP_USER>
password: <SMTP_PASSWORD>
skip_verify: true
from_address: <EMAIL_TO_BE_USED_AS_SENDER>
from_name: <SENDER_NAME_FOR_THE_EMAIL> Installing the New Grafana instance
Once all of the above entries are made in the values.yaml file, the command below can be used to install the Grafana instance:
helm upgrade my-grafana grafana/grafana -n monitoring -f values.yaml --install --create-namespaceExample:
$ helm upgrade my-grafana grafana/grafana -n monitoring -f values.yaml --install --create-namespace
Release "my-grafana" does not exist. Installing it now.
NAME: my-grafana
LAST DEPLOYED: Thu Jun 6 17:40:36 2024
NAMESPACE: monitoring
STATUS: deployed
REVISION: 1
NOTES:
1. Get your 'admin' user password by running:
kubectl get secret --namespace monitoring my-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
2. The Grafana server can be accessed via port 80 on the following DNS name from within your cluster:
my-grafana.monitoring.svc.cluster.local
Get the Grafana URL to visit by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace monitoring -l "app.kubernetes.io/name=grafana,app.kubernetes.io/instance=my-grafana" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace monitoring port-forward $POD_NAME 3000
3. Login with the password from step 1 and the username: admin
As mentioned in the output of the helm install command, the kubectl port-forward command can be used to access the newly installed Grafana. Data sources and dashboards can also be checked to ensure the configured ones have been created.
Based on the requirement, the respective helm field can be configured if the Grafana instance needs to be exposed via a LoadBalancer service or Ingress resource. An exhaustive list of supported helm fields can be found in the Grafana Helm Chart Documentation.
Migrating Grafana Resources
The following steps may be performed if there is a need to migrate other Grafana resources (e.g., alert rules and contact points) to the newly installed Grafana instance.
Back-Up
First, a backup of the existing Grafana instance needs to be created.
The following command can be used to backup the Grafana database that gets installed with K10:
kubectl cp <kasten-namespace>/<grafana-podname>:<GF_PATHS_DATA>/grafana.db grafana.db Restore
Once the backup has been taken, the backup file (grafana.db) must be copied/restored to the newly installed Grafana pod at the correct location (GF_PATHS_DATA).
The following command can be used to copy/restore the backup file (grafana.db) to the newly installed Grafana pod:
kubectl cp grafana.db <new-grafana-namespace>/<grafana-pod-name>:<GF_PATHS_DATA>/grafana.db Restart Grafana
After the database backup file has been restored to the newly installed Grafana pod, the Grafana instance needs to be restarted.
To restart the Grafana pod, scale down the Grafana deployment to 0 replicas and then scale it up back to 1 replicas:
kubectl scale deployment -n monitoring my-grafana --replicas 0
kubectl scale deployment -n monitoring my-grafana --replicas 1 Reset Admin Password
Once the database file is restored and the Grafana pod is restarted, the admin password of the newly installed Grafana instance changes to what it was in the original Grafana instance.
To change that admin password, execute the following command in the Grafana pod to change the admin password:
grafana-cli admin reset-admin-password <new-password> To report a typo on this page, highlight the typo with your mouse and press CTRL + Enter.