Getting Started

Setup new configuration#

You can create new context configuration using an interactive setup. This is the easiest and recommended way of getting setup.

$ gdg tools contexts new mycontext

When creating a new context, you will be asked for authorization type, your default datasource and username/password, along with which folders you wish to manage under the context. You have three options:

1. Default option (“General”)
2. List of folders you wish to manage
3. Wildcard configuration (all folders)

Authentication Concepts#

First let’s touch on a few things regrading Grafana and authentication. You can connect to the Grafana API (which is what GDG is using) by either using basic authentication (aka. username/password) or using a service token.

Tokens are bound to a specific org and cannot cross the Org separation no matter what permission they are given. Users can be Grafana admins, org admins ect. What they can/cannot do will vary on what entities you’re trying to access.

Anything to do with Org will require a Grafana admin. If you’re trying to fetch dashboards a service token will work fine.

1. Using Config:#

The simplest way to set up you auth is to have everything in the gdg.yml. It’s not a very secure pattern if you’re deploying this to a remote server as everything is in plaintext, but it will get you started.

This should be perfectly fine if you’re only running this locally.

Simply use the new context wizard reference above to set that up or set a value for: password and token for the given context.

2. Using Environment Variables:#

You can also override the value using ENV var that line up to the section you want to override:

Ex:

GDG_CONTEXTS__TESTING__PASSWORD=1234
GDG_CONTEXTS__TESTING__TOKEN=1234

will set the token and password value to the one in the ENV.

Keep in mind that the config entry for token and password still need to exist in the config file even if it’s set to an empty value.

3. Using a secure auth location:#

The context name, in this example testing specifies that authentication details should be stored in a file named auth_testing.yaml. This file should be placed in your secure location, which by default is output_path/secure/, or the path you have explicitly configured for your secure storage.

For example, if your secure location is set to /home/user/secure, then the file should be located at: /home/user/secure/auth_testing.yaml

The file can contain your authentication credentials in either YAML or JSON format. If multiple conflicting files exist, the YAML version takes precedence over the JSON version.

Here’s how you might structure the auth_testing.yaml file:

password: 4321
token: shhh

Please ensure that the secure_location in your configuration matches the directory where you’ve placed this file. If not set, it defaults to . output_path/secure/

Priority#

Download Dashboards#

Minimal configuration (eg. the gdg.yml file) that you need to download your dashboards from your Grafana endpoint:

context_name: all

contexts:
  all:
    url: https://grafana.example.org
    user_name: admin # Omit if using tokens
    output_path: exports
    watched:
      - Example
      - Infrastructure
      - MyFolder/.*
      - Some+Folder+With+Spaces/subfolder
    # Either use dashboard_settings.ignore_filters=true or watched folders.
    ## The setting below will ignore all watched folders fetch everything.
    dashboard_settings:
      ignore_filters: false # When set to true all Watched filtered folders will be ignored and ALL folders will be acted on

global:
  debug: true
  ignore_ssl_errors: false

You need to adjust three parts in the configuration in order to function:

• Grafana URL: This is just a URL where your Grafana is available.
• Set the user_name to the value you want to use.
• You need to either set a ENV value or create an auth_<context_name> with the correct values for your token or password.
• API Key OR Username / Password for Admin user. See authentication section above if you need more information.
• Downloaded Folders: The watched field defines folders which will be considered for manipulation. You can see these folders in your Grafana Web UI, under Dashboards > Management. From there, you can simply define the folders you want to be downloaded in the watched list. The dashboards are downloaded as JSON files in the $OUTPUT_PATH/org_name/dashboards/$GRAFANA_FOLDER_NAME directory. Where $OUTPUT_PATH is the path defined in the output_path configuration property and $GRAFANA_FOLDER_NAME the name of the folder from which the dashboards were downloaded.
• The default org_name is “Main Org.” which is the default for Grafana, if you renamed your org, please set its name with the property: organization_name

After you are done, and you can execute ./bin/gdg dash list successfully, eg.:

$ ./bin/gdg dash list
time="2021-08-22T11:11:27+02:00" level=warning msg="Error getting organizations: HTTP error 403: returns {\"message\":\"Permission denied\"}"
time="2021-08-22T11:11:28+02:00" level=info msg="Listing dashboards for context: 'all'"
┌────┬───────────────────────────────────┬───────────────────────────────────┬────────────────┬────────────┬────────────────────────────────────────────────────────────────────────────┐
│ ID │ TITLE                             │ SLUG                              │ FOLDER         │ UID        │ URL                                                                        │
├────┼───────────────────────────────────┼───────────────────────────────────┼────────────────┼────────────┼────────────────────────────────────────────────────────────────────────────┤
│  8 │ AWS CloudWatch Logs               │ aws-cloudwatch-logs               │ Infrastructure │ AWSLogs00  │ https://grafana.example.org/d/AWSLogs00/aws-cloudwatch-logs                │
│  6 │ AWS ECS                           │ aws-ecs                           │ Infrastructure │ ly9Y95XWk  │ https://grafana.example.org/d/ly9Y95XWk/aws-ecs                            │
│  5 │ AWS ELB Application Load Balancer │ aws-elb-application-load-balancer │ Infrastructure │ bt8qGKJZz  │ https://grafana.example.org/d/bt8qGKJZz/aws-elb-application-load-balancer  │
│  4 │ AWS RDS                           │ aws-rds                           │ Infrastructure │ kCDpC5uWk  │ https://grafana.example.org/d/kCDpC5uWk/aws-rds                            │
│  3 │ AWS S3                            │ aws-s3                            │ Infrastructure │ AWSS31iWk  │ https://grafana.example.org/d/AWSS31iWk/aws-s3                             │
│ 17 │ Cluster Autoscaling               │ cluster-autoscaling               │ Example        │ iHUYtABMk  │ https://grafana.example.org/d/iHUYtABMk/cluster-autoscaling                │
└────┴───────────────────────────────────┴───────────────────────────────────┴────────────────┴────────────┴────────────────────────────────────────────────────────────────────────────┘

After executing ./bin/gdg backup dash download you can find the dashboards of the Infrastructure folder in the local directory org_main-org/dashboards/Infrastructure and the dashboards of the Example directory in the local directory org_main-org/dashboards/Example.

Upload Dashboards#

Minimal configuration (i.e. the gdg.yml file) that you need to upload your dashboards from your Grafana endpoint:

context_name: all

contexts:
  all:
    url: https://grafana.example.org
    token: "<<Grafana API Token>>"
    # user_name: admin
    output_path: exports
    watched:
      - Example
      - Infrastructure

global:
  debug: true
  ignore_ssl_errors: false

You need to adjust three parts in the configuration in order to function:

• Grafana URL: This is just a URL where your Grafana is available.
• API Key OR Username / Password for Admin user. See authentication section if you need more information.
• Uploaded Folders: The watched field defines folders which will be considered for manipulation. The dashboards should be stored as JSON files in the $OUTPUT_PATH/$ORG_NAME/dashboards/$GRAFANA_FOLDER_NAME directory. Where $OUTPUT_PATH is the path defined in the output_path configuration property and $GRAFANA_FOLDER_NAME the name of the folder to which the dashboards will be uploaded. In case of the above configuration file, the dashboards should be stored locally in the exports/org_main-org/dashboards/Example and exports/org_main-org/dashboards/Infrastructure/ directories.

├── bin
|   └── gdg
└── exports
    └── org_main-org
        |   └── dashboards
        |       └─ Example
        |       |  └── cluster-scaling.json
        |       └─ Infrastructure
        |          └── aws-ecs.json

You can execute ./bin/gdg backup dash upload to upload the local dashboards to your Grafana. Afterwards, you can try running ./bin/gdg dash list in order to confirm that your dashboards were uploaded successfully.