Fetching Secrets from HashiCorp Vault in Jenkins Pipeline

In modern DevOps practices, securing sensitive information like API tokens, passwords, and other credentials is paramount. Vault, an open-source tool developed by HashiCorp, provides a secure way to store and manage these secrets. Integrating Vault with Jenkins pipelines ensures that sensitive data remains protected throughout the software delivery process. In this tutorial, we’ll walk through the steps to fetch secrets from Vault within a Jenkins pipeline.

Prerequisites: Before proceeding, ensure you have the following:

• Jenkins server installed and configured.
• Vault server deployed and accessible.
• No need to install Jenkins Vault Plugin

Step 1: Setting Up Jenkins Global Variables for Vault Access: First, we need to set up global variables in Jenkins to allow communication with Vault. These variables typically include the Vault token (VAULT_TOKEN) and Vault address (VAULT_ADDR). You can configure these variables in the Jenkins system settings.

Step 2: Defining Environment Variables in Jenkins Pipeline: In your Jenkins pipeline script, define environment variables for accessing secrets from Vault. For instance, you may need to specify the path to the Dockerfile stored in Vault. Let’s call this variable DOCKERFILE_PATH and set its value to "v1/application/data/Dockerfile".

Step 3: Writing the Jenkins Pipeline Stage to Fetch Secrets: Now, let’s write a Jenkins pipeline stage to fetch the Dockerfile from Vault. Below is a sample stage written in Groovy script, along with an explanation of the code:

stage('Fetch from Vault') {
    steps {
        script {
            // Define a function to execute shell commands without printing the trace
            def printWithNoTrace = { cmd ->
                return sh(script: "#!/bin/sh -e\n${cmd}", returnStdout: true).trim()
            }

            // Use the defined function to execute a curl command to fetch the secret from Vault
            def vaultResponse = printWithNoTrace("curl -sS -H 'X-Vault-Token: $VAULT_TOKEN' $VAULT_ADDR/$DOCKERFILE_PATH")

            // Parse the JSON response received from Vault
            def json = readJSON text: vaultResponse

            // Write the fetched Dockerfile content to a file in the Jenkins workspace
            writeFile file: "${WORKSPACE}/Dockerfile", text: json.data.data['Dockerfile']
        }
    }
}

Explanation of the code:

• We define a function printWithNoTrace that executes shell commands (cmd) without printing the trace. This is achieved by using the sh step with the -e option to terminate the script immediately upon encountering an error (#!/bin/sh -e).
  That’s also need to hide $VAULT_TOKEN from exposing in jenkins console build output.
• Within the stage, we execute a curl command to fetch the secret from Vault. The -sS flags ensure that curl operates silently and shows errors if they occur. The -H 'X-Vault-Token: $VAULT_TOKEN' header specifies the Vault token for authentication, and $VAULT_ADDR/$DOCKERFILE_PATH represents the URL path to the secret within Vault.
• The response from Vault is captured in the vaultResponse variable.
• We use the readJSON step to parse the JSON response received from Vault into a structured JSON object, stored in the json variable.
• Finally, we extract the Dockerfile content from the JSON object (json.data.data['Dockerfile']) and write it to a file named Dockerfile in the Jenkins workspace using the writeFile step.

This code snippet demonstrates how to securely fetch secrets from Vault within a Jenkins pipeline, ensuring that sensitive information remains protected throughout the CI/CD process.

Integrating Vault with Jenkins pipelines enables secure management of sensitive data during CI/CD processes. By following the steps outlined in this tutorial, you can effectively fetch secrets from Vault within your Jenkins pipelines, enhancing the security and integrity of your software delivery pipeline.