Loading Credentials from Mounted Files in Ververica Platform

Question

Due to security requirements, I have to pass the credentials to Ververica Platform via mounted files (e.g., when retrieving credentials from Hashicorp Vault) rather than the Kubernetes Secrets or plaintext in `values.yaml`. How do I achieve that? Is there any way to load credentials from mounted files in the Ververica Platform pod?

Answer

Note: This section applies to the Ververica Platform 2.5 or later.

Scope

This article describes the ways to provide the following credentials via mounted files. The credentials are categorized into two groups as they need to be treated differently:

Common Credentials

• vvp.persistence.datasource.password
• vvp.auth.bootstrapToken.token
• vvp.auth.oidc.registration.clientId
• vvp.auth.oidc.registration.clientSecret

Blob Storage Credentials

• blobStorageCredentials.s3.accessKeyId
• blobStorageCredentials.s3.secretAccessKey

Note: to integrate Hashicorp Vault with Ververica Platform and load credentials in the Flink jobs running on Ververica Platform, please refer to this link.

Prepare the files with credentials

Common Credentials (single file)

$ cat ./creds-vault.yaml
spring:
  datasource:
    password: admin123
vvp:
  auth:
    oidc:
      registration:
        client-id: victor-test
        client-secret: 7wjuJmRNQEC903WuncsdKsrbo8ND528q
    bootstrapToken:
      token: My_Bstoken

Blob Storage Credentials (two files)

$ cat ./s3.accessKeyId
admin

$ cat ./s3.secretAccessKey
password

Mount the credentials files into the Ververica Platform containers

Getting credentials and mounting them into the Ververica Platform pod depends on where your credentials are stored. When mounting them, please follow the following guides:

Common Credentials

Mount the creds-vault.yaml into the `appmanager` container and the `gateway` container in the Ververica Platform pod under the path `/conf/common-creds/`.

Important: the mounted path and file name can be changed but should not be any existing Ververica Platform directory.

Blob Storage Credentials

Mount the two files (s3.accessKeyId and s3.secretAccessKey) into the `appmanager` container and the `gateway` container in the Ververica Platform pod under the path `/conf/blob-creds/`.

Important: the mounted directory can be changed, but the two filenames must be exactly the same as the ones used here.

Configure the mounted paths

Common Credentials

Add the `creds-vault.yaml` path to the "spring.config.additional-location" in `values.yaml`: (`file:/vvp/etc/`, the default path, must be included)

env:
  - name: "spring.config.additional-location"
    value: "file:/conf/common-creds/creds-vault.yaml,file:/vvp/etc/"

Important: remember to keep the last `/` in `file:/vvp/etc/`, as it has to be a directory.

Blob Storage Credentials

Set the environment variable "vvp.blob-storage.credentials-dir" to the mounted directory in `values.yaml`:

env:
  - name: "vvp.blob-storage.credentials-dir"
    value: "/conf/blob-creds/"

Now you can use `values.yaml` to set up your Ververica Platform. It will get those credentials from those mounted files.

Related Information

• How to integrate Hashicorp Vault with Ververica Platform