Skip to main content

Kubernetes Requirements

Alexander Fedulov
  • Updated

 

Note: This applies to Ververica Platform 2.2 - 2.6

 

This article describes the requirements imposed by Ververica platform on the underlying Kubernetes cluster.

 

Kubernetes Versions:

The current Ververica Platform release works with:

  • Kubernetes 1.11+

  • OpenShift 3.11+

Previous versions' Kubernetes compatibility is listed in our documentation.

 

Outside Network Access:

Ververica Platform requires access to registry.ververica.com during the installation and later for fetching Flink-related artifacts. The registry is provided without SLA and it is recommended to set up a private registry with all the required images.

 

Persistent Volume Claims:

By default, Ververica Platform persists its metadata locally using a SQLite DB on a persistent volume (PV). The PV claim has to be satisfied with:

  • 8Gi capacity

  • ReadWriteOnce accessMode

If you do not have access to writable PVs, an alternative option is to store the metadata in an external DB such as MySQL, PostgreSQL, or MS SQL Server.

 

High-level Overview:

The following diagram depicts a high-level overview of the resources involved in running a Ververica Platform installation. It depicts a recommended setup, where Flink jobs are deployed into separate namespaces (`vvp-jobs-1`, `vvp-jobs-2`) from the namespace where Ververica Platform is installed (`vvp`). Such separation creates the basis for multi-tenant setups. API access requirements across the namespaces are discussed in more details in the later section of this article.

482997e5-9a68-4445-a09b-7f4a76f0737e.png

Note: RDBMS is optional, an embedded SQLite database can be used instead.

 

Kubernetes API Access Requirements:

ServiceAccount:

Ververica Platform will attempt to create a service account with the required permissions for managing the cluster. You can also reuse an existing service account if it has corresponding permissions:

rules:
- apiGroups: ["apps", "extensions"]
  resources: ["deployments"]
  verbs: ["create", "delete", "get", "list", "patch", "update", "watch"]
- apiGroups: [""]
  resources: ["configmaps", "pods", "services", "secrets", "serviceaccounts"]
  verbs: ["create", "delete", "get", "list", "patch", "update", "watch"]
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["create", "delete", "get", "list", "patch", "update", "watch"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["roles", "rolebindings"]
  verbs: ["create", "delete", "get", "list", "patch", "update", "watch"]

 

Kubernetes Resources for Deployments and SessionClusters

The Kubernetes resources that Ververica Platform will create depend on the kind of Flink deployment, i.e. a standalone Deployment or a shared SessionCluster, and the features used.

Standalone Deployment (default)

0a5a899f-0dad-4466-83af-c3fc33e96173.png

Session Cluster

df4be7ca-6927-4c85-a788-55f81e929cce.png

 

Multi-tenant Deployments:

Namespaces where Flink jobs will be deployed need to be first created using native Kubernetes tooling and then added to the `values.yaml` file before installing/upgrading via helm, for example:

vvp:
  ...

rbac:
  additionalNamespaces:
    - vvp-jobs-team-1
    - vvp-jobs-team-2
Note: `rbac` is at the same root configuration level, as `vvp`.
 

Appending those namespaces will result in their addition to the ServiceAccount with the ClusterRole permissions listed above.

af3f7d5e-9c74-48d7-9450-c6cb51881963.png

Other cross-namespace access requirements:

For network access policies, see this KB article. 

 

OpenShift-specifics:

By default, a specific `fsGroup` is set in the platform’s charts. Those are controlled directly by OpenShift and need to be “unset“ by adding an empty `securityContext` to the `values.yaml` file:

vvp:
  ...

securityContext:

Note: `securityContext` is at the same root configuration level, as `vvp`.

 

Hardware Resources:

Ververica Platform itself (without any Flink jobs running) requests the following resources from Kubernetes:

appmanager:
  resources: 
    limits: 
      cpu: "1" 
      memory: 1Gi 
    requests:
      cpu: 250m 
      memory: 1Gi 
gateway:
  resources: 
    limits: 
      cpu: "1" 
      memory: 2Gi 
    requests: 
      cpu: 250m 
      memory: 2Gi 
ui:
  resources: 
    limits: 
      cpu: 100m 
      memory: 32Mi 
   requests: 
     cpu: 100m 
     memory: 32Mi

 

Related articles