How do I drain a Flink Job in Ververica Platform?
Note: This applies to Ververica Platform 2.1. The same can be achieved via a toggle button available in 2.2+
When you stop your Flink job, before the pipeline stops, you may want it to stop processing any further events from sources, finish processing all in-flight events, and also commit them into sinks. This procedure is called draining the job and works by sending
MAX_WATERMARK before stopping the pipeline. Since Flink 1.9, this can be achieved with
bin/flink stop -d ... <Job ID>
To drain Flink jobs running on Ververica Platform, set the following annotation in your Ververica Platform Deployment:
Then, whenever your job transitions to SUSPENDED state (e.g., by clicking the button Suspend), it will be drained first before a savepoint is created.
If you remove this annotation again, Ververica Platform will revert back to the original behavior, i.e., no drain.