How do I drain a Flink Job in Ververica Platform?
When you stop your Flink job, before the pipeline stops, you may want it to stop processing any further events from sources, finish processing all in-flight events, and also commit them into sinks. This procedure is called draining the job and works by sending
MAX_WATERMARK before stopping the pipeline. Since Flink 1.9, this can be achieved with
bin/flink stop -d ... <Job ID>
To drain Flink jobs running on Ververica Platform, set the following annotation in your Ververica Platform Deployment:
Then, whenever your job transitions to SUSPENDED state (e.g., by clicking the button Suspend), it will be drained first before a savepoint is created.
|Note: This is currently an undocumented feature in Ververica Platform 2.1. We are planning to make this feature available in the Web UI in a future release.|
If you remove this annotation again, Ververica Platform will revert back to the original behavior, i.e., no drain.