sparklearning

Structured Streaming

Stories about state stores, checkpointing, micro-batches, and exactly-once semantics.

Stories:

• RocksDB in Structured Streaming: The Story

Planned stories:

• The life of a micro-batch
• Exactly-once: what “commit” means
• Watermarks: when to forget