sparklearning

Shuffle

Stories about how data moves between stages: shuffle write, sort/spill, and shuffle read.

Stories:

• The Journey of a Shuffle Record

Planned stories:

• Sort Shuffle vs the old hash shuffle
• External Shuffle Service: when the executor dies but shuffle files live on

Stories will be added here gradually.