sparklearning

Execution & scheduling

Stories about how Spark turns an action into a DAG of stages and tasks, and how the driver and executors coordinate.

Planned stories:

• From one action to many tasks
• The life of a task
• Why stages exist (shuffle as the “wall”)

Stories will be added here gradually.