sparklearning

Partitioning

Stories about how data is split across partitions and how that affects parallelism and shuffle.

Planned stories:

• Why partitions?
• Coalesce vs repartition
• Partition pruning: skipping files and partitions

Stories will be added here gradually.