sparklearning

Query planning (Catalyst)

Stories about how DataFrame/SQL becomes a logical plan, gets optimized, and turns into a physical plan.

Planned stories:

• From your code to the physical plan
• The optimizer’s playbook (predicate pushdown, join order, etc.)
• Whole-stage codegen: many operators, one generated function

Stories will be added here gradually.