Lightning Talk: Implementing Single-Dim Strategies With Sharding Validator - Anshul Sinha, Meta

PyTorch · Beginner ·🛠️ AI Tools & Apps ·3w ago
Lightning Talk: Implementing Single-Dim Strategies With Sharding Validator - Anshul Sinha, Meta DTensor sharding propagation is a major bottleneck to full operator coverage: adding or fixing an op strategy is complex, bug‑prone, and gaps often surface as unexpected resharding and extra collectives. A key source of complexity is that today’s rules conflate (1) semantic correctness—valid input/output sharding combinations for an operator—with (2) search‑space pruning to avoid combinatorial blowups on N‑dimensional meshes. This talk presents a landed prototype that separates these concerns via Single Mesh Dim Strategies: each operator specifies valid placement combinations for one mesh dimension, while infra expands/composes them across the full mesh and selects low‑cost strategies. For contributors, this provides a clear path to refactor existing op_strategies into single‑dim rules that are easier to review and extend. We also introduce a Truth Table‑style sharding validator that systematically tests shapes and sharding specs to check soundness/completeness and to flag unnecessary redistribution/collectives caused by missing cases. The goal of this presentation is faster, higher‑confidence contributions that improve correctness and expand DTensor operator coverage.
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Related AI Lessons

Up next
Modern No-Code AI Route : AI Generalist to AI Builder Induction Session
Krish Naik
Watch →