@orchestra-research/distributed-llm-pretraining-torchtitan
Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.
No skills found in this collection.
SKILL.md
Loading...
Select a file to preview
Analyzing security...
Checking scan reports and verification data.
Bill of Materials
Everything this skill can do — files, network, commands, and more.
No demos yet. To add one, ask your AI agent: "Submit a demo for @orchestra-research/distributed-llm-pretraining-torchtitan" — or upload via the API.
To add a demo, ask your AI agent: "Submit a demo for @orchestra-research/distributed-llm-pretraining-torchtitan"
No eval data yet. Eval results are published via the SkillSafe API.