@orchestra-research/nemo-curator

GPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.

View in AI SkillSafe app
Scanned · no findings
0 downloads
0 stars
0 demos
SKILL.md

Embed badges

Add these to your README to show the skill's verification status.

SkillSafe verified badge
Verified badge
[![SkillSafe verified badge](https://api.skillsafe.ai/v1/badge/@orchestra-research/nemo-curator/verified)](https://skillsafe.ai/skill/@orchestra-research/nemo-curator/)
Installs badge
Installs badge
[![Installs badge](https://api.skillsafe.ai/v1/badge/@orchestra-research/nemo-curator/installs)](https://skillsafe.ai/skill/@orchestra-research/nemo-curator/)
Scan badge
Scan badge
[![Scan badge](https://api.skillsafe.ai/v1/badge/@orchestra-research/nemo-curator/scan)](https://skillsafe.ai/skill/@orchestra-research/nemo-curator/)
Eval pass rate badge
Eval pass rate
[![Eval pass rate badge](https://api.skillsafe.ai/v1/badge/@orchestra-research/nemo-curator/eval)](https://skillsafe.ai/skill/@orchestra-research/nemo-curator/)