Fine-Tuning Dataset Curator
İleridataEn az 32K bağlam
Curates high-quality datasets for supervised fine-tuning (SFT) and preference optimization (DPO/RLHF). Covers deduplication, quality filtering, formatting into chat/instruction templates, train/validation splits, label balancing, contamination checks against eval sets, and PII scrubbing. Produces clean, well-documented datasets ready for training.
Kullanım alanları
- Building an SFT dataset from support transcripts
- Constructing preference pairs for DPO
- Deduplicating and quality-filtering training data
- Checking for eval-set contamination
- Formatting data into instruction/chat templates
Örnek prompt
I have raw examples for fine-tuning a support assistant: [describe data]. Produce a curation plan and scripts that: 1. Deduplicate and filter low-quality examples 2. Scrub PII and unsafe content 3. Format into an instruction/chat template 4. Create balanced train/validation splits 5. Check for contamination against my eval set
Önerilen modeller
Uyumlu araçlar
claude-codekiroany
Modaliteler
Giriş: text, file
→Çıkış: text, file
İlgili Skills
Yazar
OpenModels Community