Training

API reference for training scripts and trainers.

SFT Training

CPT Training

Data Processing

Build a mixed Arrow dataset from multiple source datasets with configurable sampling ratios.

Usage:

python data/build_mixed_arrow_dataset.py –config configs/sft_llada.yaml

The script: 1. Loads each Arrow dataset from config.dataset.arrow_dirs 2. Applies sampling (downsample/upsample) based on config.dataset.sampling_ratios 3. Concatenates all sampled datasets 4. Shuffles globally with the seed from config.training.seed 5. Saves to a single output directory (config.experiment.output_dir/mixed_arrow) 6. Saves metadata (mixing_meta.json) with statistics

Requirements:
  • datasets (Hugging Face)

  • omegaconf

data.build_mixed_arrow_dataset.sample_dataset(dataset: Dataset, target_ratio: float, seed: int) → Dataset[source]

Adjust dataset size by downsampling or upsampling.

Parameters:
  • dataset – HuggingFace Dataset

  • target_ratio – float (e.g., 0.5, 2.0, 1.0) - 1.0: use all samples - < 1.0: downsample without replacement - > 1.0: upsample with replacement (duplicate)

  • seed – random seed for reproducibility

Returns:

Dataset with adjusted size

data.build_mixed_arrow_dataset.build_mixed_dataset(dataset_specs: List[Tuple[str, float]], seed: int, output_dir: str) → str[source]

Load, sample, concatenate, shuffle, and save mixed dataset.

Parameters:
  • dataset_specs – List of (arrow_dir, ratio) tuples

  • seed – random seed for reproducibility

  • output_dir – where to save the mixed dataset

Returns:

Path to the saved mixed Arrow dataset

data.build_mixed_arrow_dataset.main()[source]

Split wildguardmix.jsonl into four safety-categorized files based on harm labels.

Classification (4 groups): - Group A (safe_prompts_safe_response):

prompt_harm_label == “unharmful” AND response_harm_label == “unharmful”

  • Group B (safe_prompts_harmful_response):

    prompt_harm_label == “unharmful” AND response_harm_label == “harmful”

  • Group C (harmful_prompt_safe_response):

    prompt_harm_label == “harmful” AND response_harm_label == “unharmful”

  • Group D (harmful_prompt_harmful_response):

    prompt_harm_label == “harmful” AND response_harm_label == “harmful”

Samples with unknown/missing labels are discarded. All original fields including metadata are preserved in the output files.

data.split_wildguardmix.classify_sample(item: Dict[str, Any]) → str[source]

Classify a sample into one of four groups.

Returns:

“A” for safe_prompts_safe_response, “B” for safe_prompts_harmful_response, “C” for harmful_prompt_safe_response, “D” for harmful_prompt_harmful_response, None if unknown/invalid

data.split_wildguardmix.main()[source]