Configuration

Complete reference for all YAML configuration files in Safe dLLM.

Configuration Hierarchy

Safe dLLM uses OmegaConf with three priority levels:

  1. YAML file (lowest)

  2. Environment variables (optional)

  3. Command-line arguments (highest)

Override syntax:

python run_safety_benchmark.py \
    config=configs/llada_eval.yaml \
    model=/new/path \
    rollout.steps=128 \
    safety.dataset=wildguardmix_D

Config Files Overview

configs/
├── sft_llada.yaml                    # SFT training
├── sft_edit_llada.yaml               # Edit-based SFT
├── cpt_latent_llada_fullscale.yaml   # CPT training
├── llada_eval.yaml                   # Safety evaluation
├── rollout_config.yaml               # Standalone rollout params
├── safety_continue_training_*.yaml   # Safety fine-tuning variants
└── dream_eval.yaml                   # DREAM model eval

llada_eval.yaml (Safety Evaluation)

Main config for run_safety_benchmark.py:

experiment:
  project: "safe-dllm-eval"
  run_name: "llada_base_safety-bench"  # Experiment identifier
  num_node: 1
  node_index: 0

model: "/path/to/llada_8b_base"
model_base: "llada"  # llada | dream

dataset:
  eval_dataset: "MBPP"
  data_type: "code"

execute:
  num_chunk: 8  # Code execution batch size

rollout:
  num_response_per_task: 1
  temperature: 0.0
  steps: 1024
  max_gen_length: 1024
  batch_size: 1
  remasking_strategy: "low_confidence_dynamic"
  target: "confidence"
  dynamic_threshold: 0.95
  edit_threshold: -1         # -1=disable edit, 0.85-0.95=enable
  post_unmask_steps: 0       # Extra edit steps after full unmask
  block_size: 128
  further_horizon: 128       # Causal Guidance forward window
  output_unmasking_history: true
  use_cache: true

safety:
  dataset: "harmbench"
  max_samples: null
  scorer: "llama_guard_3"        # llama_guard_3 | harmbench | pattern
  scorer_model: null             # null=use default
  scorer_device: "cuda:0"
  scorer_batch_size: 8
  task_type: "simple"            # auto | simple | math | code_function | code_stdio | option
  save_responses: true
  no_save: false

attack:
  type: "pair"  # null | prefill | pair | renellm

  # Shared PAIR/ReNeLLM params
  attack_model_name: "openai"
  attack_model_path: "gpt-4o"
  target_model_name: "local"
  target_model_path: null
  target_vllm_mode: false
  eval_model_name: "openai"
  eval_model_path: "gpt-4o"
  openai_key: null               # null → from env
  openai_url: "https://yunwu.ai/v1"

  attack_max_n_tokens: 500
  max_n_attack_attempts: 3
  attack_temperature: 0.5
  attack_top_p: 0.9
  target_max_n_tokens: 1536
  target_temperature: 0.9
  target_top_p: 0.9
  judge_max_n_tokens: 150
  judge_temperature: 1.0

  # PAIR algorithm
  n_streams: 2
  keep_last_n: 3
  n_iterations: 3

  devices: "cuda:0"
  target_system_prompt: null

  # LLaDA-specific
  batch_size: 8

  res_save_path: "results/attack.jsonl"
  delete_existing_res: false

Rollout Parameters Deep Dive

Parameter

Location

Default

Description

steps

rollout.steps

64

Total denoising steps (↑=quality↑ but slower)

max_gen_length

rollout.max_gen_length

128

Maximum tokens to generate

block_size

rollout.block_size

1

Consecutive tokens unmasked per step

temperature

rollout.temperature

0.0

Sampling temperature (0=deterministic)

target

rollout.target

confidence

Selection: confidence | margin_confidence | neg_entropy | random

remasking_strategy

rollout.remasking_strategy

low_confidence_static

Remasking strategy

dynamic_threshold

rollout.dynamic_threshold

None

Confidence threshold for dynamic strategy

edit_threshold

rollout.edit_threshold

-1

Edit threshold (-1=disabled, 0.85-0.95=enabled)

post_unmask_steps

rollout.post_unmask_steps

0

Extra edit steps after full unmask

use_cache

rollout.use_cache

true

Enable KV cache acceleration

further_horizon

rollout.further_horizon

None

Causal Guidance forward window

sft_llada.yaml (SFT Training)

wandb:
  entity: null
  resume: 'auto'
  run_name: "sft_llada_mixed_100k_code-35-math-35-if-20-safe-10_2epoch_bs64_260403"

experiment:
  project: "safe-dllm-sft"
  num_node: 1

model:
  pretrained_model: "/scratch/aszalay1/tianze/models/llada_8b_base"

dataset:
  arrow_dirs:
    - "/scratch/aszalay1/tianze/cpt_data/sft_pyarrows/mix_100k_sft_dataset_MCIS-3111"

training:
  gradient_checkpointing_enable: false
  gradient_accumulation_steps: 8      # Override per-SLURM: 32 for 2 GPUs, 16 for 4 GPUs
  batch_size_lm: 1
  mixed_precision: "bf16"
  enable_tf32: true
  seed: 30002
  num_train_epochs: 2
  max_grad_norm: 1.0
  save_steps: 500
  unmask_confidence_threshold: 0.9   # For edit-based training

optimizer:
  name: adamw
  params:
    learning_rate: 1e-5
    scale_lr: false
    beta1: 0.9
    beta2: 0.999
    weight_decay: 0.0
    epsilon: 1e-8

lr_scheduler:
  scheduler: "cosine"
  params:
    learning_rate: ${optimizer.params.learning_rate}
    warmup_steps: 100
    min_lr_scale: 0.1
    power: 2.0

Training Parameter Guide

Parameter

Typical Value

Effect

gradient_accumulation_steps

8 (4 GPUs) → 32 total batch

Effective batch size = batch_size_lm × this × num_gpus

batch_size_lm

1

Per-GPU micro-batch size

num_train_epochs

2-3

Total training passes over data

learning_rate

1e-5

SFT: 1e-5 ~ 5e-5, CPT: 5e-6 ~ 1e-5

save_steps

500

Checkpoint frequency

mixed_precision

bf16

bf16 | fp16 | no

rollout_config.yaml (Standalone)

For direct use with sample/llada_sample.py:

rollout:
  steps: 64
  max_gen_length: 128
  block_size: 1
  temperature: 0.0
  target: "confidence"
  use_cache: true
  remasking_strategy: "low_confidence_static"
  dynamic_threshold: null
  further_horizon: null

experiment:
  num_node: 1
  node_index: 0

model: "/path/to/model"

dataset:
  eval_dataset: "harmbench_standard"
  data_type: "harmbench"

Edit Threshold Guide

Value

Mode

Use Case

-1

Base LLaDA (no edit)

Standard diffusion generation

0.85

Aggressive edit

More token refinement, potentially safer

0.90

Moderate edit

Recommended starting point

0.95

Conservative edit

Only high-confidence tokens edited

>0.98

Very conservative

Minimal edit, near base behavior

Recommendation: Start with 0.90 and adjust based on ASR trade-offs.

Task Types Reference

safety:
  task_type: "simple"  # auto | simple | math | code_function | code_stdio | option

Templates defined in sample/llada_sample.py → TEMPLATE_MAP:

  • simple: "{query}"

  • math: "Solve the math problem. Put final answer in \\boxed{{}}.\n{query}"

  • code_function: "Write a Python function. Put code in code block.\n{query}"

  • code_stdio: "Use input()/print(). Put code in code block.\n{query}"

  • option: "Multiple choice. Put answer (A/B/C/D) in \\boxed{{}}.\n{query}"

Multi-Dataset Sampling

Configure in data/build_mixed_arrow_dataset.py:

dataset_specs = [
    ("/path/to/math_data", 0.4),     # 40% math
    ("/path/to/code_data", 0.4),     # 40% code
    ("/path/to/safety_data", 0.2),   # 20% safety
]

Ratios:

  • 1.0 → use all samples

  • < 1.0 → downsample without replacement

  • > 1.0 → upsample with replacement

Naming Conventions

experiment:
  run_name: "sft_llada_{data}_{model}_{epochs}ep_{bs}bs_{date}"

Examples:

  • sft_llada_mixed100k_base_2ep_bs32_260414

  • eval_harmbench_edit095_260415

  • pair_attack_llada_sft_260416

Validation

Dry-run to verify config:

python run_safety_benchmark.py \
    config=configs/llada_eval.yaml \
    model=/path/to/model \
    --dry_run

Inspect merged config:

from omegaconf import OmegaConf

conf = OmegaConf.load("configs/llada_eval.yaml")
cli = OmegaConf.from_cli(["model=/new/path", "rollout.steps=128"])
final = OmegaConf.merge(conf, cli)
print(OmegaConf.to_yaml(final))

Tips

Large steps / max_gen_length

Quality improves but memory/time increase linearly. Balance based on GPU capacity.

Small batch_size

Use when OOM. Can parallelize across multiple GPUs by setting --device cuda:0 --scorer_device cuda:1.

Edit mode not working

Ensure:

  1. Model was trained with unmask_confidence_threshold set

  2. edit_threshold is in [0.85, 0.98] range (not -1)

  3. post_unmask_steps ≥ 1 for noticeable effect

Multiple datasets

List multiple arrow_dirs in config. Use sampling_ratios to control proportions.

See Also

  • guide/training - Training with these configs

  • guide/evaluation - Evaluation configuration

  • guide/slurm - Slurm parameter overrides

  • Full API for programmatic config