Configuration
Complete reference for all YAML configuration files in Safe dLLM.
Configuration Hierarchy
Safe dLLM uses OmegaConf with three priority levels:
YAML file (lowest)
Environment variables (optional)
Command-line arguments (highest)
Override syntax:
python run_safety_benchmark.py \
config=configs/llada_eval.yaml \
model=/new/path \
rollout.steps=128 \
safety.dataset=wildguardmix_D
Config Files Overview
configs/
├── sft_llada.yaml # SFT training
├── sft_edit_llada.yaml # Edit-based SFT
├── cpt_latent_llada_fullscale.yaml # CPT training
├── llada_eval.yaml # Safety evaluation
├── rollout_config.yaml # Standalone rollout params
├── safety_continue_training_*.yaml # Safety fine-tuning variants
└── dream_eval.yaml # DREAM model eval
llada_eval.yaml (Safety Evaluation)
Main config for run_safety_benchmark.py:
experiment:
project: "safe-dllm-eval"
run_name: "llada_base_safety-bench" # Experiment identifier
num_node: 1
node_index: 0
model: "/path/to/llada_8b_base"
model_base: "llada" # llada | dream
dataset:
eval_dataset: "MBPP"
data_type: "code"
execute:
num_chunk: 8 # Code execution batch size
rollout:
num_response_per_task: 1
temperature: 0.0
steps: 1024
max_gen_length: 1024
batch_size: 1
remasking_strategy: "low_confidence_dynamic"
target: "confidence"
dynamic_threshold: 0.95
edit_threshold: -1 # -1=disable edit, 0.85-0.95=enable
post_unmask_steps: 0 # Extra edit steps after full unmask
block_size: 128
further_horizon: 128 # Causal Guidance forward window
output_unmasking_history: true
use_cache: true
safety:
dataset: "harmbench"
max_samples: null
scorer: "llama_guard_3" # llama_guard_3 | harmbench | pattern
scorer_model: null # null=use default
scorer_device: "cuda:0"
scorer_batch_size: 8
task_type: "simple" # auto | simple | math | code_function | code_stdio | option
save_responses: true
no_save: false
attack:
type: "pair" # null | prefill | pair | renellm
# Shared PAIR/ReNeLLM params
attack_model_name: "openai"
attack_model_path: "gpt-4o"
target_model_name: "local"
target_model_path: null
target_vllm_mode: false
eval_model_name: "openai"
eval_model_path: "gpt-4o"
openai_key: null # null → from env
openai_url: "https://yunwu.ai/v1"
attack_max_n_tokens: 500
max_n_attack_attempts: 3
attack_temperature: 0.5
attack_top_p: 0.9
target_max_n_tokens: 1536
target_temperature: 0.9
target_top_p: 0.9
judge_max_n_tokens: 150
judge_temperature: 1.0
# PAIR algorithm
n_streams: 2
keep_last_n: 3
n_iterations: 3
devices: "cuda:0"
target_system_prompt: null
# LLaDA-specific
batch_size: 8
res_save_path: "results/attack.jsonl"
delete_existing_res: false
Rollout Parameters Deep Dive
Parameter |
Location |
Default |
Description |
|---|---|---|---|
|
|
64 |
Total denoising steps (↑=quality↑ but slower) |
|
|
128 |
Maximum tokens to generate |
|
|
1 |
Consecutive tokens unmasked per step |
|
|
0.0 |
Sampling temperature (0=deterministic) |
|
|
|
Selection: confidence | margin_confidence | neg_entropy | random |
|
|
|
Remasking strategy |
|
|
None |
Confidence threshold for dynamic strategy |
|
|
-1 |
Edit threshold (-1=disabled, 0.85-0.95=enabled) |
|
|
0 |
Extra edit steps after full unmask |
|
|
true |
Enable KV cache acceleration |
|
|
None |
Causal Guidance forward window |
sft_llada.yaml (SFT Training)
wandb:
entity: null
resume: 'auto'
run_name: "sft_llada_mixed_100k_code-35-math-35-if-20-safe-10_2epoch_bs64_260403"
experiment:
project: "safe-dllm-sft"
num_node: 1
model:
pretrained_model: "/scratch/aszalay1/tianze/models/llada_8b_base"
dataset:
arrow_dirs:
- "/scratch/aszalay1/tianze/cpt_data/sft_pyarrows/mix_100k_sft_dataset_MCIS-3111"
training:
gradient_checkpointing_enable: false
gradient_accumulation_steps: 8 # Override per-SLURM: 32 for 2 GPUs, 16 for 4 GPUs
batch_size_lm: 1
mixed_precision: "bf16"
enable_tf32: true
seed: 30002
num_train_epochs: 2
max_grad_norm: 1.0
save_steps: 500
unmask_confidence_threshold: 0.9 # For edit-based training
optimizer:
name: adamw
params:
learning_rate: 1e-5
scale_lr: false
beta1: 0.9
beta2: 0.999
weight_decay: 0.0
epsilon: 1e-8
lr_scheduler:
scheduler: "cosine"
params:
learning_rate: ${optimizer.params.learning_rate}
warmup_steps: 100
min_lr_scale: 0.1
power: 2.0
Training Parameter Guide
Parameter |
Typical Value |
Effect |
|---|---|---|
|
8 (4 GPUs) → 32 total batch |
Effective batch size = batch_size_lm × this × num_gpus |
|
1 |
Per-GPU micro-batch size |
|
2-3 |
Total training passes over data |
|
1e-5 |
SFT: 1e-5 ~ 5e-5, CPT: 5e-6 ~ 1e-5 |
|
500 |
Checkpoint frequency |
|
|
|
rollout_config.yaml (Standalone)
For direct use with sample/llada_sample.py:
rollout:
steps: 64
max_gen_length: 128
block_size: 1
temperature: 0.0
target: "confidence"
use_cache: true
remasking_strategy: "low_confidence_static"
dynamic_threshold: null
further_horizon: null
experiment:
num_node: 1
node_index: 0
model: "/path/to/model"
dataset:
eval_dataset: "harmbench_standard"
data_type: "harmbench"
Edit Threshold Guide
Value |
Mode |
Use Case |
|---|---|---|
|
Base LLaDA (no edit) |
Standard diffusion generation |
|
Aggressive edit |
More token refinement, potentially safer |
|
Moderate edit |
Recommended starting point |
|
Conservative edit |
Only high-confidence tokens edited |
|
Very conservative |
Minimal edit, near base behavior |
Recommendation: Start with 0.90 and adjust based on ASR trade-offs.
Task Types Reference
safety:
task_type: "simple" # auto | simple | math | code_function | code_stdio | option
Templates defined in sample/llada_sample.py → TEMPLATE_MAP:
simple:"{query}"math:"Solve the math problem. Put final answer in \\boxed{{}}.\n{query}"code_function:"Write a Python function. Put code in code block.\n{query}"code_stdio:"Use input()/print(). Put code in code block.\n{query}"option:"Multiple choice. Put answer (A/B/C/D) in \\boxed{{}}.\n{query}"
Multi-Dataset Sampling
Configure in data/build_mixed_arrow_dataset.py:
dataset_specs = [
("/path/to/math_data", 0.4), # 40% math
("/path/to/code_data", 0.4), # 40% code
("/path/to/safety_data", 0.2), # 20% safety
]
Ratios:
1.0→ use all samples< 1.0→ downsample without replacement> 1.0→ upsample with replacement
Naming Conventions
experiment:
run_name: "sft_llada_{data}_{model}_{epochs}ep_{bs}bs_{date}"
Examples:
sft_llada_mixed100k_base_2ep_bs32_260414eval_harmbench_edit095_260415pair_attack_llada_sft_260416
Validation
Dry-run to verify config:
python run_safety_benchmark.py \
config=configs/llada_eval.yaml \
model=/path/to/model \
--dry_run
Inspect merged config:
from omegaconf import OmegaConf
conf = OmegaConf.load("configs/llada_eval.yaml")
cli = OmegaConf.from_cli(["model=/new/path", "rollout.steps=128"])
final = OmegaConf.merge(conf, cli)
print(OmegaConf.to_yaml(final))
Tips
Large steps / max_gen_length
Quality improves but memory/time increase linearly. Balance based on GPU capacity.
Small batch_size
Use when OOM. Can parallelize across multiple GPUs by setting --device cuda:0 --scorer_device cuda:1.
Edit mode not working
Ensure:
Model was trained with
unmask_confidence_thresholdsetedit_thresholdis in [0.85, 0.98] range (not -1)post_unmask_steps≥ 1 for noticeable effect
Multiple datasets
List multiple arrow_dirs in config. Use sampling_ratios to control proportions.
See Also
guide/training - Training with these configs
guide/evaluation - Evaluation configuration
guide/slurm - Slurm parameter overrides
Full API for programmatic config