Welcome to Safe dLLM — a comprehensive framework for safety evaluation, adversarial testing, and alignment research of LLaDA (Latent Diffusion Language Model).
What is LLaDA?
LLaDA (Latent Diffusion Language Model) is a novel language modeling approach that applies diffusion models to text generation:
Diffusion Process: Instead of autoregressive token-by-token generation, LLaDA starts with fully masked sequences and iteratively denoises
Parallel Token Prediction: Multiple tokens can be predicted simultaneously at each diffusion step
Edit-Based Training: Post-training, an
edit_headcan refine already-unmasked tokens for higher quality
Key advantages:
Better performance on math and code tasks
More controllable generation via
edit_thresholdNatural multi-turn capabilities
Project Structure
safe_dllm/
├── AISafetyLab/ # Submodule: attack & evaluation framework
│ └── aisafetylab/
│ ├── attack/ # PAIR, ReNeLLM, Prefill attacks
│ ├── evaluation/ # Scorers (LlamaGuard3, HarmBench)
│ ├── models/ # LocalModel, VLLMModel, OpenAIModel
│ └── dataset/ # AttackDataset
├── models/ # LLaDA & DREAM model implementations
│ ├── llada/ # Core LLaDA transformer + diffusion
│ ├── dream/ # DREAM model variant
│ ├── sampling.py # Sampling strategies
│ └── lr_schedulers.py # Learning rate scheduling
├── sample/ # Inference & generation
│ ├── llada_sample.py # Main diffusion generation
│ └── dream_sample.py # DREAM generation
├── train/ # Training scripts
│ ├── sft_llada.py # Supervised Fine-Tuning
│ ├── sft_dream.py # DREAM SFT
│ ├── cpt_latent_llada_fullscale.py # Continued Pre-Training
│ └── utils.py # Training utilities
├── safety/ # Safety evaluation & attacks
│ ├── attack.py # LLaDA wrapper for AISafetyLab
│ ├── evaluator.py # Scorer wrapper
│ ├── dataset.py # Dataset loaders
│ ├── prepare_pair_attack.py
│ ├── prepare_renellm_attack.py
│ └── prepare_prefill_attack.py
├── reward/ # RLHF reward functions
│ ├── reward.py # Main reward
│ ├── execute.py # Code execution
│ ├── math_utils.py # Math verification
│ └── rl_*.py # RL-specific rewards
├── data/ # Data processing
│ ├── split_wildguardmix.py
│ ├── build_mixed_arrow_dataset.py
│ └── preprocess_sft.py
├── configs/ # YAML configurations
│ ├── sft_llada.yaml
│ ├── llada_eval.yaml
│ ├── rollout_config.yaml
│ └── safety_continue_training_*.yaml
├── scripts/ # Slurm submission scripts
│ ├── submit_*_sft_llada.slurm
│ ├── submit_*_eval_bench.slurm
│ └── submit_*_eval_safety.slurm
├── run_safety_benchmark.py # Main evaluation entry point
├── eval_bench.py # Benchmark evaluation
├── generate.py # Standalone generation
├── quick_test_dataset.py # Quick dataset test
├── docs/ # This documentation
└── pyproject.toml # Dependencies
Core Concepts
Diffusion Generation
LLaDA’s generation is an iterative denoising process:
Initialize: All response positions are masked with
[MASK]tokenDenoise: At each step, predict a subset of masked tokens
Select: Choose tokens with highest confidence/lowest entropy
Repeat: Continue until all masks are replaced
Output: Final generated sequence
Configuration via rollout parameters:
rollout:
steps: 64 # Number of denoising steps
max_gen_length: 128 # Maximum response length
block_size: 1 # Tokens unmasked per step
temperature: 0.0 # Sampling temperature
target: confidence # Selection strategy
edit_threshold: -1 # Edit mode threshold (-1=disabled)
Edit-Based Training
After standard SFT, LLaDA can be further fine-tuned with an edit_head:
Tokens with confidence ≥
edit_thresholdget refinedpost_unmask_stepscontrols additional edit iterationsImproves safety and response quality
training:
unmask_confidence_threshold: 0.9
rollout:
edit_threshold: 0.9
post_unmask_steps: 5
Attack Methods
Three attack paradigms are supported:
PAIR (Prompt Automatic Iterative Refinement) - Iterative optimization via attack/judge LLMs (GPT-4o, Claude) - Multi-stream, multi-iteration - Most effective but requires API
ReNeLLM (Reasoning-enhanced Jailbreaking) - Exploits model’s own reasoning capability - Cached prompts for reuse - Strongest attack method
Prefill Attack - Static harmful prompts pre-filled - No API required - Fast for batch testing
Evaluation
Integrated with AISafetyLab for standardized safety assessment:
Datasets: HarmBench (~400), WildGuardMix (~2000 per group A/B/C/D)
Scorers: LlamaGuard3 (14 categories), HarmBench (binary), OverRefusal, PatternScorer
Metrics: Attack Success Rate (ASR), category breakdown, overrefusal rate
Typical workflow:
python run_safety_benchmark.py \
--model_path /path/to/checkpoint \
--dataset harmbench \
--scorer llama_guard_3
Use Cases
Safe dLLM is designed for:
Safety Alignment Research - Measure model safety with standardized benchmarks - Track safety drift during training - Compare alignment methods
Adversarial Testing - Test robustness against PAIR/ReNeLLM attacks - Build attack prompt libraries - Evaluate attack success rate (ASR)
Model Training - SFT on safety datasets - CPT on large corpora - Edit-based fine-tuning for safety
RLHF / Reward Modeling - Math/code reward functions - Format validation - Combined reward optimization
Getting Help
Next Steps
Quick Start - Get running in 5 minutes
Training - Train your first model
Evaluation - Run safety evaluations
Attacks - Test with adversarial attacks