Welcome to Safe dLLM — a comprehensive framework for safety evaluation, adversarial testing, and alignment research of LLaDA (Latent Diffusion Language Model).

What is LLaDA?

LLaDA (Latent Diffusion Language Model) is a novel language modeling approach that applies diffusion models to text generation:

  • Diffusion Process: Instead of autoregressive token-by-token generation, LLaDA starts with fully masked sequences and iteratively denoises

  • Parallel Token Prediction: Multiple tokens can be predicted simultaneously at each diffusion step

  • Edit-Based Training: Post-training, an edit_head can refine already-unmasked tokens for higher quality

Key advantages:

  • Better performance on math and code tasks

  • More controllable generation via edit_threshold

  • Natural multi-turn capabilities

Project Structure

safe_dllm/
├── AISafetyLab/              # Submodule: attack & evaluation framework
│   └── aisafetylab/
│       ├── attack/           # PAIR, ReNeLLM, Prefill attacks
│       ├── evaluation/        # Scorers (LlamaGuard3, HarmBench)
│       ├── models/           # LocalModel, VLLMModel, OpenAIModel
│       └── dataset/          # AttackDataset
├── models/                   # LLaDA & DREAM model implementations
│   ├── llada/                # Core LLaDA transformer + diffusion
│   ├── dream/                # DREAM model variant
│   ├── sampling.py           # Sampling strategies
│   └── lr_schedulers.py      # Learning rate scheduling
├── sample/                   # Inference & generation
│   ├── llada_sample.py       # Main diffusion generation
│   └── dream_sample.py       # DREAM generation
├── train/                    # Training scripts
│   ├── sft_llada.py          # Supervised Fine-Tuning
│   ├── sft_dream.py          # DREAM SFT
│   ├── cpt_latent_llada_fullscale.py  # Continued Pre-Training
│   └── utils.py              # Training utilities
├── safety/                   # Safety evaluation & attacks
│   ├── attack.py             # LLaDA wrapper for AISafetyLab
│   ├── evaluator.py          # Scorer wrapper
│   ├── dataset.py            # Dataset loaders
│   ├── prepare_pair_attack.py
│   ├── prepare_renellm_attack.py
│   └── prepare_prefill_attack.py
├── reward/                   # RLHF reward functions
│   ├── reward.py             # Main reward
│   ├── execute.py            # Code execution
│   ├── math_utils.py         # Math verification
│   └── rl_*.py               # RL-specific rewards
├── data/                     # Data processing
│   ├── split_wildguardmix.py
│   ├── build_mixed_arrow_dataset.py
│   └── preprocess_sft.py
├── configs/                  # YAML configurations
│   ├── sft_llada.yaml
│   ├── llada_eval.yaml
│   ├── rollout_config.yaml
│   └── safety_continue_training_*.yaml
├── scripts/                  # Slurm submission scripts
│   ├── submit_*_sft_llada.slurm
│   ├── submit_*_eval_bench.slurm
│   └── submit_*_eval_safety.slurm
├── run_safety_benchmark.py   # Main evaluation entry point
├── eval_bench.py             # Benchmark evaluation
├── generate.py               # Standalone generation
├── quick_test_dataset.py     # Quick dataset test
├── docs/                     # This documentation
└── pyproject.toml            # Dependencies

Core Concepts

Diffusion Generation

LLaDA’s generation is an iterative denoising process:

  1. Initialize: All response positions are masked with [MASK] token

  2. Denoise: At each step, predict a subset of masked tokens

  3. Select: Choose tokens with highest confidence/lowest entropy

  4. Repeat: Continue until all masks are replaced

  5. Output: Final generated sequence

Configuration via rollout parameters:

rollout:
  steps: 64              # Number of denoising steps
  max_gen_length: 128    # Maximum response length
  block_size: 1          # Tokens unmasked per step
  temperature: 0.0       # Sampling temperature
  target: confidence     # Selection strategy
  edit_threshold: -1     # Edit mode threshold (-1=disabled)

Edit-Based Training

After standard SFT, LLaDA can be further fine-tuned with an edit_head:

  • Tokens with confidence ≥ edit_threshold get refined

  • post_unmask_steps controls additional edit iterations

  • Improves safety and response quality

training:
  unmask_confidence_threshold: 0.9

rollout:
  edit_threshold: 0.9
  post_unmask_steps: 5

Attack Methods

Three attack paradigms are supported:

  1. PAIR (Prompt Automatic Iterative Refinement) - Iterative optimization via attack/judge LLMs (GPT-4o, Claude) - Multi-stream, multi-iteration - Most effective but requires API

  2. ReNeLLM (Reasoning-enhanced Jailbreaking) - Exploits model’s own reasoning capability - Cached prompts for reuse - Strongest attack method

  3. Prefill Attack - Static harmful prompts pre-filled - No API required - Fast for batch testing

Evaluation

Integrated with AISafetyLab for standardized safety assessment:

  • Datasets: HarmBench (~400), WildGuardMix (~2000 per group A/B/C/D)

  • Scorers: LlamaGuard3 (14 categories), HarmBench (binary), OverRefusal, PatternScorer

  • Metrics: Attack Success Rate (ASR), category breakdown, overrefusal rate

Typical workflow:

python run_safety_benchmark.py \
    --model_path /path/to/checkpoint \
    --dataset harmbench \
    --scorer llama_guard_3

Use Cases

Safe dLLM is designed for:

  1. Safety Alignment Research - Measure model safety with standardized benchmarks - Track safety drift during training - Compare alignment methods

  2. Adversarial Testing - Test robustness against PAIR/ReNeLLM attacks - Build attack prompt libraries - Evaluate attack success rate (ASR)

  3. Model Training - SFT on safety datasets - CPT on large corpora - Edit-based fine-tuning for safety

  4. RLHF / Reward Modeling - Math/code reward functions - Format validation - Combined reward optimization

Getting Help

Next Steps