Installation

This guide covers installing Safe dLLM from source.

System Requirements

Component

Minimum

Recommended

Python

3.10

3.10 or 3.11

CUDA

12.1

12.6+

GPU Memory

16GB

40GB+ (A100/H100)

Disk Space

20GB

100GB+ (for datasets & checkpoints)

OS

Linux

Ubuntu 20.04+ / CentOS 7+

Step-by-Step Installation

1. Clone Repository

git clone https://github.com/your-org/safe_dllm.git
cd safe_dllm

# Initialize submodules (AISafetyLab)
git submodule update --init --recursive

2. Create Virtual Environment

Using uv (recommended, fastest):

uv venv .venv --python 3.10
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

Using venv:

python -m venv .venv
source .venv/bin/activate

3. Install Core Dependencies

# Install project dependencies
uv pip install -e .

# Or with pip
pip install -e .

This installs:

  • torch==2.6.0

  • transformers==4.52.4

  • accelerate>=1.12.0

  • datasets>=4.5.0

  • peft>=0.18.1

  • omegaconf>=2.3.0

  • wandb>=0.24.2

  • And all other requirements from pyproject.toml

4. Install AISafetyLab

cd AISafetyLab
pip install -e .
cd ..

This installs the attack and evaluation framework.

5. Verify Installation

Run the quick test:

python quick_test_dataset.py

Expected output:

============================================================
Testing HarmBench dataset
============================================================
✅ AISafetyLab imported successfully
📥 Loading HarmBench...
✅ Loaded 400 samples
✅ All checks passed!

Optional: Install Sphinx (for building docs)

pip install sphinx sphinx-rtd-theme sphinx-autodoc-typehints

# Build HTML docs
cd docs
make html

# View in browser
open _build/html/index.html

Environment Variables

Optional environment variables for tuning:

# HuggingFace cache location
export HF_HOME="/scratch/yourname/.cache/huggingface"

# Torch extensions (for flash-attn, etc.)
export TORCH_EXTENSIONS_DIR="/scratch/yourname/.cache/torch_extensions"

# Triton cache
export TRITON_CACHE_DIR="/scratch/yourname/.cache/triton"

# W&B offline mode (for clusters without internet)
export WANDB_MODE=offline

# Debug mode (more verbose logging)
export DEBUG_MODE="1"

# HuggingFace timeout (for slow connections)
export HF_HUB_TIMEOUT=60

Cluster Installation (Slurm)

On a Slurm cluster, add these to your job script:

#!/bin/bash
#SBATCH --gres=gpu:4
#SBATCH --cpus-per-task=24
#SBATCH --mem=512G

module purge
module load cuda/12.6.3
source /path/to/.venv/bin/activate

export TORCH_EXTENSIONS_DIR="/scratch/$USER/.cache/torch_extensions"
export TRITON_CACHE_DIR="/scratch/$USER/.cache/triton"
export HF_HOME="/scratch/$USER/.cache/huggingface"

Common Installation Issues

Issue: flash-attn installation fails

Solution: Install from pre-built wheel:

# Download wheel from GitHub releases
pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu12torch2.6cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

Issue: CUDA errors / torch.cuda.is_available() returns False

Solution: Verify CUDA installation:

nvcc --version
python -c "import torch; print(torch.cuda.is_available())"

If False, reinstall PyTorch with correct CUDA version:

pip install torch==2.6.0+cu126 torchvision==0.21.0+cu126 --index-url https://download.pytorch.org/whl/cu126

Issue: AISafetyLab import errors

Solution: Ensure AISafetyLab is installed in editable mode:

cd AISafetyLab
pip install -e .
cd ..

# Test import
python -c "import aisafetylab; print('OK')"

Issue: Out of memory during import

Solution: Reduce GPU allocation or enable model offloading:

import torch
torch.cuda.set_per_process_memory_fraction(0.9)  # Limit to 90%

Next Steps

✅ Installation complete? Move on to Quick Start!