Installation
This guide covers installing Safe dLLM from source.
System Requirements
Component |
Minimum |
Recommended |
|---|---|---|
Python |
3.10 |
3.10 or 3.11 |
CUDA |
12.1 |
12.6+ |
GPU Memory |
16GB |
40GB+ (A100/H100) |
Disk Space |
20GB |
100GB+ (for datasets & checkpoints) |
OS |
Linux |
Ubuntu 20.04+ / CentOS 7+ |
Step-by-Step Installation
1. Clone Repository
git clone https://github.com/your-org/safe_dllm.git
cd safe_dllm
# Initialize submodules (AISafetyLab)
git submodule update --init --recursive
2. Create Virtual Environment
Using uv (recommended, fastest):
uv venv .venv --python 3.10
source .venv/bin/activate # On Windows: .venv\Scripts\activate
Using venv:
python -m venv .venv
source .venv/bin/activate
3. Install Core Dependencies
# Install project dependencies
uv pip install -e .
# Or with pip
pip install -e .
This installs:
torch==2.6.0
transformers==4.52.4
accelerate>=1.12.0
datasets>=4.5.0
peft>=0.18.1
omegaconf>=2.3.0
wandb>=0.24.2
And all other requirements from
pyproject.toml
4. Install AISafetyLab
cd AISafetyLab
pip install -e .
cd ..
This installs the attack and evaluation framework.
5. Verify Installation
Run the quick test:
python quick_test_dataset.py
Expected output:
============================================================
Testing HarmBench dataset
============================================================
✅ AISafetyLab imported successfully
📥 Loading HarmBench...
✅ Loaded 400 samples
✅ All checks passed!
Optional: Install Sphinx (for building docs)
pip install sphinx sphinx-rtd-theme sphinx-autodoc-typehints
# Build HTML docs
cd docs
make html
# View in browser
open _build/html/index.html
Environment Variables
Optional environment variables for tuning:
# HuggingFace cache location
export HF_HOME="/scratch/yourname/.cache/huggingface"
# Torch extensions (for flash-attn, etc.)
export TORCH_EXTENSIONS_DIR="/scratch/yourname/.cache/torch_extensions"
# Triton cache
export TRITON_CACHE_DIR="/scratch/yourname/.cache/triton"
# W&B offline mode (for clusters without internet)
export WANDB_MODE=offline
# Debug mode (more verbose logging)
export DEBUG_MODE="1"
# HuggingFace timeout (for slow connections)
export HF_HUB_TIMEOUT=60
Cluster Installation (Slurm)
On a Slurm cluster, add these to your job script:
#!/bin/bash
#SBATCH --gres=gpu:4
#SBATCH --cpus-per-task=24
#SBATCH --mem=512G
module purge
module load cuda/12.6.3
source /path/to/.venv/bin/activate
export TORCH_EXTENSIONS_DIR="/scratch/$USER/.cache/torch_extensions"
export TRITON_CACHE_DIR="/scratch/$USER/.cache/triton"
export HF_HOME="/scratch/$USER/.cache/huggingface"
Common Installation Issues
Issue: flash-attn installation fails
Solution: Install from pre-built wheel:
# Download wheel from GitHub releases
pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu12torch2.6cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
Issue: CUDA errors / torch.cuda.is_available() returns False
Solution: Verify CUDA installation:
nvcc --version
python -c "import torch; print(torch.cuda.is_available())"
If False, reinstall PyTorch with correct CUDA version:
pip install torch==2.6.0+cu126 torchvision==0.21.0+cu126 --index-url https://download.pytorch.org/whl/cu126
Issue: AISafetyLab import errors
Solution: Ensure AISafetyLab is installed in editable mode:
cd AISafetyLab
pip install -e .
cd ..
# Test import
python -c "import aisafetylab; print('OK')"
Issue: Out of memory during import
Solution: Reduce GPU allocation or enable model offloading:
import torch
torch.cuda.set_per_process_memory_fraction(0.9) # Limit to 90%
Next Steps
✅ Installation complete? Move on to Quick Start!