CUDA-Demux

CUDA-Demux converts Illumina BCL/CBCL run folders directly into per-sample FASTQ files on the GPU. CBCL ingestion runs in parallel across cycles with OpenMP, base calling and barcode matching execute as CUDA kernels, and matched reads stream straight into per-(sample, lane) gzipped FASTQ writers — no per-cluster std::string allocations, no batched-then-merged buffers.

Features

Performance

Downloads

Latest stable release: v1.2.1 — corrected CBCL decoding (non-PF-excluded tiles, header quality bins, no-calls), i5 orientation from RunInfo.xml, per-index barcode mismatches, and a parallel writer that takes a 32.2 M-cluster MiSeq i100 run end to end in 8 s. Output is verified read by read against an independent reference decoder.

v1.2.1 decodes identically to v1.2.0; its packages are simply built with libdeflate linked statically, which is the difference between 8.1 s and 11.2 s on that run. The v1.2.0 packages are correct, just slower.

v1.1.0 and v1.0.1 produce incorrect FASTQ and have been superseded. Their CBCL decoder ignored the per-file non-PF-excluded flag, so reads are wrong from roughly cycle 26 (NovaSeq) or 46 (MiSeq i100) onward, and on layouts with the index cycles in that region the sample assignments are wrong too. Re-run anything demultiplexed with them using v1.2.1.

Installation

Prerequisites

• NVIDIA GPU with compute capability 8.0, 8.6, or 8.9 (A100 / RTX 30-series / RTX 40-series — adjust -DCMAKE_CUDA_ARCHITECTURES for other targets)
• CUDA toolkit 12.x or 13.x and a matching NVIDIA driver
• g++ 13 or 14 (CUDA 13 + g++-15 is not yet supported by nvcc)
• CMake ≥ 3.16 and Ninja (or Make)
• zlib, OpenMP runtime, TinyXML2

Docker (recommended)

The repository ships a Dockerfile that pins a compatible nvidia/cuda:13.0.1-devel-ubuntu24.04 toolchain, installs all build dependencies, runs the unit tests, and produces a ready-to-run image:

# Build the image (sets CUDA arch to 8.6 by default; override with --build-arg CUDA_ARCH=80)
git clone https://github.com/mmorri/cuda-demux.git
cd cuda-demux
docker build -t cuda-demux:dev .

# Run on a host run folder; mount input read-only and an output dir
docker run --rm --gpus all \
  -v /path/to/RunFolder:/work/run:ro \
  -v /path/to/output:/work/out \
  cuda-demux:dev \
  --input /work/run \
  --samplesheet /work/run/SampleSheet.csv \
  --output /work/out \
  --gzip

The image's ENTRYPOINT is the cuda-demux binary, so any of the CLI flags below can be appended to docker run.

Package installation (v1.2.1)

wget https://github.com/mmorri/cuda-demux/releases/download/v1.2.1/cuda-demux_1.2.1_amd64.deb
sudo apt install ./cuda-demux_1.2.1_amd64.deb
# Or:
sudo dpkg -i cuda-demux_1.2.1_amd64.deb
sudo apt-get install -f

wget https://github.com/mmorri/cuda-demux/releases/download/v1.2.1/cuda-demux-1.2.1-1.x86_64.rpm
sudo dnf install ./cuda-demux-1.2.1-1.x86_64.rpm
# Older systems:
sudo yum install ./cuda-demux-1.2.1-1.x86_64.rpm

Build from source

# Debian/Ubuntu — install build dependencies
sudo apt install -y \
    cmake ninja-build \
    g++-14 \
    libtinyxml2-dev libomp-dev zlib1g-dev

git clone https://github.com/mmorri/cuda-demux.git
cd cuda-demux

cmake -S . -B build -G Ninja \
    -DCMAKE_BUILD_TYPE=Release \
    -DBUILD_TESTING=ON \
    -DCMAKE_CUDA_ARCHITECTURES=86         # 80 for A100, 89 for RTX 40xx

cmake --build build -j
ctest --test-dir build --output-on-failure

# Resulting binary:
./build/cuda-demux --input ... --samplesheet ... --output ...

Usage

Basic command

cuda-demux \
    --input /path/to/RunFolder \
    --samplesheet /path/to/SampleSheet.csv \
    --output /path/to/fastq_output \
    --gzip

--input must point at the run folder root (the directory containing RunInfo.xml, RunParameters.xml, and Data/Intensities/BaseCalls/). The sample sheet is an Illumina v2 SampleSheet ([BCLConvert_Data]) or a legacy v1 sheet ([Data]).

Command-line options

A read is assigned when its closest barcode is within --barcode-mismatches in each index separately and beats the runner-up by at least one mismatch (bcl-convert semantics). CPU thread count for CBCL ingest and FASTQ compression is controlled by OMP_NUM_THREADS (defaults to all cores).

Example

cuda-demux \
    --input /data/runs/250712_NovaSeq_RunA \
    --samplesheet /data/runs/250712_NovaSeq_RunA/SampleSheet.csv \
    --output /scratch/RunA_fastq \
    --gzip \
    --device 0 \
    --gpu-mem-fraction 0.5

Environment variables

All CLI tunables are also reachable as env vars (the CLI flags simply set them before main runs). A few additional knobs exist only as env vars:

Output layout

Files in the output directory are named:

<Sample_ID>_L<lane>_R1_001.fastq[.gz]
<Sample_ID>_L<lane>_R2_001.fastq[.gz]   # paired-end runs only
undetermined_L<lane>_R1_001.fastq[.gz]
undetermined_L<lane>_R2_001.fastq[.gz]

One pair of files per sample per lane (created even when a sample receives no reads). Reads that did not match any sample within the mismatch budget are written to undetermined_*. Records are emitted in input cluster order and named @<Sample_ID>_<n>/1; no-calls are N with quality #, and quality scores come from each CBCL header's bin table. Every write is checked, so a disk-full or quota error fails loudly rather than silently truncating.

Contributing

Contributions are welcome — issues, feature requests, and pull requests on the GitHub repository.