feat: add Docker support for NVIDIA RTX 5000 series (Blackwell / sm_120)#250
Open
ziobudda wants to merge 1 commit intoOpenBMB:mainfrom
Open
feat: add Docker support for NVIDIA RTX 5000 series (Blackwell / sm_120)#250ziobudda wants to merge 1 commit intoOpenBMB:mainfrom
ziobudda wants to merge 1 commit intoOpenBMB:mainfrom
Conversation
Adds Dockerfile and docker-compose.yml targeting CUDA 13.0 with native sm_120 (Blackwell) support. PyTorch is force-reinstalled from the cu130 index to ensure the correct CUDA version wins over transitive CPU deps. Model weights are cached in a named volume to survive container restarts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Dockerfilebased onnvidia/cuda:13.0.1-cudnn-devel-ubuntu22.04with native sm_120 (Blackwell) supportdocker-compose.ymlwith GPU passthrough, named volume for model cache, and optionalVOXCPM_MODEL_IDoverridecu130index last, so the correct CUDA version always wins over transitive CPU deps pulled by packages likefunasrMotivation
CUDA 12.x does not ship Triton wheels with native sm_120 support, causing silent fallbacks or crashes on RTX 5000 series (Blackwell) GPUs. CUDA 13.0 resolves this and allows VoxCPM2 to run at full speed on RTX 5070/5080/5090.
Changes
Dockerfiledocker-compose.ymlUsage
docker compose up --build Override the model: VOXCPM_MODEL_ID=openbmb/VoxCPM2 docker compose up Test plan - docker compose build completes without errors on a host with nvidia-container-toolkit - nvidia-smi inside the container shows the RTX 5xxx GPU - App starts and responds on http://localhost:8808 - Model weights are cached in the named volume and survive docker compose restart