Installation, Toolkit & nvcc

3.1 Requirements

An NVIDIA GPU (check nvidia-smi).

CUDA Toolkit (nvcc compiler, libraries, headers, tools) from developer.nvidia.com/cuda-downloads.

A host C++ compiler: GCC/Clang on Linux, MSVC (Visual Studio) on Windows.

No GPU? Use cloud instances (AWS, GCP, Azure, Lambda), Google Colab / Kaggle notebooks (free GPUs), or remote servers.

# Linux (Ubuntu) — via NVIDIA's repository
sudo apt install cuda-toolkit
# Conda (any OS)
conda install -c nvidia cuda-toolkit
# Check
nvcc --version
nvidia-smi

3.2 nvcc — the CUDA compiler driver

nvcc splits .cu files into host code (sent to the host compiler) and device code (compiled to PTX, a virtual ISA, then to SASS, the native machine code).

nvcc hello.cu -o hello                              # compile & link
nvcc -O3 -arch=sm_80 kernel.cu -o app               # target compute capability 8.0 (A100)
nvcc -arch=native app.cu                            # target the installed GPU
nvcc -gencode arch=compute_80,code=sm_80 -gencode arch=compute_90,code=sm_90 app.cu   # fat binary
nvcc -lineinfo -G -g app.cu                         # debug info (-G disables optimizations)
nvcc --ptxas-options=-v app.cu                      # print register/shared memory usage
nvcc -ptx kernel.cu                                 # emit PTX
nvcc -std=c++17 --expt-extended-lambda app.cu
nvcc -c a.cu -o a.o && g++ main.o a.o -lcudart      # separate compilation
nvcc -rdc=true a.cu b.cu                            # relocatable device code (cross-file device calls)
nvcc -lcublas -lcufft app.cu                        # link libraries

3.3 Build systems

cmake_minimum_required(VERSION 3.24)
project(myapp LANGUAGES CXX CUDA)
set(CMAKE_CUDA_ARCHITECTURES native)        # or "80;90"
add_executable(app main.cu kernels.cu)
set_target_properties(app PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
find_package(CUDAToolkit REQUIRED)
target_link_libraries(app PRIVATE CUDA::cublas CUDA::cufft)

3.4 Editors & tools

VS Code (C/C++ + Nsight VS Code Edition), Visual Studio + Nsight Integration, CLion; Compiler Explorer supports nvcc for viewing PTX/SASS.