Appendices

Cheat sheet, runtime API quick reference, architecture feature table, learning resources and suggested study plan (12 weeks).

Appendices

41.1 Cheat sheet 41.2 Runtime API quick reference cudaGetDeviceCount, cudaSetDevice, cudaGetDeviceProperties, cudaDeviceGetAttribute, cudaDeviceReset, cudaDeviceSetLimit

cudaMalloc, cudaFree, cudaMemcpy[Async], cudaMemset[Async], cudaMallocHost, cudaHostAlloc, cudaHostRegister, cudaMallocManaged, cudaMemPrefetchAsync, cudaMemAdvise, cudaMallocAsync, cudaFreeAsync, cudaMallocPitch, cudaMemcpy2D, cudaMalloc3D, cudaMemGetInfo

cudaStreamCreate[WithFlags/WithPriority], cudaStreamDestroy, cudaStreamSynchronize, cudaStreamQuery, cudaStreamWaitEvent, cudaLaunchHostFunc

cudaEventCreate[WithFlags], cudaEventRecord, cudaEventSynchronize, cudaEventElapsedTime, cudaEventQuery, cudaEventDestroy

<<<>>>, cudaLaunchKernel, cudaLaunchKernelEx (clusters, attributes), cudaLaunchCooperativeKernel, cudaFuncSetAttribute, cudaFuncGetAttributes

cudaOccupancyMaxPotentialBlockSize, cudaOccupancyMaxActiveBlocksPerMultiprocessor

cudaStreamBeginCapture, cudaStreamEndCapture, cudaGraphInstantiate, cudaGraphLaunch, cudaGraphExecUpdate

cudaDeviceCanAccessPeer, cudaDeviceEnablePeerAccess, cudaMemcpyPeer[Async], cudaIpcGetMemHandle, cudaIpcOpenMemHandle

cudaCreateTextureObject, cudaDestroyTextureObject, cudaMallocArray, cudaCreateSurfaceObject

cudaGetLastError, cudaPeekAtLastError, cudaGetErrorString, cudaGetErrorName 41.3 Architecture feature table

Unified memory with page faulting, FP16 arithmetic Tensor Cores (FP16), independent thread scheduling INT8/INT4 Tensor Cores

TF32/BF16 Tensor Cores, async copy, L2 persistence, MIG, 2:4 sparsity

TMA, thread block clusters, DSMEM, FP8 Transformer Engine, wgmma FP4/FP6, 5th-gen Tensor Cores, tcgen05 41.4 Learning resources

Official: CUDA C++ Programming Guide, CUDA C++ Best Practices Guide, CUDA Runtime API reference, Nsight documentation, NVIDIA Developer Blog, NVIDIA DLI courses, GTC talks (on-demand), cuda-samples (GitHub), CCCL docs

*Programming Massively Parallel Processors* (4th ed.) — Hwu, Kirk, El Hajj *CUDA by Example* — Sanders & Kandrot

*Professional CUDA C Programming* — Cheng, Grossman, McKercher

*CUDA Programming: A Developer's Guide to Parallel Computing with GPUs* — Shane Cook *The CUDA Handbook* — Nicholas Wilt *Learn CUDA Programming* — Han & Sharma *GPU Gems* series (free online)

Online: Oak Ridge/NERSC CUDA training series, "An Even Easier Introduction to CUDA" (NVIDIA blog), Simon Boehm's "How to Optimize a CUDA Matmul Kernel", Mark Harris's reduction slides, GPU MODE community lectures, leetgpu.com / Tensara (GPU programming challenges)

Community: NVIDIA Developer Forums, Stack Overflow [cuda], r/CUDA, GPU MODE Discord 41.5 Suggested study plan (12 weeks)

GPU computing concepts, architecture, toolkit, first kernels

Thread hierarchy, memory basics, errors, device query — Part I exercises

Memory hierarchy, coalescing, bandwidth measurement Shared memory, tiling, synchronization, atomics — Project 1

Warp programming, reduction, scan, histogram, stencils Constant/texture memory, streams, pinned & unified memory, occupancy — Part II exercises, Project 2

Profiling with Nsight, roofline, optimization techniques Cooperative groups, dynamic parallelism, graphs, memory pools

Mixed precision, Tensor Cores, modern features — Project 3 CUDA C++, libraries, Thrust & CUB

Multi-GPU, debugging, PTX/NVRTC, interop — Project 4 or 5 Python GPU programming, portability, case studies, best practices — Project 6 🎓 Final Words

You've gone from launching your first <<<1, 1>>> kernel to tiling, warp shuffles, streams, CUDA graphs, Tensor Cores, multi-GPU communication, and Python GPU programming. GPU performance comes from understanding the hardware: keep thousands of threads busy, move data as little as possible, and let the profiler guide you. Build the capstone projects, read *Programming Massively Parallel Processors*, and join the GPU programming community. Happy (massively parallel) hacking! 🟩

// Qualifiers
__global__ void k(...);            // kernel (host-launched)
__device__ float f(float);         // device function
__host__ __device__ int g(int);    // both
__shared__ float s[256];           // per-block shared memory
__constant__ float c[64];          // constant memory
extern __shared__ float dyn[];     // dynamic shared memory

// Launch & indices
k<<<grid, block, shmemBytes, stream>>>(args);
int i = blockIdx.x * blockDim.x + threadIdx.x;
for (int i = idx; i < n; i += blockDim.x * gridDim.x) { /* grid-stride */ }

// Memory
cudaMalloc(&d, bytes); cudaFree(d);
cudaMemcpy(dst, src, bytes, cudaMemcpyHostToDevice);
cudaMemcpyAsync(dst, src, bytes, kind, stream);
cudaMallocHost(&h, bytes); cudaMallocManaged(&u, bytes); cudaMallocAsync(&d, bytes, stream);
cudaMemcpyToSymbol(c, h_c, sizeof(h_c));

// Sync
__syncthreads(); __syncwarp(); cudaDeviceSynchronize(); cudaStreamSynchronize(s);

// Atomics & warp ops
atomicAdd(&x, 1); atomicCAS(&x, old, new);
__shfl_down_sync(0xffffffff, v, offset); __ballot_sync(mask, pred); __any_sync(mask, pred);

// Streams & events
cudaStreamCreate(&s); cudaEventCreate(&e); cudaEventRecord(e, s);
cudaEventElapsedTime(&ms, start, stop); cuda
…