Language and Runtime Concurrency Models: C and POSIX

C gives you the raw machinery the other languages are built on: threads, locks and atomics with almost no safety net.

Learning Objectives

Use POSIX threads, mutexes and condition variables, POSIX semaphores, C11 atomics and memory ordering, and thread-local storage.

Handle signals in multithreaded programs, cancellation and cleanup handlers, fork, pipes and shared memory.

Find bugs with ThreadSanitizer and debuggers.

Plain-Language Explanation

C gives you the raw machinery the other languages are built on: threads, locks and atomics with almost no safety net. Every rule from the earlier modules applies with full force — and a data race is not just a wrong answer but undefined behavior. The payoff is complete control and the ability to see exactly what the operating system provides.

Precise Technical Explanation

pthread_create, pthread_join, pthread_detach, pthread_self, attributes (stack size, detach state, scheduling). POSIX, not ISO C: available on Linux, macOS, BSDs; Windows needs a compatibility library. C11 added optional <threads.h> (thrd_create, mtx_t, cnd_t), implemented by glibc 2.28+ and musl but not by MSVC until recently.

Mutexes and condition variables pthread_mutex_t (types: normal, error-checking, recursive, robust; protocols: none, inherit, protect), pthread_cond_t with pthread_cond_wait in a while loop, pthread_cond_timedwait, pthread_rwlock_t, pthread_spinlock_t, pthread_barrier_t, pthread_once.

Unnamed (sem_init — shared between threads, or between processes if placed in shared memory with pshared=1) and named (sem_open("/name")). sem_wait is a cancellation point and may return EINTR. macOS does not support unnamed semaphores.

<stdatomic.h>: _Atomic types, atomic_load/store/fetch_add/exchange/compare_exchange_weak/strong with _explicit variants, atomic_flag, atomic_thread_fence, atomic_is_lock_free.

memory_order_relaxed, consume, acquire, release, acq_rel, seq_cst — same semantics as C++ (Modules 6–7).

_Thread_local (C11; thread_local keyword in C23), GCC's __thread, pthread_key_create with destructors for dynamically created keys.

Signals and thread safety caveats Signal dispositions are process-wide; delivery goes to one thread that does not block the signal. Robust pattern: block signals in all threads with pthread_sigmask before creating them, then dedicate one thread to sigwait (demonstrated below). Handlers may call only async-signal-safe functions.

Cancellation and cleanup handlers pthread_cancel with deferred cancellation acts at cancellation points (sleep, read, pthread_cond_wait, …); pthread_cleanup_push/pop register handlers that run on cancellation (unlock mutexes, free memory). Module 14 showed what happens without them; below, a cleanup handler unlocks the mutex.

fork, pipes and shared memory fork + pipe + waitpid; shm_open + mmap or mmap(MAP_SHARED|MAP_ANONYMOUS) + a PTHREAD_PROCESS_SHARED mutex for cross-process locking. After fork in a multithreaded process, only async-signal-safe calls until exec.

Thread sanitizers and debuggers -fsanitize=thread (Modules 3, 9 of the Memory course), Helgrind/DRD, gdb (info threads, thread apply all bt), strace -f, perf.

Key Terms

Run an initialiser once Reports EOWNERDEAD if the owner died holding it Mutex/condition usable across processes

Function where deferred cancellation can act Function run when a thread is cancelled or exits Synchronously wait for a blocked signal

Complete Code Example

A bounded queue (mutex + two condition variables), a POSIX semaphore, C11 <threads.h> with atomics, a dedicated signal-handling thread, and a cancellation cleanup handler (GCC 11.2, glibc 2.35, Linux):

Cross-process synchronization: a PTHREAD_PROCESS_SHARED mutex in shared memory protecting a counter incremented by a parent and two forked children:

#include <pthread.h>
#include <semaphore.h>
#include <signal.h>
#include <stdatomic.h>
#include <stdio.h>
#include <threads.h>
#include <unistd.h>

/* 1. bounded queue */
#define CAP 4
static int buf[CAP], head, tail, count;
static pthread_mutex_t qm = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t not_full = PTHREAD_COND_INITIALIZER, not_empty = PTHREAD_COND_INITIALIZER;

static void put(int v)
{
    pthread_mutex_lock(&qm);
    while (count == CAP) pthread_cond_wait(&not_full, &qm);      /* predicate loop */
    buf[tail] = v; tail = (tail + 1) % CAP; count++;
    pthread_cond_signal(&not_empty);
    pthread_mutex_unlock(&qm);
}

static int take(void)
{
    pthread_mutex_lock(&qm);
    while (count == 0) pthread_cond_wait(&not_empty, &qm);
    int v = buf[head]; head = (head + 1) % CAP; count--;
    pthread_cond_signal(&not_full);
    pthread_mutex_unlock(&qm);
    return v;
}

static void *consumer(void *arg)
{
    long *sum = arg;
    for (int i = 0; i < 1000; ++i) *sum += take();
    return NULL;
}

/* 2. C11 threads + atomics */
static atomic_long hits;
static int c11_worker(void *arg)
{
    (void)arg;
    for (int i = 0; i < 100000; ++i) atomic_fetch_add_explicit(&hits, 1,
…

bounded queue: consumer sum 500500 (expected 500500)
semaphore(2): third sem_trywait failed (no permits left)
C11 threads + atomic_fetch_add: 400000 (lock-free: yes)
cleanup handler: mutex unlocked during cancellation
after cancellation the mutex is free (cleanup worked)
signal thread: received SIGTERM, starting graceful shutdown

#include <pthread.h>
#include <stdio.h>
#include <sys/mman.h>
#include <sys/wait.h>
#include <unistd.h>

struct Shared { pthread_mutex_t lock; long counter; };

int main(void)
{
    struct Shared *sh = mmap(NULL, sizeof *sh, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
    if (sh == MAP_FAILED) return 1;
    pthread_mutexattr_t attr;
    pthread_mutexattr_init(&attr);
    pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);   /* usable across processes */
    pthread_mutex_init(&sh->lock, &attr);
    sh->counter = 0;

    for (int p = 0; p < 2; ++p) {
        if (fork() == 0) {
            for (int i = 0; i < 100000; ++i) {
                pthread_mutex_lock(&sh->lock);
                sh->counter++;
                pthread_mutex_unlock(&sh->lock);
            }
            _exit(0);
        }
    }
    for (int i = 0; i < 100000; ++i) { pthread_mutex_lock(&sh->lock); sh->counter++; pthread_mutex_unlock(&sh->lock); }
    while (wait(NULL) > 0) {}
    printf("3 processes x 100000 locked increments in shared memory = %ld\n", sh->counter);
    munmap(sh, sizeof *sh);
    return 0;
}

3 processes x 100000 locked increments in shared memory = 300000

Line-by-Line Explanation

The queue's two condition variables let producers wait for "not full" and consumers for "not empty"; every wait sits in a while loop under the mutex.

sem_trywait fails when no permits remain instead of blocking.

C11 thrd_create and atomic_fetch_add_explicit work on glibc 2.28+; the counter is exact.

pthread_sigmask before creating threads means every thread inherits the blocked mask, so SIGTERM can only be consumed by sigwait — normal code, no async-signal-safety constraints.

pthread_cleanup_push registers unlock_on_cancel; when cancellation hits the sleep, the handler releases the mutex (compare Module 14).

The process-shared mutex lives in a MAP_SHARED mapping inherited by both children; all 300,000 increments are serialised across three processes.

Interleaving or State Trace

Signal flow: kill(getpid(), SIGTERM) → the kernel finds a thread with SIGTERM unblocked — none, because all block it — so the signal stays pending until sigwait in the signal thread consumes it.

Correctness and Safety Analysis

Each mechanism keeps one invariant: queue bounds (0 ≤ count ≤ CAP) under qm; counter exactness via atomics or the process-shared mutex; mutex release on cancellation via the cleanup handler; signal handling confined to one thread.

Common Mistakes and Misconceptions

Treating pthread_* return values like errno.

if around pthread_cond_wait.

Calling printf/malloc in signal handlers.

Cancelling threads without cleanup handlers.

Process-shared mutexes in non-shared memory (or forgetting the attribute).

Assuming <threads.h> exists everywhere.

Failure Modes and Debugging Steps

gdb -p + thread apply all bt for hangs; -fsanitize=thread for races; error-checking mutexes (PTHREAD_MUTEX_ERRORCHECK) to catch unlock-by-non-owner and self-deadlock; robust mutexes to recover when a process dies holding a shared lock.

Performance and Scalability Implications

glibc mutexes are futex-based: uncontended lock/unlock stay in user space (~20 ns); contended ones enter the kernel. Process-shared mutexes cost the same; the IPC win of shared memory is avoiding copies.

Security Considerations

Shared memory between processes of different trust levels must be treated as hostile input (validate everything you read). Signal handlers are a classic source of exploitable bugs; prefer sigwait/signalfd.

Platform and Runtime Caveats

Linux/glibc specifics: <threads.h>, unnamed semaphores, signalfd; macOS lacks unnamed POSIX semaphores and pthread_barrier_t.

Windows: use Win32 (CreateThread, SRWLOCK, CONDITION_VARIABLE, InterlockedXxx) or C11 threads in recent MSVC.

Real-World Applications

Database engines (PostgreSQL's shared buffers and LWLocks), web servers, language runtimes (CPython, Node's libuv), embedded Linux daemons.

Best Practices

Check every return code; prefer error-checking mutexes in debug builds.

Block signals in worker threads; handle them in one thread.

Prefer cooperative cancellation; if you use pthread_cancel, push cleanup handlers.

Run TSan in CI.

Exercises

E1 · Beginner. Why must pthread_sigmask be called *before* creating worker threads?

Answer: New threads inherit the creating thread's signal mask; blocking afterwards would leave earlier threads able to receive the signal asynchronously.

E2 · Intermediate. A process dies while holding a process-shared mutex. What happens to the other processes, and which mutex type helps?

Answer: With a normal mutex they block forever. A robust mutex (pthread_mutexattr_setrobust) makes the next locker get EOWNERDEAD; it can repair the shared state and call pthread_mutex_consistent.

E3 · Advanced challenge. Replace the bounded queue's two condition variables with two POSIX semaphores plus a mutex.

Answer: sem_t empty_slots = CAP, full_slots = 0. put: sem_wait(&empty_slots); lock; insert; unlock; sem_post(&full_slots). take: sem_wait(&full_slots); lock; remove; unlock; sem_post(&empty_slots) — the classic Dijkstra solution.

Quiz

? Is pthread part of ISO standard C?
- Yes, since C11 :: C11 added the optional <threads.h>, not pthreads.
+ No, it is a POSIX API :: Available on POSIX systems; Windows needs other APIs or a compatibility layer.
- Yes, since C89 :: C89 had no threads.
- Only on Windows :: The reverse.

? What does pthread_cleanup_push provide?
+ A handler that runs if the thread is cancelled (or pops it with execute=1). :: Releases resources during cancellation.
- Automatic garbage collection :: C has none.
- Signal masking :: That is pthread_sigmask.
- Thread-local storage :: That is pthread_key_create / _Thread_local.

? Which are async-signal-safe and usable in a signal handler? (select all that apply)
+ write() :: Listed as async-signal-safe.
+ setting a volatile sig_atomic_t flag :: The canonical safe action.
- printf() :: Not async-signal-safe.
- malloc() :: Not async-signal-safe.

Summary

POSIX C exposes the full machinery: pthreads, mutexes (including process-shared and robust), condition variables, semaphores, C11 atomics and threads, TLS, signal masks with sigwait, cancellation with cleanup handlers, fork, pipes and shared memory — with data races as undefined behavior and TSan as the essential tool.