Locks & Synchronization
Race conditions and deadlocks are what happen when concurrent threads touch shared state without coordination — Lock, RLock, Condition, Semaphore, Event, and Barrier are the standard tools for coordinating them safely.
As soon as more than one thread can touch the same mutable state, correctness depends on coordinating access to it. Python's threading module provides a small, well-defined set of synchronization primitives — Lock, RLock, Condition, Semaphore, Event, and Barrier — each solving a specific shape of coordination problem.
Race Conditions: The Core Problem
A race condition occurs when the correctness of a program depends on the unpredictable timing/interleaving of concurrent operations. A classic example: two threads both read a shared counter's current value, each independently compute value + 1, and each write their result back — one increment can be lost because the second write overwrites the first without ever seeing it.
counter += 1 is not a single atomic operation at the bytecode level; between reading the current value and writing the incremented one, another thread can interleave and read the same stale value, causing an increment to be lost.
import threading
counter = 0
def increment() -> None:
global counter
for _ in range(100_000):
counter += 1 # NOT atomic: read, add, write — three separate steps
threads = [threading.Thread(target=increment) for _ in range(4)]
for t in threads:
t.start()
for t in threads:
t.join()
print(counter) # often NOT 400000 — lost updates from the race conditionLock: Mutual Exclusion
threading.Lock is the fundamental synchronization primitive: only one thread can hold it at a time. Code that reads-then-writes shared state should acquire the lock first, do the read-modify-write as a single critical section, then release it.
with lock: is equivalent to lock.acquire() followed by a guaranteed lock.release() (even on exception) — always prefer this context-manager form over calling acquire()/release() manually.
A Lock is not reentrant: a thread that already holds a Lock and tries to acquire it again (e.g. via recursive calls) will deadlock against itself. That's exactly the problem RLock solves.
import threading
counter = 0
lock = threading.Lock()
def increment() -> None:
global counter
for _ in range(100_000):
with lock: # acquire, then release automatically on exit
counter += 1
threads = [threading.Thread(target=increment) for _ in range(4)]
for t in threads:
t.start()
for t in threads:
t.join()
print(counter) # always 400000 — the lock makes the increment effectively atomicRLock: Reentrant Locking
threading.RLock (reentrant lock) can be acquired multiple times by the same thread without deadlocking; it tracks how many times its owning thread has acquired it and requires that many matching releases before another thread can acquire it.
With a plain Lock instead of RLock, inner()'s attempt to acquire the already-held lock (from the same thread) would block forever, since a Lock cannot tell 'the same thread again' from 'a different thread'.
Use RLock specifically when a lock must protect a piece of state accessed by methods that can call each other recursively or re-enter the same locked section from the same thread; otherwise prefer plain Lock, which is slightly cheaper.
import threading
rlock = threading.RLock()
def outer() -> None:
with rlock:
print("outer acquired")
inner()
def inner() -> None:
with rlock: # same thread re-acquiring — fine with RLock
print("inner acquired")
t = threading.Thread(target=outer)
t.start()
t.join()Deadlocks
A deadlock occurs when two or more threads each hold a resource the other needs and neither can proceed — a circular wait. The classic case: two threads acquire two locks in opposite order.
The fix is consistent lock ordering: every thread that needs both locks always acquires them in the same global order (e.g. always lock_a before lock_b), which makes the circular wait impossible.
Other deadlock-avoidance strategies: acquire all needed locks with a timeout and back off/retry on failure, or minimize the number of locks held simultaneously by keeping critical sections small and using a single coarser lock where practical.
import threading
import time
lock_a = threading.Lock()
lock_b = threading.Lock()
def thread_1() -> None:
with lock_a:
time.sleep(0.1) # gives thread_2 a chance to acquire lock_b first
with lock_b: # blocks: thread_2 holds lock_b, waiting on lock_a
print("thread_1 got both locks")
def thread_2() -> None:
with lock_b:
time.sleep(0.1)
with lock_a: # blocks: thread_1 holds lock_a, waiting on lock_b
print("thread_2 got both locks")
# Running these two functions concurrently deadlocks: neither ever finishes.Condition: Wait/Notify Coordination
threading.Condition combines a lock with the ability for a thread to wait for some condition to become true, and for another thread to notify waiters when it changes. It's the building block behind producer/consumer patterns.
wait() releases the underlying lock while blocked (so other threads can acquire it) and re-acquires it before returning; always re-check the condition in a while loop after wait() returns, since it can wake up without the condition actually being true (a 'spurious wakeup') or another thread may have consumed the item first.
import threading
import collections
queue = collections.deque()
condition = threading.Condition()
def producer() -> None:
for i in range(5):
with condition:
queue.append(i)
condition.notify() # wake one waiting consumer
def consumer() -> None:
for _ in range(5):
with condition:
while not queue: # re-check after waking (spurious wakeups possible)
condition.wait()
item = queue.popleft()
print("consumed", item)
t1 = threading.Thread(target=producer)
t2 = threading.Thread(target=consumer)
t1.start(); t2.start()
t1.join(); t2.join()Semaphore: Bounding Concurrent Access
threading.Semaphore(n) maintains an internal counter starting at n; acquire() decrements it (blocking if it would go below zero) and release() increments it. It generalizes Lock (a Semaphore(1)) to allow up to n threads through a section concurrently — useful for capping concurrent access to a limited resource, like a pool of database connections.
With 10 threads and a Semaphore(3), only 3 run the body at a time; the rest wait until a slot is released — useful for rate-limiting concurrent access to a resource like a connection pool or an external API.
import threading
import time
# At most 3 threads may run the critical section concurrently
semaphore = threading.Semaphore(3)
def limited_worker(n: int) -> None:
with semaphore:
print(f"worker {n} entered (at most 3 concurrent)")
time.sleep(1)
print(f"worker {n} leaving")
threads = [threading.Thread(target=limited_worker, args=(i,)) for i in range(10)]
for t in threads:
t.start()
for t in threads:
t.join()Event: One-Shot Signaling
threading.Event wraps a simple boolean flag with thread-safe methods: set(), clear(), is_set(), and wait(timeout). It's the simplest tool for 'wait until something happens' or cooperative stop signaling (as seen in the Threading topic).
Unlike Condition, Event doesn't require the waiting thread to hold a lock; it's simpler and appropriate when you just need a 'has this happened yet?' flag rather than coordination around shared mutable state.
import threading
import time
ready = threading.Event()
def waiter() -> None:
print("waiting for ready signal...")
ready.wait() # blocks until set() is called elsewhere
print("proceeding now that ready is set")
def setter() -> None:
time.sleep(2)
ready.set()
t1 = threading.Thread(target=waiter)
t2 = threading.Thread(target=setter)
t1.start(); t2.start()
t1.join(); t2.join()Barrier: Synchronizing a Fixed Group of Threads
threading.Barrier(n) makes n threads all wait at a checkpoint until all n have arrived, then releases them together — useful when independent parallel work must reach the same point before any of them proceeds to the next phase.
All 'reached the barrier' lines print in some order first, then all 'proceeding past the barrier' lines — no thread moves past barrier.wait() until every participant has arrived.
import threading
import time
import random
barrier = threading.Barrier(3)
def phase_worker(n: int) -> None:
time.sleep(random.uniform(0, 1))
print(f"worker {n} reached the barrier")
barrier.wait() # blocks until all 3 threads have called wait()
print(f"worker {n} proceeding past the barrier")
threads = [threading.Thread(target=phase_worker, args=(i,)) for i in range(3)]
for t in threads:
t.start()
for t in threads:
t.join()Choosing the Right Primitive
Primitive Use when...
-------------------------------------------------------------
Lock Only one thread may touch this shared state at a time
RLock Same as Lock, but the same thread may re-acquire it (recursion)
Condition Threads must wait for a state change another thread produces
Semaphore Up to N threads may access a limited resource concurrently
Event A simple one-shot "has this happened yet?" signal
Barrier A fixed group of threads must all reach a point before any proceedPractical Usage: A Thread-Safe Counter Class
Encapsulating the lock inside the class alongside the state it protects (rather than passing a lock around externally) is the standard pattern for building thread-safe objects — a preview of the Thread-Safe Design topic.
import threading
class SafeCounter:
def __init__(self) -> None:
self._value = 0
self._lock = threading.Lock()
def increment(self, amount: int = 1) -> None:
with self._lock:
self._value += amount
@property
def value(self) -> int:
with self._lock:
return self._value
counter = SafeCounter()
threads = [threading.Thread(target=lambda: [counter.increment() for _ in range(100_000)])
for _ in range(4)]
for t in threads:
t.start()
for t in threads:
t.join()
print(counter.value) # always 400000Edge Cases and Best Practices
Never rely on CPython's GIL as a substitute for explicit locking — even though the GIL serializes bytecode execution, compound operations like x += 1 still involve multiple bytecode steps that can interleave between threads. Always use with lock: rather than manual acquire()/release() pairs, since an exception between the two would otherwise leave the lock held forever. Keep critical sections (the code between acquire and release) as small as possible — holding a lock during slow I/O unnecessarily blocks every other thread waiting on it.
import threading
lock = threading.Lock()
# BAD: lock held during slow I/O, blocking all other threads unnecessarily
def slow_and_locked() -> None:
with lock:
data = fetch_from_network() # slow, doesn't need the lock
shared_state.update(data) # this is the part that actually needs protection
# BETTER: do the slow work first, hold the lock only for the shared update
def fast_and_locked() -> None:
data = fetch_from_network()
with lock:
shared_state.update(data)Key Takeaways
A race condition happens when shared mutable state is read-modified-written by multiple threads without coordination, causing lost or corrupted updates.
Lock provides mutual exclusion; RLock allows the same thread to re-acquire it (needed for recursive code).
A deadlock arises from a circular wait on multiple locks; consistent lock-acquisition ordering across all threads prevents it.
Condition supports wait/notify coordination; always re-check the condition in a while loop after wait() returns.
Semaphore bounds how many threads can access a resource concurrently; Event is a simple one-shot signal; Barrier synchronizes a fixed group of threads at a checkpoint.
Always use with lock: instead of manual acquire()/release(), and keep critical sections as small as possible.
counter = 0 def increment(): global counter for _ in range(100000): counter += 1 # not atomic counter = 0 lock = threading.Lock() def increment(): global counter for _ in range(100000): with lock: counter += 1 counter += 1 is a read-modify-write sequence of multiple bytecode operations; without a lock, threads can interleave between the read and write, losing updates.
lock.acquire() do_work() # if this raises, the lock is never released lock.release() with lock: do_work()
If do_work() raises an exception between acquire() and release(), the lock stays held forever (a permanent deadlock for anyone else waiting on it); the context-manager form guarantees release even on exception.
A circular wait (each thread holding what the other needs) is the root cause of deadlock; consistent acquisition ordering across all threads eliminates the possibility.
with condition: if not queue: condition.wait() item = queue.popleft() # queue might be empty again (spurious wakeup, or another consumer beat you to it) with condition: while not queue: condition.wait() item = queue.popleft() wait() can return without the condition truly being satisfied (spurious wakeup) or another thread may have already consumed the item; always re-check the condition in a while loop, not a single if.
Why is counter += 1 unsafe across multiple threads even though it looks like one statement?
It actually involves separate read, compute, and write steps at the bytecode level; another thread can run between those steps and read the same stale value, so one thread's increment can overwrite another's, silently losing updates.
What problem does RLock solve that a plain Lock does not?
It lets the same thread acquire the lock multiple times (e.g. via recursive function calls or nested methods) without deadlocking against itself, tracking an acquisition count so it isn't truly released until the same thread has released it an equal number of times.
What are the two necessary ingredients of a deadlock involving locks, and how is it typically avoided?
A deadlock needs a circular wait — at least two threads each holding a resource the other needs — usually caused by acquiring multiple locks in inconsistent order; it's typically avoided by having every thread acquire shared locks in the same fixed global order.
Why must a Condition's wait() be called inside a while loop rather than a single if check?
wait() can return due to a spurious wakeup (unrelated to the actual condition becoming true) or the condition it signaled may already have been consumed by another thread by the time this thread resumes, so the condition must be re-verified after every wait() return.
What is the difference between Semaphore and Lock?
A Lock allows exactly one thread through at a time (equivalent to Semaphore(1)); a Semaphore(n) allows up to n threads to hold it concurrently, useful for capping concurrent access to a limited resource.
Why should critical sections (code inside a with lock: block) be kept as small as possible?
Every other thread waiting on that lock is blocked for the entire duration it's held; including slow operations like network calls or disk I/O inside the locked section unnecessarily serializes work that didn't need to be protected.
Can you rely on CPython's GIL instead of explicit locks to make compound operations on shared state safe?
No — the GIL only guarantees that one thread executes Python bytecode at a time, but a single Python statement like x += 1 still compiles to multiple bytecode instructions that can be interleaved with another thread's; explicit locking around the whole logical operation is still required, and relying on the GIL is a CPython-specific implementation detail, not a portable guarantee.
Write a thread-safe BankAccount class with deposit(amount) and withdraw(amount) methods that never let the balance go negative, using a Lock.
import threading class BankAccount: def __init__(self, balance: int = 0) -> None: self._balance = balance self._lock = threading.Lock() def deposit(self, amount: int) -> None: with self._lock: self._balance += amount def withdraw(self, amount: int) -> bool: with self._lock: if amount > self._balance: return False self._balance -= amount return True @property def balance(self) -> int: with self._lock: return self._balance
The lock wraps the entire check-then-modify sequence in withdraw() as one atomic critical section, preventing two concurrent withdrawals from both passing the balance check based on a stale value and together overdrawing the account.
If f() and g() are run concurrently on separate threads, what can happen, and how would you fix it?
They can deadlock: if f()'s thread acquires lock_a while g()'s thread simultaneously acquires lock_b, each then blocks waiting for the lock the other holds, and neither ever proceeds. The fix is to make both functions acquire the locks in the same order (e.g. always lock_a then lock_b).
Write a producer/consumer pair using threading.Condition where the producer adds items to a shared list and the consumer waits for and removes one item at a time, printing each consumed item.
import threading items = [] condition = threading.Condition() def producer(n: int) -> None: for i in range(n): with condition: items.append(i) condition.notify() def consumer(n: int) -> None: for _ in range(n): with condition: while not items: condition.wait() print("consumed", items.pop(0))
The producer appends under the lock and calls notify() to wake a waiting consumer; the consumer waits in a while loop (re-checking items is non-empty after each wakeup) before popping, ensuring it never pops from an empty list even under spurious wakeups or multiple consumers.
import threading
lock_a = threading.Lock()
lock_b = threading.Lock()
def f():
with lock_a:
with lock_b:
pass
def g():
with lock_b:
with lock_a:
pass