Synchronization and race conditions

A race condition occurs when correctness depends on the timing of concurrent operations. Async tasks run on one thread by default, but they can still race whenever an invariant spans an await.

An async race

import asyncio

balance = 100

async def withdraw(amount: int) -> None:
    global balance
    if balance >= amount:
        await asyncio.sleep(0)  # another task can change balance here
        balance -= amount

Two withdrawals can both pass the check. Protect the complete check-and-update operation:

lock = asyncio.Lock()

async def withdraw(amount: int) -> bool:
    global balance
    async with lock:
        if balance < amount:
            return False
        balance -= amount
        return True

No await is needed while changing a plain integer, so the critical section stays short. asyncio.Lock is fair among waiting tasks, but it is not a thread lock and is not thread-safe.

Keep critical sections small

Do not hold a lock while making a slow network request. Snapshot protected state, release the lock, perform I/O, then reacquire and validate before committing if needed. Never assume a value remained unchanged while the lock was released.

Avoid acquiring multiple locks. If necessary, define and consistently follow one global lock order to prevent deadlock. Locks are not reentrant: a task that acquires the same asyncio.Lock twice waits forever.

Semaphores limit capacity

import asyncio

limit = asyncio.Semaphore(10)

async def bounded_fetch(url: str) -> bytes:
    async with limit:
        return await fetch(url)

A semaphore limits concurrent entries; it does not enforce a request rate per second. Use BoundedSemaphore during development when releasing too many times should be detected. Place the limit around the scarce resource, not around unrelated preparation.

Events and conditions

An Event is a level-triggered flag:

ready = asyncio.Event()

async def consumer() -> None:
    await ready.wait()
    await consume()

async def initialize() -> None:
    await prepare()
    ready.set()

All current waiters wake after set(), and future waiters pass until clear(). An event carries no payload and does not count notifications.

A Condition combines a lock with notifications for state predicates:

condition = asyncio.Condition()
items: list[str] = []

async def take() -> str:
    async with condition:
        await condition.wait_for(lambda: bool(items))
        return items.pop()

async def put(item: str) -> None:
    async with condition:
        items.append(item)
        condition.notify()

Always wait on a predicate because state may change before a waiter reacquires the lock.

Queues provide ownership transfer

Queues are often clearer than sharing mutable state:

import asyncio

queue: asyncio.Queue[str | None] = asyncio.Queue(maxsize=100)

async def producer() -> None:
    for item in ["a", "b", "c"]:
        await queue.put(item)
    await queue.put(None)

async def consumer() -> None:
    while (item := await queue.get()) is not None:
        try:
            await process(item)
        finally:
            queue.task_done()
    queue.task_done()

maxsize creates backpressure. Every successful get() must have one task_done() if queue.join() is used. With several consumers, send one sentinel per consumer or use a separate shutdown signal. Python 3.13+ also provides queue shutdown APIs, but sentinel patterns remain portable to 3.12.

Threads, processes, or async?

The GIL does not eliminate thread races: operations can span several bytecodes, C extensions may release the GIL, and compound invariants are not atomic. Use threading primitives for threads; never use asyncio.Lock to protect data accessed by threads.

Processes reduce accidental shared-state races but introduce serialization, startup, IPC, and shutdown concerns. Pass coarse work units and prefer immutable messages.

Synchronization is unnecessary when data is immutable, confined to one worker, or transferred by message with clear ownership. This is often the best design.

Common mistakes and practice

• Assuming "single-threaded" means race-free.
• Holding locks across arbitrary callbacks or network I/O.
• Using a semaphore as a mutex or a rate limiter.
• Creating unbounded queues and moving overload into memory.
• Mixing primitives across thread and async domains.
• Forgetting cancellation while a producer or consumer owns an item.

Practice: implement a bounded producer/consumer pipeline with three workers. Make one worker fail, ensure all tasks terminate, account for every dequeued item, and explain where cancellation can interrupt ownership transfer.