Standard library quick reference
A selection map for common APIs, tradeoffs, pitfalls, and further practice.
Use this page as a decision aid. The official Python documentation remains the authoritative API reference for your installed Python version.
Files, processes, and runtime
| Task | API | Notes |
|---|---|---|
| Compose paths | pathlib.Path and / | Do not concatenate separators |
| Read small text | Path.read_text(encoding="utf-8") | Loads the whole file |
| Stream text | Path.open() | Use a context manager |
| Copy/move trees | shutil | Prefer high-level operations |
| Traverse efficiently | os.scandir() | Decide whether to follow symlinks |
| Environment | os.environ, os.getenv() | Values are strings; avoid secret logging |
| External command | subprocess.run([...], check=True) | Avoid shell=True for untrusted input |
| CLI parsing | argparse | Produces help and validation |
| Runtime/streams | sys.argv, sys.stderr, sys.version_info | Keep library code process-neutral |
| Temporary files | tempfile | Context-managed and race-resistant |
Data, text, and numbers
| Task | API | Notes |
|---|---|---|
| JSON | json.load, json.dump | Validate shape after parsing |
| CSV | csv.reader, csv.DictReader | Open files with newline="" |
| Pattern matching | re | Prefer plain string methods for simple cases |
| Immutable records | dataclasses.dataclass(frozen=True) | More evolvable than raw tuples |
| Count values | collections.Counter | Missing entries read as zero |
| FIFO/double-ended queue | collections.deque | Fast operations at both ends |
| Exact decimal rules | decimal.Decimal | Construct from strings |
| Exact ratios | fractions.Fraction | Numerators can grow large |
| Robust float summation | math.fsum | Still produces a float |
| Descriptive measures | statistics | State population/sample and missing-data policy |
| Secure random tokens | secrets | Do not use random for security |
Dates and clocks
| Need | API |
|---|---|
| Current instant | datetime.now(timezone.utc) |
| IANA timezone | zoneinfo.ZoneInfo("Area/City") |
| Machine timestamp | datetime.isoformat() |
| Fixed duration | datetime.timedelta |
| Elapsed deadline | time.monotonic() or event-loop time |
| Benchmark | time.perf_counter() |
Never mix naive and aware timestamps. Store UTC instants, retain the timezone name for future local schedules, and define daylight-saving policies.
Iteration
| Need | API |
|---|---|
| Lazy custom transform | Generator expression/function |
| Slice an iterator | itertools.islice() |
| Batch lazily | itertools.batched() |
| Flatten iterables | itertools.chain() |
| Adjacent grouping | itertools.groupby() |
| Cartesian product | itertools.product() |
| Running totals | itertools.accumulate() |
Estimate combinatoric output sizes and never consume an infinite iterator without a bound.
HTTP and URLs
| Need | API | Boundary |
|---|---|---|
| Query encoding | urllib.parse.urlencode() | Not for path segments |
| Path segment encoding | urllib.parse.quote() | Choose safe deliberately |
| Parse URL | urlsplit() | Parsing is not validation |
| Basic sync request | urllib.request.urlopen() | Set timeout and size limits |
| HTTP errors | urllib.error.HTTPError | Response-like exception |
| Network errors | urllib.error.URLError | Inspect reason |
For high-volume clients, pooled connections, granular timeouts, or native async HTTP, evaluate a maintained third-party client. Dependency-free does not automatically mean simpler operations.
Asyncio
| Need | API | Guidance |
|---|---|---|
| Program boundary | asyncio.run(main()) | Usually once |
| Related child tasks | asyncio.TaskGroup | Preferred structured scope |
| Ordered aggregate await | asyncio.gather() | Understand failure semantics |
| Timeout scope | asyncio.timeout() | Catch TimeoutError outside |
| Blocking I/O bridge | asyncio.to_thread() | Blocking call still needs a timeout |
| Exclusive access | asyncio.Lock | Keep critical section short |
| Capacity limit | asyncio.Semaphore | Not a rate limiter |
| State notification | asyncio.Event, Condition | Conditions wait on predicates |
| Ownership transfer | asyncio.Queue(maxsize=n) | Bounded queues create backpressure |
| TCP stream | open_connection() | Drain writes and close writers |
Failure checklist
Before shipping code that owns resources or concurrent work, ask:
- Is every file, socket, task, executor, and process owned by a visible scope?
- Are all external waits bounded by a timeout or deadline?
- Can input grow memory without a limit?
- What happens if cancellation arrives at each
await? - Can two workers violate a shared invariant?
- Are errors preserved with enough type and context to act on?
- Does shutdown stop intake, finish or abandon owned work, and release resources?
Capstone practice
Build a command-line URL checker:
- read URLs line-by-line from a UTF-8 file;
- validate HTTPS scheme and an explicit host allowlist;
- process checks concurrently under a
TaskGroup; - bridge
urllib.requestwithasyncio.to_thread; - enforce both per-request timeouts and one total deadline;
- limit concurrency with a semaphore and input buffering with a bounded queue;
- aggregate status counts with
Counter; - emit timestamped JSON using aware UTC values;
- write output through an atomic replacement;
- test cancellation, DNS failure, oversized responses, duplicate URLs, and daylight-saving display conversion.
Explain whether a synchronous thread-pool design would be simpler. Async is justified only when its operational benefits outweigh the additional cancellation and lifetime complexity.
LEARNING RECORD
Finish this lesson
Mark it complete when you can explain the main decision without looking.
KNOWLEDGE CHECK