# Code Reference β Python
_102 pages_
---
# Python (/docs/python)
# Python [#python]
Language core, stdlib, tooling, examples.
Browse the sidebar for every cheat sheet in this section.
## Sections [#sections]
* [Examples](./examples)
---
# ABC and Protocols (/docs/python/abc-protocols)
# ABC and Protocols [#abc-and-protocols]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Abstract base classes (`abc`) define explicit interfaces with optional `@abstractmethod`. `typing.Protocol` enables structural (duck) typing for static checkers without inheritance. Use ABCs when you need runtime checks; protocols when you want flexibility.
## π§ Core concepts [#-core-concepts]
| Tool | Role |
| -------------------- | ----------------------------------- |
| `ABC` | Abstract base |
| `@abstractmethod` | Must override |
| `isinstance` + ABC | Runtime virtual subclassing |
| `Protocol` | Structural typing |
| `@runtime_checkable` | `isinstance` on protocols (limited) |
| `collections.abc` | Ready-made `Iterable`, `Mapping`, β¦ |
Register virtual subclasses with `ABC.register` when you cannot change the class hierarchy.
## π‘ Examples [#-examples]
**ABC:**
```python
from abc import ABC, abstractmethod
class Repository(ABC):
@abstractmethod
def get(self, key: str) -> str: ...
def get_or_default(self, key: str, default: str = "") -> str:
try:
return self.get(key)
except KeyError:
return default
class MemoryRepo(Repository):
def __init__(self) -> None:
self._data: dict[str, str] = {}
def get(self, key: str) -> str:
return self._data[key]
```
**Protocol:**
```python
from typing import Protocol, runtime_checkable
@runtime_checkable
class Closable(Protocol):
def close(self) -> None: ...
class Handle:
def close(self) -> None:
print("closed")
def shutdown(x: Closable) -> None:
x.close()
assert isinstance(Handle(), Closable)
```
**collections.abc:**
```python
from collections.abc import Sequence
def third(xs: Sequence[str]) -> str:
return xs[2]
print(third(("a", "b", "c")))
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Instantiating a class with unimplemented abstract methods raises `TypeError`.
* `@runtime_checkable` only checks method **presence**, not signatures.
* Protocols with non-method members have more `isinstance` limitations.
* Prefer `collections.abc` types in annotations over concrete `list` when appropriate.
* Don't force ABC inheritance when a Protocol suffices for typing-only needs.
## π Related [#-related]
* [Typing hints](/docs/python/typing-hints)
* [Classes](/docs/python/classes)
* [Inheritance](/docs/python/inheritance)
* [Collections module](/docs/python/collections-module)
* [Functions](/docs/python/functions)
* [Magic methods](/docs/python/magic-methods)
---
# Argparse (/docs/python/argparse)
# Argparse [#argparse]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`argparse` builds CLI interfaces: options, flags, subcommands, and help text. Prefer it from the stdlib for scripts; consider `typer`/`click` for richer apps.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| ---------------- | ------------------------- |
| `ArgumentParser` | Root parser |
| `add_argument` | Positional or optional |
| `nargs` | `?` `*` `+` or int |
| `type=` | Converter (`int`, `Path`) |
| `choices=` | Allowed values |
| `required=` | For optionals |
| Subparsers | Subcommands |
| `parse_args` | Returns namespace |
Flags start with `-`/`--`. Positionals do not. Auto `--help` is included.
## π‘ Examples [#-examples]
**Simple script:**
```python
import argparse
from pathlib import Path
def main() -> None:
p = argparse.ArgumentParser(description="Resize images")
p.add_argument("input", type=Path, help="input file")
p.add_argument("-o", "--output", type=Path, required=True)
p.add_argument("-v", "--verbose", action="store_true")
p.add_argument("--quality", type=int, default=85, choices=range(1, 101))
args = p.parse_args()
if args.verbose:
print("in", args.input, "out", args.output)
if __name__ == "__main__":
main()
```
**Subcommands:**
```python
import argparse
parser = argparse.ArgumentParser(prog="tool")
sub = parser.add_subparsers(dest="cmd", required=True)
run = sub.add_parser("run", help="run job")
run.add_argument("--workers", type=int, default=1)
show = sub.add_parser("show", help="show status")
show.add_argument("job_id")
args = parser.parse_args()
print(args)
```
**Mutual exclusion:**
```python
import argparse
p = argparse.ArgumentParser()
g = p.add_mutually_exclusive_group()
g.add_argument("--json", action="store_true")
g.add_argument("--plain", action="store_true")
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `type=bool` is wrong for flags β use `action="store_true"`.
* `choices=range(...)` works but help text is huge β validate manually if needed.
* Forgetting `required=True` on subparsers (pre-3.7 patterns) allows empty cmd.
* Don't parse args at import time β gate with `main` / `if __name__`.
* Path args still need existence checks β `type=Path` only converts.
## π Related [#-related]
* [Examples: cli tool](/docs/python/examples/cli-tool)
* [Pathlib](/docs/python/pathlib)
* [Print / input](/docs/python/print-input)
* [Logging](/docs/python/logging)
* [Packaging](/docs/python/packaging)
* [Venv](/docs/python/venv)
---
# Async (/docs/python/async)
# Async [#async]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`async` / `await` run concurrent I/O with an event loop (`asyncio`). Prefer async for many waiting network/file ops; use threads/processes for CPU-bound work.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| --------------------- | --------------------------------- |
| `async def` | Coroutine function |
| `await` | Suspend until awaitable completes |
| `asyncio.run(main())` | Entry point |
| `asyncio.gather` | Run many coroutines |
| `asyncio.create_task` | Schedule concurrently |
| `asyncio.sleep` | Non-blocking delay |
## π‘ Examples [#-examples]
```python
import asyncio
async def fetch(n: int) -> int:
await asyncio.sleep(0.05)
return n * 2
async def main() -> None:
results = await asyncio.gather(fetch(1), fetch(2), fetch(3))
print(results)
asyncio.run(main())
```
```python
import asyncio
async def worker(name: str) -> None:
print(name, "start")
await asyncio.sleep(0.1)
print(name, "done")
async def main() -> None:
t1 = asyncio.create_task(worker("a"))
t2 = asyncio.create_task(worker("b"))
await t1
await t2
asyncio.run(main())
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Never call blocking I/O inside coroutines without an executor.
* Forgetting `await` returns a coroutine object (often a warning).
* `asyncio.run` cannot nest inside a running loop.
* Mixing sync Django views with async requires care (ASGI).
## π Related [#-related]
* [concurrency](/docs/python/concurrency)
* [generators](/docs/python/generators)
* [context\_managers](/docs/python/context-managers)
* [Examples/server](/docs/python/examples/server)
---
# Boolean Logic (/docs/python/boolean-logic)
# Boolean Logic [#boolean-logic]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Boolean logic deals with `True` and `False`. You use it in `if` conditions, loops, and filters. Pythonβs operators are `and`, `or`, and `not` (words, not `&&` / `||`).
## π§ Core concepts [#-core-concepts]
| Piece | Meaning |
| ---------- | -------------------------------------- |
| `bool` | Type with values `True` / `False` |
| Comparison | `==`, `!=`, `<`, `>`, `<=`, `>=` |
| `and` | True only if both sides are true |
| `or` | True if at least one side is true |
| `not` | Flips truthiness |
| Chaining | `10 < x < 20` works in Python |
Comparisons return booleans. Many other values are *truthy* or *falsy* when used in conditions (see related note).
## π‘ Examples [#-examples]
**Comparisons:**
```python
age = 18
print(age >= 18) # True
print(age == 21) # False
print(age != 0) # True
```
**and / or / not:**
```python
has_ticket = True
is_weekend = False
print(has_ticket and is_weekend) # False
print(has_ticket or is_weekend) # True
print(not is_weekend) # True
```
**if with boolean expressions:**
```python
temp_c = 22
raining = False
if temp_c > 20 and not raining:
print("Nice walk weather")
elif raining:
print("Take an umbrella")
else:
print("Maybe stay in")
```
**Short-circuit evaluation:**
```python
x = 0
# Right side is skipped because left is False
print(x != 0 and (10 / x > 1)) # False, no ZeroDivisionError
name = ""
label = name or "guest"
print(label) # guest
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `and` / `or` return one of the *operands*, not always a strict `bool`.
* `==` vs `is`: use `==` for values; `is` for identity (`None` is the common exception).
* `1 < x > 0` is valid chaining β read carefully.
* Do not write `if flag == True:` β prefer `if flag:`.
## π Related [#-related]
* [none\_and\_truthy.md](/docs/python/none-and-truthy)
* [variables.md](/docs/python/variables)
* [indentation.md](/docs/python/indentation)
* [truthiness.md](/docs/python/truthiness)
---
# Bytes and Bytearray (/docs/python/bytes-bytearray)
# Bytes and Bytearray [#bytes-and-bytearray]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`bytes` is an immutable sequence of ints 0β255; `bytearray` is mutable. Use for binary protocols, files, and encodings. Text is `str` β convert explicitly with `.encode()` / `.decode()`.
## π§ Core concepts [#-core-concepts]
| Type | Mutability | Create |
| ------------ | ---------- | ----------------------------------- |
| `bytes` | Immutable | `b"hi"`, `bytes([65])`, `bytes(10)` |
| `bytearray` | Mutable | `bytearray(b"hi")` |
| `memoryview` | View | Zero-copy slices |
Encoding: `s.encode("utf-8")` β `bytes`; `b.decode("utf-8")` β `str`. Hex/base helpers: `.hex()`, `bytes.fromhex`.
## π‘ Examples [#-examples]
**Encode / decode:**
```python
text = "cafΓ©"
data = text.encode("utf-8")
print(data) # b'caf\xc3\xa9'
print(data.decode("utf-8"))
```
**bytearray mutation:**
```python
buf = bytearray(b"python")
buf[0] = ord("P")
buf.extend(b"!")
print(buf) # bytearray(b'Python!')
```
**Binary file + memoryview:**
```python
from pathlib import Path
path = Path("blob.bin")
path.write_bytes(b"\x00\x01\x02\x03")
raw = path.read_bytes()
view = memoryview(raw)
print(view[1:3].tobytes()) # b'\x01\x02'
```
**Struct packing:**
```python
import struct
packet = struct.pack(">I4s", 42, b"ping")
code, payload = struct.unpack(">I4s", packet)
print(code, payload)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Indexing returns `int`, not a 1-length bytes: `b"ab"[0] == 97`.
* Mixing `str` and `bytes` raises `TypeError`.
* Wrong encoding (e.g. `latin-1` vs `utf-8`) corrupts text β be explicit.
* `bytes(5)` is five zero bytes, not `b"5"`.
* Large concatenations in a loop β prefer `bytearray` or `b"".join`.
## π Related [#-related]
* [Strings](/docs/python/strings)
* [Files I/O](/docs/python/files-io)
* [Slicing](/docs/python/slicing)
* [Types](/docs/python/types)
* [Examples: encrypt](/docs/python/examples/encrypt)
* [Pathlib](/docs/python/pathlib)
---
# Classes (/docs/python/classes)
# Classes [#classes]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Classes define object types: attributes + methods. Prefer clear `__init__`, type hints, and small cohesive classes. For data-heavy records, consider [dataclasses](/docs/python/dataclasses) first.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| --------------- | -------------------------------------- |
| `class Name:` | Creates a type object |
| `__init__` | Instance initializer |
| `self` | Instance reference (conventional name) |
| Instance attr | `self.x = ...` |
| Class attr | Shared on the class body |
| `@staticmethod` | No `self` / `cls` |
| `@classmethod` | Receives `cls` |
| `@property` | Managed attribute |
Methods are functions on the class; binding supplies `self` on instance access.
## π‘ Examples [#-examples]
**Basic class:**
```python
class User:
species = "human" # class attribute
def __init__(self, name: str, age: int) -> None:
self.name = name
self.age = age
def greet(self) -> str:
return f"Hi, I'm {self.name}"
@classmethod
def from_dict(cls, data: dict[str, object]) -> "User":
return cls(str(data["name"]), int(data["age"])) # type: ignore[arg-type]
@staticmethod
def is_adult(age: int) -> bool:
return age >= 18
u = User.from_dict({"name": "Ada", "age": 36})
print(u.greet(), User.is_adult(u.age))
```
**Property:**
```python
class Temperature:
def __init__(self, celsius: float) -> None:
self._celsius = celsius
@property
def celsius(self) -> float:
return self._celsius
@celsius.setter
def celsius(self, value: float) -> None:
if value < -273.15:
raise ValueError("below absolute zero")
self._celsius = value
```
**Slots (memory / attr lock):**
```python
class Point:
__slots__ = ("x", "y")
def __init__(self, x: float, y: float) -> None:
self.x = x
self.y = y
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Mutable class attributes are shared across instances β use instance attrs in `__init__`.
* Forgetting `self` in method signatures causes confusing `TypeError`s.
* `__init__` should not return a value (except `None`).
* String annotations / `from __future__ import annotations` affect runtime `typing.get_type_hints`.
* Prefer composition over deep inheritance trees.
## π Related [#-related]
* [Inheritance](/docs/python/inheritance)
* [Dataclasses](/docs/python/dataclasses)
* [Magic methods](/docs/python/magic-methods)
* [Properties / descriptors](/docs/python/properties-descriptors)
* [Enums](/docs/python/enums)
* [ABC / protocols](/docs/python/abc-protocols)
---
# Closures (/docs/python/closures)
# Closures [#closures]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
A closure is a function that remembers variables from an enclosing scope after that scope has finished. Closures power decorators, callbacks, and lightweight factories without full classes.
## π§ Core concepts [#-core-concepts]
| Idea | Detail |
| ------------- | --------------------------------------------------- |
| Free variable | Name used but not defined in the inner function |
| Cell | Storage for closed-over values |
| `nonlocal` | Rebind an enclosing (non-global) name |
| `global` | Rebind a module-level name |
| Late binding | Looks up the name at call time, not definition time |
Inner functions are closures when they reference enclosing locals. Inspect with `fn.__closure__`.
## π‘ Examples [#-examples]
**Factory:**
```python
def make_multiplier(factor: int):
def multiply(x: int) -> int:
return x * factor
return multiply
double = make_multiplier(2)
print(double(5)) # 10
```
**Stateful counter with nonlocal:**
```python
def counter(start: int = 0):
value = start
def inc(step: int = 1) -> int:
nonlocal value
value += step
return value
return inc
c = counter()
print(c(), c(), c(5)) # 1 2 7
```
**Late binding in loops:**
```python
funcs = []
for i in range(3):
funcs.append(lambda: i) # all see final i
print([f() for f in funcs]) # [2, 2, 2]
funcs = [lambda i=i: i for i in range(3)]
print([f() for f in funcs]) # [0, 1, 2]
```
**Decorator as closure:**
```python
def ensure_positive(fn):
def wrapper(x: int) -> int:
if x < 0:
raise ValueError("x must be >= 0")
return fn(x)
return wrapper
@ensure_positive
def sqrt_int(x: int) -> int:
return int(x**0.5)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Late binding: closures capture the variable, not the value at creation.
* Assignment in an inner function makes a name local unless `nonlocal`/`global`.
* Closures keep objects alive β can delay GC of large data.
* Prefer explicit objects when state has many fields or methods.
* Mutable closed-over containers can be mutated without `nonlocal`.
## π Related [#-related]
* [Functions](/docs/python/functions)
* [Decorators](/docs/python/decorators)
* [Scope / namespaces](/docs/python/scope-namespaces)
* [Lambda](/docs/python/lambda)
* [Classes](/docs/python/classes)
* [Walrus](/docs/python/walrus)
---
# Collections Module (/docs/python/collections-module)
# Collections Module [#collections-module]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`collections` provides specialized container types beyond built-in `list`/`dict`/`set`. Reach for them when counting, defaulting, dequeuing, or naming tuples.
## π§ Core concepts [#-core-concepts]
| Type | Use |
| ----------------------- | ---------------------------------- |
| `Counter` | Multiset / tallies |
| `defaultdict` | Dict with factory for missing keys |
| `deque` | Fast append/pop both ends |
| `namedtuple` | Lightweight immutable records |
| `OrderedDict` | Rarely needed (dict keeps order) |
| `ChainMap` | Layered mappings |
| `UserDict` / `UserList` | Wrap for subclassing |
Also see `collections.abc` for abstract interfaces (`Iterable`, `Mapping`, β¦).
## π‘ Examples [#-examples]
**Counter:**
```python
from collections import Counter
counts = Counter("abracadabra")
print(counts.most_common(3))
print(counts["a"])
counts.update("aaa")
```
**defaultdict:**
```python
from collections import defaultdict
groups: defaultdict[str, list[int]] = defaultdict(list)
for n in [1, 2, 3, 4]:
groups["even" if n % 2 == 0 else "odd"].append(n)
```
**deque as queue:**
```python
from collections import deque
q: deque[str] = deque(maxlen=3)
q.append("a")
q.append("b")
q.appendleft("z")
print(q.popleft())
```
**namedtuple / ChainMap:**
```python
from collections import ChainMap, namedtuple
Point = namedtuple("Point", ["x", "y"])
p = Point(1, 2)
defaults = {"color": "blue", "size": 10}
overrides = {"size": 12}
cfg = ChainMap(overrides, defaults)
print(cfg["color"], cfg["size"])
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `defaultdict` creates entries on `__getitem__` β use `.get` if you must not insert.
* `Counter` missing keys return `0`, not `KeyError`.
* `namedtuple` is immutable; for mutability use dataclass.
* `OrderedDict` equality historically cared about order β prefer plain `dict`.
* `deque` is not sliceable like a list.
## π Related [#-related]
* [Dictionaries](/docs/python/dictionaries)
* [Lists](/docs/python/lists)
* [Dataclasses](/docs/python/dataclasses)
* [Itertools](/docs/python/itertools)
* [ABC / protocols](/docs/python/abc-protocols)
* [Examples: list to dict](/docs/python/examples/list-to-dict)
---
# Comments (/docs/python/comments)
# Comments [#comments]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Comments are notes for humans. The interpreter ignores them. Use them to explain *why*, not to narrate obvious code. Python also has docstrings for documenting modules, classes, and functions.
## π§ Core concepts [#-core-concepts]
| Form | Syntax | Use |
| -------------- | -------------------------- | ------------------------------------ |
| Line comment | `# ...` | Short notes on a line |
| Inline comment | `code # note` | Clarify one statement |
| Docstring | `"""..."""` or `'''...'''` | Document APIs (not ignored by tools) |
| Shebang | `#!/usr/bin/env python3` | Unix script runner hint (first line) |
There is no dedicated multi-line comment syntax. Use multiple `#` lines, or a docstring at the top of a block when documenting.
## π‘ Examples [#-examples]
**Line and inline comments:**
```python
# Calculate weekly pay from hourly rate
rate = 20
hours = 37.5
pay = rate * hours # overtime not included yet
print(pay)
```
**Docstring on a function:**
```python
def greet(name: str) -> str:
"""Return a friendly greeting for name."""
return f"Hello, {name}!"
print(greet("Ada"))
print(greet.__doc__)
```
**Module header:**
```python
#!/usr/bin/env python3
"""tiny_demo.py β practice script for comments and print."""
# TODO: add command-line args later
print("ready")
```
**Temporarily disable code (prefer Git, not huge comment blocks):**
```python
# old_way = compute_v1(data)
result = compute_v2(data)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Commented-out code rots quickly β delete it and use version control.
* Docstrings are *strings*, not comments; they become `__doc__`.
* Nesting quotes wrong inside docstrings can break the string.
* Over-commenting (`i = i + 1 # add one`) adds noise.
## π Related [#-related]
* [getting\_started.md](/docs/python/getting-started)
* [hello\_world.md](/docs/python/hello-world)
* [variables.md](/docs/python/variables)
* [indentation.md](/docs/python/indentation)
---
# Comprehensions (/docs/python/comprehensions)
# Comprehensions [#comprehensions]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Comprehensions build lists, sets, dicts, and generator expressions concisely. Prefer them over manual loops for simple map/filter transforms. Keep them readable β nest sparingly.
## π§ Core concepts [#-core-concepts]
| Form | Result |
| ---------------------------- | --------------------------- |
| `[expr for x in it if cond]` | `list` |
| `\{expr for x in it\}` | `set` |
| `\{k: v for ...\}` | `dict` |
| `(expr for x in it)` | Generator (lazy) |
| Nested | `for` clauses left-to-right |
| Walrus | `:=` inside filters (3.8+) |
Scope: iteration variables leak in list/set/dict comps in 3.x? Actually in Python 3, comprehension iteration variables are isolated in their own scope (unlike 2.x).
## π‘ Examples [#-examples]
**List / set / dict:**
```python
nums = range(8)
squares = [n * n for n in nums if n % 2 == 0]
unique_lens = {len(w) for w in ["a", "bb", "ccc", "bb"]}
index = {c: i for i, c in enumerate("abc")}
```
**Nested:**
```python
matrix = [[1, 2], [3, 4]]
flat = [x for row in matrix for x in row]
pairs = [(i, j) for i in range(3) for j in range(3) if i != j]
```
**Generator expression:**
```python
total = sum(n * n for n in range(1_000_000)) # no giant list
lines = (line.strip() for line in open("f.txt", encoding="utf-8"))
# prefer pathlib + context manager in real code
```
**Walrus in comprehension:**
```python
def normalize(s: str) -> str:
return s.strip().lower()
names = [" Ada ", "Bob", " "]
clean = [n for name in names if (n := normalize(name))]
# ["ada", "bob"]
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Over-nested comps are worse than a plain loop.
* Dict comps: later keys overwrite earlier ones silently.
* Side effects inside comps hurt readability β avoid prints/mutations.
* Generator expressions are single-pass β materialize with `list()` if reused.
* Huge list comps can exhaust memory; use generators.
## π Related [#-related]
* [Lists](/docs/python/lists)
* [Dictionaries](/docs/python/dictionaries)
* [Sets](/docs/python/sets)
* [Generators](/docs/python/generators)
* [Loops](/docs/python/loops)
* [Walrus](/docs/python/walrus)
---
# Concurrency (/docs/python/concurrency)
# Concurrency [#concurrency]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Python offers threads, processes, and asyncio for concurrent work. Choose based on the bottleneck: **I/O-bound** β threads or asyncio; **CPU-bound** β multiprocessing (GIL limits CPU threads). See [async.md](/docs/python/async) for `async`/`await` detail.
## π§ Core concepts [#-core-concepts]
| Model | Best for | Module |
| --------------- | --------------------------------------- | ---------------------------------------- |
| Threading | I/O, shared memory | `threading`, `concurrent.futures` |
| Multiprocessing | CPU parallelism | `multiprocessing`, `ProcessPoolExecutor` |
| Asyncio | Many sockets / timers | `asyncio` |
| GIL | One bytecode thread at a time (CPython) | β |
`concurrent.futures` unifies thread/process pools behind `Executor.map` / `submit`.
## π‘ Examples [#-examples]
**ThreadPoolExecutor (I/O):**
```python
from concurrent.futures import ThreadPoolExecutor, as_completed
from urllib.request import urlopen
urls = ["https://example.com", "https://httpbin.org/get"]
def fetch(url: str) -> int:
with urlopen(url, timeout=10) as resp:
return resp.status
with ThreadPoolExecutor(max_workers=8) as pool:
futures = {pool.submit(fetch, u): u for u in urls}
for fut in as_completed(futures):
print(futures[fut], fut.result())
```
**ProcessPoolExecutor (CPU):**
```python
from concurrent.futures import ProcessPoolExecutor
def crunch(n: int) -> int:
return sum(i * i for i in range(n))
if __name__ == "__main__":
with ProcessPoolExecutor() as pool:
print(list(pool.map(crunch, [10**6, 10**6])))
```
**asyncio overview:**
```python
import asyncio
async def sleeper(name: str, delay: float) -> str:
await asyncio.sleep(delay)
return name
async def main() -> None:
results = await asyncio.gather(sleeper("a", 0.1), sleeper("b", 0.2))
print(results)
asyncio.run(main())
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* CPU-bound work in threads won't scale on CPython due to the GIL.
* Share state only with locks/queues β races are subtle.
* On Windows, process entry must be under `if __name__ == "__main__"`.
* Mixing blocking I/O inside asyncio stalls the event loop β use `to_thread`.
* Deadlocks: never lock in inconsistent orders across threads.
## π Related [#-related]
* [Async / await](/docs/python/async)
* [Context managers](/docs/python/context-managers)
* [Logging](/docs/python/logging)
* [Examples: server](/docs/python/examples/server)
* [Examples: retry backoff](/docs/python/examples/retry-backoff)
* [Testing with pytest](/docs/python/testing-pytest)
---
# Conditionals (/docs/python/conditionals)
# Conditionals [#conditionals]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Branch with `if` / `elif` / `else` and structural `match` / `case` (3.10+). Prefer clear boolean expressions and pattern matching for nested dict/tag shapes over deep `if` ladders.
## π§ Core concepts [#-core-concepts]
| Form | Use |
| ---------------------- | --------------------------- |
| `if` / `elif` / `else` | Boolean branches |
| Ternary | `a if cond else b` |
| `match` / `case` | Structural pattern matching |
| Guards | `case x if pred:` |
| OR patterns | `case 1 \| 2:` |
| Capture | `case \{"id": int(uid)\}:` |
| Wildcard | `case _:` |
Truthiness rules apply in `if` β see [truthiness](/docs/python/truthiness). `match` compares structure and can bind names.
## π‘ Examples [#-examples]
**Classic if:**
```python
def grade(score: int) -> str:
if score >= 90:
return "A"
if score >= 80:
return "B"
if score >= 70:
return "C"
return "F"
label = "pass" if grade(85) != "F" else "fail"
```
**Match literals and guards:**
```python
def handle(code: int) -> str:
match code:
case 200 | 201:
return "ok"
case 404:
return "missing"
case n if 500 <= n < 600:
return "server"
case _:
return "other"
```
**Match mapping / sequence:**
```python
def route(event: dict[str, object]) -> str:
match event:
case {"type": "user", "id": int(uid)} if uid > 0:
return f"user:{uid}"
case {"type": "ping"}:
return "pong"
case {"items": [first, *rest]}:
return f"first={first}, n={len(rest)+1}"
case _:
return "unknown"
```
**Match class patterns:**
```python
from dataclasses import dataclass
@dataclass
class Point:
x: int
y: int
def quadrant(p: Point) -> str:
match p:
case Point(x=0, y=0):
return "origin"
case Point(x=x, y=y) if x > 0 and y > 0:
return "I"
case Point():
return "elsewhere"
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `case x:` captures everything β put specific patterns first.
* `match` subject is evaluated once; patterns do not fall through like C `switch`.
* Avoid chained ternaries β hard to read.
* `==` vs `is`: use `is` for `None` / singletons.
* Empty patterns and overlapping captures can surprise β test edge cases.
## π Related [#-related]
* [Truthiness](/docs/python/truthiness)
* [Operators](/docs/python/operators)
* [Enums](/docs/python/enums)
* [Dataclasses](/docs/python/dataclasses)
* [Dictionaries](/docs/python/dictionaries)
* [Walrus](/docs/python/walrus)
---
# Context Managers (/docs/python/context-managers)
# Context Managers [#context-managers]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Context managers guarantee setup/teardown via `with`. They close files, release locks, and restore state even when exceptions occur. Implement with `__enter__`/`__exit__` or `@contextlib.contextmanager`.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ------------------------------- | -------------------------------------- |
| `with cm as x:` | Enter, bind result, exit on leave |
| Multiple | `with a, b:` |
| Parenthesized | Multi-line `with` (3.10+) |
| `__enter__` | Setup; return bound value |
| `__exit__` | Cleanup; return truthy to suppress exc |
| `contextlib.contextmanager` | Generator-based CM |
| `ExitStack` | Dynamic stack of contexts |
| `AsyncExitStack` / `async with` | Async variants |
`__exit__(exc_type, exc, tb)` β return `True` to suppress the exception (rare).
## π‘ Examples [#-examples]
**Built-in file and lock:**
```python
from pathlib import Path
from threading import Lock
lock = Lock()
path = Path("out.txt")
with lock, path.open("w", encoding="utf-8") as f:
f.write("hello\n")
```
**Generator context manager:**
```python
from contextlib import contextmanager
from collections.abc import Iterator
@contextmanager
def temp_attr(obj: object, name: str, value: object) -> Iterator[None]:
old = getattr(obj, name)
setattr(obj, name, value)
try:
yield
finally:
setattr(obj, name, old)
class C:
mode = "prod"
with temp_attr(C, "mode", "test"):
assert C.mode == "test"
```
**Class-based:**
```python
class suppress_os:
def __enter__(self) -> "suppress_os":
return self
def __exit__(self, exc_type, exc, tb) -> bool:
return exc_type is not None and issubclass(exc_type, OSError)
```
**ExitStack:**
```python
from contextlib import ExitStack
from pathlib import Path
paths = [Path("a.txt"), Path("b.txt")]
with ExitStack() as stack:
files = [stack.enter_context(p.open("w", encoding="utf-8")) for p in paths]
for f in files:
f.write("x\n")
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Returning `True` from `__exit__` swallows exceptions β usually wrong.
* Yielding inside `@contextmanager` after errors needs careful `try/finally`.
* Nested `with` order: enter left-to-right, exit right-to-left.
* Don't open resources without `with` unless you manage close yourself.
* Async code needs `async with` and async context managers.
## π Related [#-related]
* [Exceptions](/docs/python/exceptions)
* [Files I/O](/docs/python/files-io)
* [Pathlib](/docs/python/pathlib)
* [Magic methods](/docs/python/magic-methods)
* [Concurrency](/docs/python/concurrency)
* [Async / await](/docs/python/async)
---
# Copy and Deepcopy (/docs/python/copy-deepcopy)
# Copy and Deepcopy [#copy-and-deepcopy]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Assignment binds names to objects; it does not copy. Use slicing/`list()`/`copy.copy` for shallow copies and `copy.deepcopy` for nested structures. Know when shared references are intentional.
## π§ Core concepts [#-core-concepts]
| Operation | Effect |
| --------------------------- | ---------------------------------------------------- |
| `b = a` | Same object |
| `a.copy()` / `copy.copy(a)` | Shallow copy |
| `a[:]` (list) | Shallow copy |
| `copy.deepcopy(a)` | Recursive copy |
| Immutable | Often no need to copy (`str`, `tuple` of immutables) |
Shallow: new container, same nested objects. Deep: nested containers cloned (with cycle handling).
## π‘ Examples [#-examples]
**Alias vs shallow:**
```python
import copy
a = [[1], [2]]
alias = a
shallow = copy.copy(a) # or a[:] / list(a)
alias[0].append(9)
print(a, shallow) # both see [1, 9] in first row
```
**Deepcopy:**
```python
import copy
a = [[1], [2]]
deep = copy.deepcopy(a)
a[0].append(9)
print(a) # [[1, 9], [2]]
print(deep) # [[1], [2]]
```
**Dict / custom:**
```python
import copy
cfg = {"db": {"host": "localhost"}, "debug": True}
cfg2 = {**cfg} # shallow
cfg3 = copy.deepcopy(cfg)
class Node:
def __init__(self, value: int) -> None:
self.value = value
self.child: Node | None = None
n = Node(1)
n.child = Node(2)
m = copy.deepcopy(n)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `list * n` / `[[0]*3]*3` creates shared inner rows β classic bug.
* Deepcopy can be slow and fail on locks, sockets, modules, or some C objects.
* Dataclass `replace` is shallow for nested mutables.
* Copying is not serialization β for disk/network use JSON/pickle carefully.
* `__copy__` / `__deepcopy__` can customize behavior on your classes.
## π Related [#-related]
* [Lists](/docs/python/lists)
* [Dictionaries](/docs/python/dictionaries)
* [Dataclasses](/docs/python/dataclasses)
* [Slicing](/docs/python/slicing)
* [Types](/docs/python/types)
* [Scope / namespaces](/docs/python/scope-namespaces)
---
# Csv (/docs/python/csv)
# Csv [#csv]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
The stdlib `csv` module reads and writes delimited text. Use `DictReader`/`DictWriter` when rows have named columns; plain `reader`/`writer` for positional fields. Prefer `newline=""` when opening files on Windows.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ------------------------------- | ------------------------------- |
| `csv.reader(f)` | Iterate rows as lists |
| `csv.DictReader(f)` | Rows as `dict` (header β value) |
| `csv.writer(f)` | Write list rows |
| `csv.DictWriter(f, fieldnames)` | Write dict rows |
| `writerow` / `writerows` | One / many rows |
| `writeheader()` | Emit header for DictWriter |
| `dialect` / `delimiter` | CSV flavor (excel, `;`, etc.) |
| `csv.QUOTE_MINIMAL` | Quoting strategy |
For large analytics, use [Pandas](/docs/python/pandas) `read_csv` instead.
## π‘ Examples [#-examples]
**DictReader:**
```python
import csv
from pathlib import Path
with Path("people.csv").open(newline="", encoding="utf-8") as f:
for row in csv.DictReader(f):
print(row["name"], row["age"])
```
**reader (lists):**
```python
import csv
with open("data.csv", newline="", encoding="utf-8") as f:
rows = list(csv.reader(f))
header, *body = rows
print(header, body[:2])
```
**DictWriter:**
```python
import csv
from pathlib import Path
fields = ["name", "age"]
rows = [{"name": "Ada", "age": 36}, {"name": "Lin", "age": 41}]
with Path("out.csv").open("w", newline="", encoding="utf-8") as f:
w = csv.DictWriter(f, fieldnames=fields)
w.writeheader()
w.writerows(rows)
```
**Custom delimiter:**
```python
import csv
with open("eu.csv", newline="", encoding="utf-8") as f:
for row in csv.reader(f, delimiter=";"):
print(row)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Always open with `newline=""` so the csv module owns line endings.
* Pass `encoding="utf-8"` (or the real encoding) β never rely on locale default.
* `DictWriter` needs `fieldnames`; missing keys become empty cells unless `extrasaction="raise"`.
* Excel may expect BOM / `;` β test with the consumer.
* Nested commas need quoting; let `csv` handle it β donβt hand-split lines.
## π Related [#-related]
* [Pandas](/docs/python/pandas)
* [Files I/O](/docs/python/files-io)
* [Pathlib](/docs/python/pathlib)
* [Json](/docs/python/json)
---
# Dataclasses (/docs/python/dataclasses)
# Dataclasses [#dataclasses]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`dataclasses.dataclass` auto-generates `__init__`, `__repr__`, `__eq__`, and optionally ordering/hash/slots for classes that mainly store data. Prefer dataclasses over hand-written boilerplate for records and DTOs.
## π§ Core concepts [#-core-concepts]
| Option | Effect |
| -------------------- | ----------------------------------------- |
| `@dataclass` | Generate methods from annotated fields |
| `frozen=True` | Immutable instances (approx.) |
| `slots=True` | `__slots__` (3.10+) β less memory |
| `order=True` | `<`, `<=`, β¦ from field order |
| `kw_only=True` | Keyword-only fields (3.10+) |
| `field(...)` | Defaults, factories, exclude from compare |
| `asdict` / `astuple` | Convert to built-ins |
Fields need type annotations. Default values must follow non-default fields (or use `kw_only`).
## π‘ Examples [#-examples]
**Basic and factory defaults:**
```python
from dataclasses import dataclass, field
@dataclass
class Config:
host: str
port: int = 8000
tags: list[str] = field(default_factory=list)
cfg = Config("localhost", tags=["dev"])
```
**Frozen + slots:**
```python
from dataclasses import dataclass
@dataclass(frozen=True, slots=True)
class Point:
x: float
y: float
p = Point(1.0, 2.0)
# p.x = 3 # FrozenInstanceError
```
**Post-init validation:**
```python
from dataclasses import dataclass
@dataclass
class User:
name: str
age: int
def __post_init__(self) -> None:
if self.age < 0:
raise ValueError("age must be >= 0")
```
**Replace and asdict:**
```python
from dataclasses import asdict, replace
u = User("Ada", 36)
u2 = replace(u, age=37)
print(asdict(u2)) # {"name": "Ada", "age": 37}
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Mutable defaults need `field(default_factory=...)`, not `= []`.
* `frozen=True` is shallow β nested mutables can still change.
* `asdict` deep-copies nested dataclasses into dicts β can be expensive.
* Inheritance: dataclass parents/children need care with field ordering.
* For complex validation / ORM models, consider Pydantic or attrs.
## π Related [#-related]
* [Classes](/docs/python/classes)
* [Enums](/docs/python/enums)
* [Typing hints](/docs/python/typing-hints)
* [Copy / deepcopy](/docs/python/copy-deepcopy)
* [Examples: dataclass pipeline](/docs/python/examples/dataclass-pipeline)
* [Tuples](/docs/python/tuples)
---
# Datetime (/docs/python/datetime)
# Datetime [#datetime]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`datetime` handles dates, times, timedeltas, and timezones. Prefer **aware** datetimes (`tzinfo` set) over naive ones. Use `zoneinfo` (3.9+) for IANA timezones.
## π§ Core concepts [#-core-concepts]
| Type | Role |
| ----------------------------- | ------------- |
| `date` | Calendar date |
| `time` | Time of day |
| `datetime` | Date + time |
| `timedelta` | Duration |
| `timezone.utc` | Fixed UTC |
| `zoneinfo.ZoneInfo` | IANA TZ |
| `isoformat` / `fromisoformat` | ISO-8601 |
Arithmetic: `datetime Β± timedelta`. Comparing naive vs aware raises `TypeError`.
## π‘ Examples [#-examples]
**UTC now and ISO:**
```python
from datetime import datetime, timedelta, timezone
now = datetime.now(timezone.utc)
later = now + timedelta(hours=2)
print(now.isoformat())
print(datetime.fromisoformat("2024-07-10T12:00:00+00:00"))
```
**ZoneInfo:**
```python
from datetime import datetime
from zoneinfo import ZoneInfo
ny = ZoneInfo("America/New_York")
local = datetime(2024, 7, 10, 9, 0, tzinfo=ny)
utc = local.astimezone(ZoneInfo("UTC"))
print(local, utc)
```
**Parsing common formats:**
```python
from datetime import datetime
dt = datetime.strptime("10/07/2024 14:30", "%d/%m/%Y %H:%M")
print(dt.strftime("%Y-%m-%d"))
```
**Date math:**
```python
from datetime import date, timedelta
start = date(2024, 7, 10)
print(start + timedelta(days=7))
print((date.today() - start).days)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Naive `datetime.now()` is ambiguous β prefer `datetime.now(timezone.utc)`.
* `utcnow()` is deprecated in 3.12 β use `now(timezone.utc)`.
* DST transitions: local arithmetic can skip/repeat hours β convert via UTC.
* `timestamp()` on naive assumes local time β be explicit.
* Don't store local times without offset in databases when possible.
## π Related [#-related]
* [JSON](/docs/python/json)
* [Logging](/docs/python/logging)
* [Types](/docs/python/types)
* [F-strings](/docs/python/f-string)
* [Examples: retry backoff](/docs/python/examples/retry-backoff)
* [Argparse](/docs/python/argparse)
---
# Decorators (/docs/python/decorators)
# Decorators [#decorators]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
A decorator is a callable that wraps another callable to add behavior (logging, timing, caching, auth). Syntax `@decorator` above `def`/`class` is sugar for `f = decorator(f)`. Preserve metadata with `functools.wraps`.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| --------------------- | --------------------------------------- |
| `@dec` | `f = dec(f)` |
| `@dec(args)` | `f = dec(args)(f)` β factory |
| `functools.wraps` | Copy `__name__`, `__doc__`, annotations |
| `functools.lru_cache` | Memoization |
| `functools.cache` | Unbounded cache (3.9+) |
| Class decorators | Transform / register classes |
| Stacked | Applied bottom-up |
Decorators work on methods too; be careful with `self` and descriptors.
## π‘ Examples [#-examples]
**Simple wrapper:**
```python
import functools
import time
from collections.abc import Callable
from typing import TypeVar
F = TypeVar("F", bound=Callable[..., object])
def timed(fn: F) -> F:
@functools.wraps(fn)
def wrapper(*args: object, **kwargs: object) -> object:
start = time.perf_counter()
try:
return fn(*args, **kwargs)
finally:
print(f"{fn.__name__}: {time.perf_counter() - start:.4f}s")
return wrapper # type: ignore[return-value]
@timed
def work(n: int) -> int:
return sum(range(n))
```
**Decorator with arguments:**
```python
def repeat(times: int):
def deco(fn: Callable[..., object]) -> Callable[..., object]:
@functools.wraps(fn)
def wrapper(*args: object, **kwargs: object) -> object:
result = None
for _ in range(times):
result = fn(*args, **kwargs)
return result
return wrapper
return deco
@repeat(3)
def ping() -> str:
print("ping")
return "ok"
```
**Built-ins:**
```python
from functools import cache, lru_cache
@cache
def fib(n: int) -> int:
return n if n < 2 else fib(n - 1) + fib(n - 2)
@lru_cache(maxsize=128)
def load(key: str) -> str:
return key.upper()
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Forgetting `@wraps` breaks introspection, tests, and OpenAPI tools.
* Order of stacked decorators matters (`@a` then `@b` β `a(b(f))`).
* Caching mutable/unhashable args fails with `lru_cache`.
* Decorating instance methods: wrapper still receives `self` as first arg.
* Class decorators replace the class object β return the class (or a new one).
## π Related [#-related]
* [Functions](/docs/python/functions)
* [Closures](/docs/python/closures)
* [Classes](/docs/python/classes)
* [Properties / descriptors](/docs/python/properties-descriptors)
* [Typing hints](/docs/python/typing-hints)
* [Logging](/docs/python/logging)
---
# Dictionaries (/docs/python/dictionaries)
# Dictionaries [#dictionaries]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
A `dict` maps hashable keys to values. Insertion order is preserved (3.7+). Use for lookups, configs, JSON-like data, and counting. Prefer `dict` / `collections.Counter` / `defaultdict` over ad-hoc nested structures when possible.
## π§ Core concepts [#-core-concepts]
**Create / access / mutate**
| Operation | Example |
| ---------- | ---------------------------------------------------------------------------- |
| Create | `\{"a": 1\}`, `dict(a=1)`, `dict([("a", 1)])`, `dict.fromkeys(keys, v=None)` |
| Get | `d["a"]` (KeyError), `d.get("a", default)` |
| Set | `d["a"] = 1`, `d.setdefault("a", 0)` |
| Delete | `del d["a"]`, `d.pop("a", default)`, `d.popitem()` (LIFO 3.7+) |
| Clear | `d.clear()` |
| Membership | `k in d`, `k not in d` (keys only) |
**Instance methods**
| Method | Notes |
| -------------------------------- | ------------------------------------------------ |
| `get(k[, default])` | Return value or default (`None`) |
| `setdefault(k[, default])` | Get or insert default; return value |
| `update([other], **kw)` | Merge mappings/pairs in place |
| `pop(k[, default])` | Remove key; return value (or default / KeyError) |
| `popitem()` | Remove and return `(k, v)` last inserted |
| `clear()` | Remove all items |
| `copy()` | Shallow copy |
| `fromkeys(iterable, value=None)` | Classmethod: new dict with shared default |
**Merge operators (3.9+)**
| Op | Notes |
| -------------- | ------------------------------- |
| `a \| b` | New dict; `b` wins on conflicts |
| `a \|= b` | In-place update like `update` |
| `\{**a, **b\}` | Unpack merge (literal) |
**Views β `keys` / `values` / `items`**
| View | Notes |
| --------------------- | -------------------------------------------------------- |
| `d.keys()` | Dynamic set-like view of keys |
| `d.values()` | Dynamic view of values |
| `d.items()` | Dynamic set-like view of `(k, v)` |
| Set ops on keys/items | `&`, `\|`, `-`, `^` with other views/sets |
| Live views | Reflect dict changes; do not mutate dict while iterating |
| Materialize | `list(d)`, `list(d.items())` for a stable snapshot |
**Comprehension / related helpers**
| Pattern | Notes |
| ---------------------------- | ------------------------------------------------ |
| `\{k: v for k, v in pairs\}` | Dict comprehension |
| `collections.Counter` | Multiset / counts |
| `collections.defaultdict` | Auto-create missing values |
| `collections.OrderedDict` | Reorder helpers; plain `dict` keeps insert order |
Keys must be hashable (`str`, `int`, `tuple` of hashablesβnot `list`/`dict`).
## π‘ Examples [#-examples]
**Safe access and merge:**
```python
user = {"id": 1, "name": "Ada"}
role = {"role": "admin"}
print(user.get("email", "n/a"))
merged = user | role # {"id": 1, "name": "Ada", "role": "admin"}
```
**Counting and grouping:**
```python
from collections import Counter, defaultdict
words = ["a", "b", "a", "c", "b", "a"]
print(Counter(words)) # Counter({'a': 3, 'b': 2, 'c': 1})
by_len: dict[int, list[str]] = defaultdict(list)
for w in words:
by_len[len(w)].append(w)
```
**Dict comprehension / invert:**
```python
scores = {"alice": 10, "bob": 7}
passed = {k: v for k, v in scores.items() if v >= 8}
# invert (values must be unique/hashable)
inv = {v: k for k, v in scores.items()}
```
**Pattern matching (3.10+):**
```python
payload = {"type": "user", "id": 42}
match payload:
case {"type": "user", "id": int(uid)}:
print("user", uid)
case _:
print("unknown")
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `d[key]` raises `KeyError` if missingβuse `.get` or `in`.
* Mutating a dict while iterating its views is unsafe; iterate over `list(d)` or copy.
* Nested dicts share referencesβ`copy()` is shallow; use `copy.deepcopy` when needed.
* Using unhashable types as keys raises `TypeError`.
* `dict` equality ignores order; do not rely on order for identity checks.
## π Related [#-related]
* [Types](/docs/python/types)
* [Loops](/docs/python/loops)
* [\*args and \*\*kwargs](/docs/python/kwags)
* [Strings](/docs/python/strings)
* [Examples: list to dict](/docs/python/examples/list-to-dict)
* [Examples: merge lists](/docs/python/examples/merge-lists)
---
# Docstrings (/docs/python/docstrings)
# Docstrings [#docstrings]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Docstrings are string literals as the first statement in a module, class, function, or method. Tools (`help()`, Sphinx, IDEs) read them via `__doc__`. Prefer clear one-liners for simple APIs; use structured styles (Google, NumPy, reST) for public libraries.
## π§ Core concepts [#-core-concepts]
| Rule | Detail |
| ---------- | ----------------------------------------------------------- |
| Placement | First statement inside the body |
| Style | Triple quotes `"""..."""` (PEP 257) |
| One-liner | Imperative mood: `"""Return the user's display name."""` |
| Multi-line | Summary line, blank line, then details |
| Attributes | Document on the class or via Attributes section |
| Type hints | Prefer annotations; docstring describes behavior, not types |
Common sections: `Args`, `Returns`, `Raises`, `Yields`, `Examples`, `Note`.
## π‘ Examples [#-examples]
**Google style:**
```python
def divide(a: float, b: float) -> float:
"""Divide ``a`` by ``b``.
Args:
a: Numerator.
b: Denominator; must be non-zero.
Returns:
The quotient ``a / b``.
Raises:
ZeroDivisionError: If ``b`` is 0.
"""
return a / b
```
**Module and class:**
```python
"""User authentication helpers.
This module wraps password hashing and session tokens.
"""
class Session:
"""In-memory session store for a single process."""
def get(self, key: str) -> str | None:
"""Return the value for ``key``, or ``None`` if missing."""
...
```
**Access at runtime:**
```python
def greet(name: str) -> str:
"""Return a greeting for ``name``."""
return f"Hello, {name}"
print(greet.__doc__)
help(greet)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Comments (`#`) are not docstrings and are not exposed via `help()`.
* Putting code between the signature and the docstring breaks `__doc__`.
* Do not duplicate type information already in annotations.
* Keep examples in docstrings executable when using doctest; stale examples mislead.
* Private helpers can use short one-liners; over-documenting noise hurts readability.
## π Related [#-related]
* [Types](/docs/python/types)
* [F-strings](/docs/python/f-string)
* [Import / export](/docs/python/import-export)
* [\*args and \*\*kwargs](/docs/python/kwags)
* [Strings](/docs/python/strings)
---
# Dotenv (/docs/python/dotenv)
# Dotenv [#dotenv]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`python-dotenv` loads a `.env` file into `os.environ` so local secrets stay out of source. Install: `pip install python-dotenv`. Call `load_dotenv()` early (app entrypoint), then read with `os.getenv`.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ------------------- | ------------------------------------------------ |
| `load_dotenv()` | Load `.env` from cwd / parents into `os.environ` |
| `load_dotenv(path)` | Explicit file path |
| `override=False` | Default: donβt clobber existing env |
| `find_dotenv()` | Locate nearest `.env` |
| `os.getenv("KEY")` | Read after load |
| `os.environ["KEY"]` | Read/set in process |
Typical `.env` lines: `KEY=value` (no quotes needed for simple values). Never commit real secrets β keep `.env` in `.gitignore`.
## π‘ Examples [#-examples]
**Basic load:**
```python
import os
from dotenv import load_dotenv
load_dotenv() # reads .env
db_url = os.getenv("DATABASE_URL")
debug = os.getenv("DEBUG", "0") == "1"
print(bool(db_url), debug)
```
**Explicit path:**
```python
from pathlib import Path
from dotenv import load_dotenv
import os
env_path = Path(__file__).resolve().parent / ".env"
load_dotenv(env_path)
print(os.environ.get("API_KEY", ""))
```
**Override existing vars:**
```python
from dotenv import load_dotenv
import os
os.environ["APP_ENV"] = "prod"
load_dotenv(override=True) # .env wins
print(os.getenv("APP_ENV"))
```
**Fail fast if required:**
```python
import os
from dotenv import load_dotenv
load_dotenv()
token = os.getenv("SECRET_TOKEN")
if not token:
raise SystemExit("SECRET_TOKEN is required")
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `load_dotenv` does not raise if `.env` is missing β check return value or required keys.
* Default `override=False`: shell/CI env wins over `.env`.
* Values are strings; parse bools carefully (`"false"` is truthy in Python).
* Donβt commit `.env`; ship `.env.example` with dummy keys.
* Calling `load_dotenv` late (after code already read env) is a common bug.
## π Related [#-related]
* [Os](/docs/python/os)
* [Venv](/docs/python/venv)
* [Pip](/docs/python/pip)
* [Argparse](/docs/python/argparse)
---
# Enums (/docs/python/enums)
# Enums [#enums]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`enum.Enum` defines a set of named symbolic values. Use enums instead of magic strings/ints for status codes, modes, and options. Python 3.11+ adds `StrEnum`, improved `Flag`, and more.
## π§ Core concepts [#-core-concepts]
| Type | Use |
| ------------------ | ---------------------------------- |
| `Enum` | Classic named values |
| `IntEnum` | Comparable to `int` |
| `StrEnum` | `str` mixin (3.11+) |
| `Flag` / `IntFlag` | Bitwise combinations |
| `auto()` | Auto-assign values |
| `enum.unique` | Decorator: forbid duplicate values |
Members are singletons: `Color.RED is Color.RED`. Access by name (`Color["RED"]`) or value (`Color(1)`).
## π‘ Examples [#-examples]
**Basic Enum:**
```python
from enum import Enum, auto
class Status(Enum):
PENDING = auto()
RUNNING = auto()
DONE = auto()
s = Status.RUNNING
print(s.name, s.value)
print(Status["DONE"])
```
**StrEnum (3.11+):**
```python
from enum import StrEnum
class Role(StrEnum):
ADMIN = "admin"
USER = "user"
def greet(role: Role) -> str:
return f"hello {role}" # role is a str
assert Role.ADMIN == "admin"
```
**Flags:**
```python
from enum import Flag, auto
class Perm(Flag):
READ = auto()
WRITE = auto()
EXEC = auto()
rw = Perm.READ | Perm.WRITE
print(Perm.READ in rw)
print(bool(rw & Perm.EXEC))
```
**Match (3.10+):**
```python
match Status.DONE:
case Status.PENDING | Status.RUNNING:
print("busy")
case Status.DONE:
print("finished")
case _:
print("unknown")
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Comparing `Enum` to raw ints fails unless you use `IntEnum`.
* Functional API `Enum("Color", "RED GREEN")` is less readable for large enums.
* Pickling / JSON: serialize `.value` or `.name` explicitly.
* Do not mutate `.value` on members.
* `auto()` values depend on declaration order β stable only if you don't reorder carelessly.
## π Related [#-related]
* [Classes](/docs/python/classes)
* [Dataclasses](/docs/python/dataclasses)
* [Conditionals](/docs/python/conditionals)
* [Typing hints](/docs/python/typing-hints)
* [JSON](/docs/python/json)
* [Exceptions](/docs/python/exceptions)
---
# Examples (/docs/python/examples)
# Examples [#examples]
Python notes in **Examples**.
---
# CLI Tool (/docs/python/examples/cli-tool)
# CLI Tool [#cli-tool]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Build a small command-line tool with `argparse`, structured logging, and an entrypoint under `if __name__ == "__main__"`. Package it later with a console script in `pyproject.toml`.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| ---------- | ------------------------ |
| Parser | Flags, args, subcommands |
| Exit codes | `0` ok, non-zero failure |
| Logging | `-v` / `-q` adjust level |
| `Path` | File arguments |
| `main()` | Testable entry |
Keep business logic in functions; CLI layer only parses and prints.
## π‘ Examples [#-examples]
**greet.py:**
```python
from __future__ import annotations
import argparse
import logging
import sys
from pathlib import Path
log = logging.getLogger("greet")
def greet(name: str, shout: bool = False) -> str:
msg = f"Hello, {name}!"
return msg.upper() if shout else msg
def build_parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser(description="Greet someone")
p.add_argument("name", help="person to greet")
p.add_argument("-s", "--shout", action="store_true")
p.add_argument("-o", "--output", type=Path)
p.add_argument("-v", "--verbose", action="count", default=0)
return p
def main(argv: list[str] | None = None) -> int:
args = build_parser().parse_args(argv)
level = logging.WARNING - (10 * min(args.verbose, 2))
logging.basicConfig(level=level, format="%(levelname)s %(message)s")
message = greet(args.name, shout=args.shout)
log.info("writing greeting")
if args.output:
args.output.write_text(message + "\n", encoding="utf-8")
else:
print(message)
return 0
if __name__ == "__main__":
sys.exit(main())
```
**Usage:**
```shellscript
python greet.py Ada
python greet.py Ada --shout -v -o out.txt
```
**Test main without subprocess:**
```python
def test_main_stdout(capsys):
assert main(["Ada"]) == 0
assert "Hello, Ada!" in capsys.readouterr().out
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Parsing at import time breaks tests and packaging.
* Printing errors to stdout β use stderr + non-zero exit.
* Relative paths depend on cwd β document or resolve carefully.
* Catching all exceptions without logging hides failures.
* For richer CLIs, consider Typer/Click once argparse gets painful.
## π Related [#-related]
* [Pytest example](/docs/python/examples/pytest-example)
* [HTTP requests](/docs/python/examples/http-requests)
* [../argparse.md](/docs/python/argparse)
* [../logging.md](/docs/python/logging)
* [../pathlib.md](/docs/python/pathlib)
* [../packaging.md](/docs/python/packaging)
---
# Convert files (/docs/python/examples/convert-files)
# Convert files [#convert-files]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Convert between common text/data formats (CSV β JSON, encoding changes, line endings) with the standard library. Keep conversions streaming when files are large; always specify encodings explicitly on Windows.
## π§ Core concepts [#-core-concepts]
| Task | Tools |
| -------------- | ------------------------------------------------------ |
| CSV β JSON | `csv`, `json` |
| Encoding | `open(..., encoding=...)`, `pathlib.Path.read_text` |
| Paths | `pathlib.Path` |
| Binary vs text | Use text mode for CSV/JSON; binary for images/archives |
| Streaming | Iterate rows instead of `read()` whole file |
## π‘ Examples [#-examples]
**CSV to JSON:**
```python
import csv
import json
from pathlib import Path
def csv_to_json(src: Path, dest: Path) -> None:
with src.open(newline="", encoding="utf-8") as f:
rows = list(csv.DictReader(f))
dest.write_text(json.dumps(rows, indent=2, ensure_ascii=False), encoding="utf-8")
if __name__ == "__main__":
csv_to_json(Path("users.csv"), Path("users.json"))
```
**JSON to CSV:**
```python
import csv
import json
from pathlib import Path
def json_to_csv(src: Path, dest: Path) -> None:
rows = json.loads(src.read_text(encoding="utf-8"))
if not rows:
dest.write_text("", encoding="utf-8")
return
fieldnames = list(rows[0].keys())
with dest.open("w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=fieldnames)
writer.writeheader()
writer.writerows(rows)
if __name__ == "__main__":
json_to_csv(Path("users.json"), Path("users.csv"))
```
**Re-encode a text file (e.g. latin-1 β utf-8):**
```python
from pathlib import Path
def reencode(src: Path, dest: Path, from_enc: str = "latin-1", to_enc: str = "utf-8") -> None:
text = src.read_text(encoding=from_enc)
dest.write_text(text, encoding=to_enc)
if __name__ == "__main__":
reencode(Path("legacy.txt"), Path("utf8.txt"))
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Omitting `encoding=` uses locale defaultsβbreaks cross-platform.
* CSV needs `newline=""` to avoid blank lines on Windows.
* Nested JSON objects do not map cleanly to flat CSVβflatten or keep JSON.
* Loading huge files with `json.loads` / `list(DictReader)` can exhaust memory.
## π Related [#-related]
* [Manage files](/docs/python/examples/manage-files)
* [Encrypt](/docs/python/examples/encrypt)
* [List to dict](/docs/python/examples/list-to-dict)
* [Merge lists](/docs/python/examples/merge-lists)
* [../strings.md](/docs/python/strings)
* [../dictionaries.md](/docs/python/dictionaries)
---
# CSV and Excel (/docs/python/examples/csv-excel)
# CSV and Excel [#csv-and-excel]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Read/write tabular data with the stdlib `csv` module. For Excel (`.xlsx`), use `openpyxl` or `pandas`. Prefer CSV for interchange; Excel when stakeholders need workbooks.
## π§ Core concepts [#-core-concepts]
| Task | Tool |
| -------------- | ------------------------------------ |
| CSV read/write | `csv.DictReader` / `DictWriter` |
| Dialect | excel, excel-tab, custom |
| Encoding | `utf-8` / `utf-8-sig` (Excel) |
| xlsx | `openpyxl`, `pandas.read_excel` |
| Large files | Stream row-by-row; avoid loading all |
Always open CSV with `newline=""` (as docs recommend) and an explicit encoding.
## π‘ Examples [#-examples]
**CSV round-trip:**
```python
import csv
from pathlib import Path
path = Path("people.csv")
rows = [
{"name": "Ada", "score": "10"},
{"name": "Bob", "score": "7"},
]
with path.open("w", encoding="utf-8", newline="") as f:
writer = csv.DictWriter(f, fieldnames=["name", "score"])
writer.writeheader()
writer.writerows(rows)
with path.open("r", encoding="utf-8", newline="") as f:
for row in csv.DictReader(f):
print(row["name"], int(row["score"]))
```
**pandas quick path:**
```python
import pandas as pd
df = pd.read_csv("people.csv")
df["score"] = df["score"].astype(int)
df.to_excel("people.xlsx", index=False)
df2 = pd.read_excel("people.xlsx")
print(df2.head())
```
**openpyxl minimal:**
```python
from openpyxl import Workbook
wb = Workbook()
ws = wb.active
ws.title = "Scores"
ws.append(["name", "score"])
ws.append(["Ada", 10])
wb.save("scores.xlsx")
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Excel often expects `utf-8-sig` for CSV to show Unicode correctly.
* Forgetting `newline=""` on Windows can insert blank lines.
* Type coercion: CSV is all strings until you parse.
* Formulas / merged cells in xlsx need library-specific handling.
* Don't load multi-GB CSVs into a single list β stream or use pandas chunks.
## π Related [#-related]
* [Manage files](/docs/python/examples/manage-files)
* [Dataclass pipeline](/docs/python/examples/dataclass-pipeline)
* [SQLite](/docs/python/examples/sqlite)
* [../pathlib.md](/docs/python/pathlib)
* [../files\_io.md](/docs/python/files-io)
* [../dataclasses.md](/docs/python/dataclasses)
---
# Dataclass Config (/docs/python/examples/dataclass-config)
# Dataclass Config [#dataclass-config]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Load app settings into a frozen `@dataclass` from environment variables with defaults and light validation.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| ------------------------- | ------------------------- |
| `@dataclass(frozen=True)` | Immutable config snapshot |
| `os.environ` | External configuration |
| `classmethod` factory | `from_env()` constructor |
| Validation | Fail fast on bad values |
## π‘ Examples [#-examples]
**dataclass\_config.py:**
```python
from __future__ import annotations
import os
from dataclasses import dataclass
@dataclass(frozen=True, slots=True)
class AppConfig:
app_name: str
debug: bool
port: int
database_url: str
@classmethod
def from_env(cls) -> AppConfig:
port_raw = os.environ.get("PORT", "8000")
try:
port = int(port_raw)
except ValueError as e:
raise ValueError(f"PORT must be an int, got {port_raw!r}") from e
if not 1 <= port <= 65535:
raise ValueError(f"PORT out of range: {port}")
debug = os.environ.get("DEBUG", "0").lower() in {"1", "true", "yes"}
database_url = os.environ.get("DATABASE_URL", "sqlite:///./app.db")
if not database_url:
raise ValueError("DATABASE_URL must not be empty")
return cls(
app_name=os.environ.get("APP_NAME", "demo"),
debug=debug,
port=port,
database_url=database_url,
)
def main() -> None:
cfg = AppConfig.from_env()
print(cfg)
if __name__ == "__main__":
main()
```
**Usage:**
```shellscript
set PORT=9000
set DEBUG=true
python dataclass_config.py
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Mutable defaults (`list`, `dict`) need `field(default_factory=...)`.
* Frozen instances cannot be assigned after creation β rebuild instead.
* Booleans from env are strings; `"False"` is truthy if you only check `bool(s)`.
## π Related [#-related]
* [Dataclass pipeline](/docs/python/examples/dataclass-pipeline)
* [../dataclasses.md](/docs/python/dataclasses)
* [../enums.md](/docs/python/enums)
---
# Dataclass Pipeline (/docs/python/examples/dataclass-pipeline)
# Dataclass Pipeline [#dataclass-pipeline]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Model rows as dataclasses, then map/filter/validate through a small pipeline. Clear types beat anonymous dicts for ETL-style scripts.
## π§ Core concepts [#-core-concepts]
| Stage | Role |
| --------- | -------------------------------------- |
| Parse | Raw dict/CSV β dataclass |
| Validate | `__post_init__` or explicit checks |
| Transform | Pure functions returning new instances |
| Sink | Write CSV/DB/JSON |
Prefer immutable (`frozen=True`) records when transforming functionally.
## π‘ Examples [#-examples]
**Model + parse:**
```python
from __future__ import annotations
from dataclasses import dataclass, asdict, replace
import csv
from pathlib import Path
@dataclass(frozen=True, slots=True)
class Order:
id: int
customer: str
total: float
def __post_init__(self) -> None:
if self.total < 0:
raise ValueError("total must be >= 0")
def parse_order(row: dict[str, str]) -> Order:
return Order(
id=int(row["id"]),
customer=row["customer"].strip(),
total=float(row["total"]),
)
```
**Pipeline:**
```python
def load_orders(path: Path) -> list[Order]:
with path.open(encoding="utf-8", newline="") as f:
return [parse_order(r) for r in csv.DictReader(f)]
def with_tax(order: Order, rate: float = 0.1) -> Order:
return replace(order, total=round(order.total * (1 + rate), 2))
def big_orders(orders: list[Order], min_total: float = 100.0) -> list[Order]:
return [o for o in orders if o.total >= min_total]
def write_jsonl(path: Path, orders: list[Order]) -> None:
import json
lines = [json.dumps(asdict(o)) for o in orders]
path.write_text("\n".join(lines) + "\n", encoding="utf-8")
def run(src: Path, dst: Path) -> None:
orders = load_orders(src)
taxed = [with_tax(o) for o in orders]
write_jsonl(dst, big_orders(taxed))
```
**Sample CSV:**
```plaintext
id,customer,total
1,Ada,120.0
2,Bob,40.0
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Mutable default fields need `field(default_factory=...)`.
* Silent bad rows: decide whether to skip + log or fail fast.
* Floating money β prefer integer cents in real billing systems.
* `asdict` is shallow for nested dataclasses.
* Mixing validation in many places β keep one parse boundary.
## π Related [#-related]
* [CSV and Excel](/docs/python/examples/csv-excel)
* [SQLite](/docs/python/examples/sqlite)
* [Pytest example](/docs/python/examples/pytest-example)
* [../dataclasses.md](/docs/python/dataclasses)
* [../json.md](/docs/python/json)
* [../typing\_hints.md](/docs/python/typing-hints)
---
# Decorators Timing (/docs/python/examples/decorators-timing)
# Decorators Timing [#decorators-timing]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Time function calls with a reusable decorator that preserves the wrapped signature via `functools.wraps`.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| ------------------- | ----------------------------- |
| Decorator | Wrap callable, return wrapper |
| `functools.wraps` | Keep `__name__` / docstring |
| `time.perf_counter` | High-resolution elapsed time |
| `*args, **kwargs` | Forward any signature |
## π‘ Examples [#-examples]
**decorators\_timing.py:**
```python
from __future__ import annotations
import functools
import time
from collections.abc import Callable
from typing import ParamSpec, TypeVar
P = ParamSpec("P")
R = TypeVar("R")
def timed(fn: Callable[P, R]) -> Callable[P, R]:
@functools.wraps(fn)
def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
start = time.perf_counter()
try:
return fn(*args, **kwargs)
finally:
elapsed_ms = (time.perf_counter() - start) * 1000
print(f"{fn.__name__} took {elapsed_ms:.2f} ms")
return wrapper
@timed
def slow_add(a: int, b: int) -> int:
time.sleep(0.05)
return a + b
if __name__ == "__main__":
print(slow_add(2, 3))
```
**Optional label / logger:**
```python
import logging
log = logging.getLogger(__name__)
def timed_log(fn: Callable[P, R]) -> Callable[P, R]:
@functools.wraps(fn)
def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
start = time.perf_counter()
try:
return fn(*args, **kwargs)
finally:
log.info("%s %.2fms", fn.__name__, (time.perf_counter() - start) * 1000)
return wrapper
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Without `@wraps`, tests and docs see `wrapper` instead of the real name.
* Timing includes exceptions β use `finally` so failures still report duration.
* Decorators on methods need care with `self`; this pattern still works.
## π Related [#-related]
* [../decorators.md](/docs/python/decorators)
* [../functions.md](/docs/python/functions)
* [../typing\_hints.md](/docs/python/typing-hints)
---
# Encrypt (/docs/python/examples/encrypt)
# Encrypt [#encrypt]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Encrypt secrets at rest with modern cryptographyβnot homemade XOR. For passwords use one-way hashing (`hashlib.scrypt` / `bcrypt`). For reversible data use authenticated encryption (Fernet or AES-GCM via the `cryptography` package). Never commit keys; load from environment.
## π§ Core concepts [#-core-concepts]
| Goal | Approach |
| ----------------- | -------------------------------------------------- |
| Password storage | Slow hash + salt (`scrypt`, `argon2`, `bcrypt`) |
| Symmetric encrypt | Fernet (AES + HMAC) or AES-GCM |
| Key storage | Env var / secret managerβnot source |
| Integrity | Prefer AEAD; detect tampering |
| Stdlib only | `hashlib`, `secrets`, `hmac`βlimited for full AEAD |
Install: `pip install cryptography`
## π‘ Examples [#-examples]
**Password hashing (stdlib scrypt):**
```python
import hashlib
import secrets
def hash_password(password: str) -> str:
salt = secrets.token_bytes(16)
digest = hashlib.scrypt(
password.encode("utf-8"),
salt=salt,
n=2**14,
r=8,
p=1,
dklen=32,
)
return f"{salt.hex()}${digest.hex()}"
def verify_password(password: str, stored: str) -> bool:
salt_hex, digest_hex = stored.split("$", 1)
salt = bytes.fromhex(salt_hex)
digest = hashlib.scrypt(
password.encode("utf-8"),
salt=salt,
n=2**14,
r=8,
p=1,
dklen=32,
)
return secrets.compare_digest(digest.hex(), digest_hex)
if __name__ == "__main__":
stored = hash_password("s3cret!")
assert verify_password("s3cret!", stored)
```
**Fernet encrypt / decrypt:**
```python
import os
from cryptography.fernet import Fernet
def get_fernet() -> Fernet:
key = os.environ.get("APP_FERNET_KEY")
if not key:
# generate once: Fernet.generate_key().decode()
raise RuntimeError("Set APP_FERNET_KEY")
return Fernet(key.encode("utf-8") if isinstance(key, str) else key)
def encrypt_text(plain: str) -> bytes:
return get_fernet().encrypt(plain.encode("utf-8"))
def decrypt_text(token: bytes) -> str:
return get_fernet().decrypt(token).decode("utf-8")
if __name__ == "__main__":
# Demo only β set a real key in production
os.environ["APP_FERNET_KEY"] = Fernet.generate_key().decode()
token = encrypt_text("hello")
print(decrypt_text(token))
```
**Generate a key once:**
```python
from cryptography.fernet import Fernet
print(Fernet.generate_key().decode())
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* MD5/SHA-1 alone are not password hashesβuse scrypt/argon2/bcrypt.
* Reusing IVs/nonces with AES-GCM is catastrophic; Fernet handles this for you.
* Logging plaintext or keys defeats encryption.
* `hashlib.md5` for βchecksumsβ is fine for non-security integrity only.
## π Related [#-related]
* [Manage files](/docs/python/examples/manage-files)
* [Convert files](/docs/python/examples/convert-files)
* [Server](/docs/python/examples/server)
* [Telegram bot](/docs/python/examples/tg-bot)
* [../import\_export.md](/docs/python/import-export)
* [../types.md](/docs/python/types)
---
# Hello CLI (/docs/python/examples/hello-cli)
# Hello CLI [#hello-cli]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Minimal command-line greeting with `argparse`: one positional argument, one flag, and a clean `main()` entrypoint.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| -------------------- | ------------------------------ |
| `ArgumentParser` | Define CLI interface |
| Positional arg | Required `name` |
| `store_true` | Optional `--shout` flag |
| `main()` + exit code | Testable entry, `0` on success |
## π‘ Examples [#-examples]
**hello\_cli.py:**
```python
from __future__ import annotations
import argparse
import sys
def greet(name: str, shout: bool = False) -> str:
msg = f"Hello, {name}!"
return msg.upper() if shout else msg
def build_parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser(description="Say hello")
p.add_argument("name", help="who to greet")
p.add_argument("-s", "--shout", action="store_true", help="uppercase")
return p
def main(argv: list[str] | None = None) -> int:
args = build_parser().parse_args(argv)
print(greet(args.name, shout=args.shout))
return 0
if __name__ == "__main__":
sys.exit(main())
```
**Usage:**
```shellscript
python hello_cli.py Ada
python hello_cli.py Ada --shout
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Do not parse `sys.argv` at import time β keep parsing inside `main()`.
* Prefer returning an exit code over `sys.exit()` deep in helpers.
* `print` alone is fine for tiny tools; add logging when behavior grows.
## π Related [#-related]
* [CLI tool](/docs/python/examples/cli-tool)
* [../argparse.md](/docs/python/argparse)
* [../print\_input.md](/docs/python/print-input)
---
# HTTP Requests (/docs/python/examples/http-requests)
# HTTP Requests [#http-requests]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Call HTTP APIs with the popular `requests` library (or stdlib `urllib`). Handle timeouts, status codes, and JSON payloads explicitly. Install: `python -m pip install requests`.
## π§ Core concepts [#-core-concepts]
| Task | Approach |
| ------------ | ----------------------------------------- |
| GET/POST | `requests.get/post` |
| JSON body | `json=` param |
| Query string | `params=` |
| Headers | `headers=` |
| Timeout | Always set `timeout=` |
| Errors | `raise_for_status()` |
| Session | `requests.Session()` for connection reuse |
Prefer sessions for multiple calls to the same host. Never hardcode secrets β use env vars.
## π‘ Examples [#-examples]
**GET JSON:**
```python
import os
import requests
def fetch_user(user_id: int) -> dict:
token = os.environ["API_TOKEN"]
url = f"https://api.example.com/users/{user_id}"
resp = requests.get(
url,
headers={"Authorization": f"Bearer {token}"},
timeout=10,
)
resp.raise_for_status()
return resp.json()
```
**POST and session:**
```python
import requests
with requests.Session() as s:
s.headers.update({"Accept": "application/json"})
r = s.post(
"https://httpbin.org/post",
json={"name": "Ada"},
timeout=10,
)
r.raise_for_status()
print(r.json()["json"])
```
**stdlib alternative:**
```python
import json
from urllib.request import Request, urlopen
req = Request(
"https://httpbin.org/get",
headers={"Accept": "application/json"},
)
with urlopen(req, timeout=10) as resp:
data = json.load(resp)
print(data["url"])
```
**Retry sketch:** see [retry backoff](/docs/python/examples/retry-backoff).
## β οΈ Pitfalls [#οΈ-pitfalls]
* Missing `timeout` can hang forever.
* Not calling `raise_for_status()` treats 404/500 as success.
* Logging response bodies may leak PII/tokens.
* `verify=False` disables TLS verification β avoid.
* Relative redirects + mixed POST/GET behavior β know `allow_redirects`.
## π Related [#-related]
* [Retry backoff](/docs/python/examples/retry-backoff)
* [Scrape HTML](/docs/python/examples/scrape-html)
* [CLI tool](/docs/python/examples/cli-tool)
* [../json.md](/docs/python/json)
* [../exceptions.md](/docs/python/exceptions)
* [../logging.md](/docs/python/logging)
---
# List to dict (/docs/python/examples/list-to-dict)
# List to dict [#list-to-dict]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Turn sequences into dictionaries for O(1) lookup: pairs β mapping, list of objects β keyed by id, or group values under a key. Prefer `dict()`, comprehensions, and `collections.defaultdict` over manual loops when clear.
## π§ Core concepts [#-core-concepts]
| Pattern | Idiom |
| ------------- | -------------------------------------------- |
| Pairs | `dict(pairs)` / `\{k: v for k, v in pairs\}` |
| Key function | `\{item.id: item for item in items\}` |
| Enumerate | `dict(enumerate(items))` |
| Group | `defaultdict(list)` |
| Zip two lists | `dict(zip(keys, values))` |
Duplicate keys: later values win in a comprehension/`dict`.
## π‘ Examples [#-examples]
**Zip keys and values:**
```python
keys = ["id", "name", "role"]
values = [1, "Ada", "admin"]
user = dict(zip(keys, values, strict=True))
print(user) # {'id': 1, 'name': 'Ada', 'role': 'admin'}
```
**Index objects by id:**
```python
from dataclasses import dataclass
@dataclass(frozen=True)
class User:
id: int
name: str
users = [User(1, "Ada"), User(2, "Bob")]
by_id = {u.id: u for u in users}
print(by_id[1].name) # Ada
```
**Group list of dicts:**
```python
from collections import defaultdict
rows = [
{"dept": "eng", "name": "Ada"},
{"dept": "hr", "name": "Bob"},
{"dept": "eng", "name": "Cary"},
]
grouped: dict[str, list[str]] = defaultdict(list)
for row in rows:
grouped[row["dept"]].append(row["name"])
print(dict(grouped))
```
**Pairs from nested lists:**
```python
pairs = [["a", 1], ["b", 2], ["a", 3]]
# last wins for duplicate keys
print(dict(pairs)) # {'a': 3, 'b': 2}
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `dict(zip(keys, values))` silently truncates if lengths differβuse `strict=True` (3.10+).
* Unhashable keys (`list`, `dict`) raise `TypeError`.
* Building `\{i: x for i, x in enumerate(xs)\}` is rarely neededβkeep the list if order matters.
* Mutating values that are shared lists/dicts affects every reference.
## π Related [#-related]
* [Merge lists](/docs/python/examples/merge-lists)
* [Convert files](/docs/python/examples/convert-files)
* [Manage files](/docs/python/examples/manage-files)
* [../dictionaries.md](/docs/python/dictionaries)
* [../loops.md](/docs/python/loops)
* [../types.md](/docs/python/types)
---
# Manage files (/docs/python/examples/manage-files)
# Manage files [#manage-files]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Create, read, write, copy, move, and delete files with `pathlib` and `shutil`. Prefer context managers (`with`) so handles close reliably. Use absolute paths in scripts; resolve relative to a known base directory.
## π§ Core concepts [#-core-concepts]
| Task | API |
| ---------------- | ------------------------------- |
| Paths | `pathlib.Path` |
| Read/write text | `Path.read_text` / `write_text` |
| Read/write bytes | `read_bytes` / `write_bytes` |
| List | `iterdir`, `glob`, `rglob` |
| Copy / move | `shutil.copy2`, `shutil.move` |
| Delete | `Path.unlink`, `shutil.rmtree` |
| Temp | `tempfile.TemporaryDirectory` |
Always pass `encoding="utf-8"` for text unless you know otherwise.
## π‘ Examples [#-examples]
**Read, write, append:**
```python
from pathlib import Path
base = Path("data")
base.mkdir(parents=True, exist_ok=True)
path = base / "notes.txt"
path.write_text("hello\n", encoding="utf-8")
path.write_text(path.read_text(encoding="utf-8") + "world\n", encoding="utf-8")
print(path.read_text(encoding="utf-8"))
```
**List and filter:**
```python
from pathlib import Path
root = Path("data")
for p in sorted(root.rglob("*.txt")):
if p.is_file():
print(p, p.stat().st_size)
```
**Copy, move, delete safely:**
```python
import shutil
from pathlib import Path
src = Path("data/notes.txt")
backup = Path("data/backup")
backup.mkdir(parents=True, exist_ok=True)
shutil.copy2(src, backup / src.name)
shutil.move(str(backup / src.name), backup / "notes.bak")
target = backup / "notes.bak"
if target.exists():
target.unlink()
```
**Atomic-ish write (write temp then replace):**
```python
from pathlib import Path
def atomic_write(path: Path, text: str) -> None:
tmp = path.with_suffix(path.suffix + ".tmp")
tmp.write_text(text, encoding="utf-8")
tmp.replace(path)
if __name__ == "__main__":
atomic_write(Path("data/config.json"), '{"ok": true}\n')
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `open` without `with` leaks handles on exceptions.
* `Path.unlink()` raises if missingβuse `missing_ok=True` (3.8+).
* `shutil.rmtree` is destructive; confirm paths before deleting.
* Relative paths depend on the process cwdβnot the script location (`Path(__file__).resolve().parent`).
## π Related [#-related]
* [Convert files](/docs/python/examples/convert-files)
* [Encrypt](/docs/python/examples/encrypt)
* [Server](/docs/python/examples/server)
* [../import\_export.md](/docs/python/import-export)
* [../strings.md](/docs/python/strings)
* [../f-string.md](/docs/python/f-string)
---
# Merge lists (/docs/python/examples/merge-lists)
# Merge lists [#merge-lists]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Combine lists by concatenation, extension, or set-like union while controlling duplicates and order. Choose the approach based on whether you need stable order, uniqueness, or pairwise zipping.
## π§ Core concepts [#-core-concepts]
| Goal | Approach |
| --------------------- | ------------------------------------ |
| Concatenate | `a + b`, `[*a, *b]` |
| Extend in place | `a.extend(b)` |
| Unique preserve order | `dict.fromkeys(a + b)` |
| Set union | `list(set(a) \| set(b))` (unordered) |
| Interleave / pair | `zip`, nested loops |
| Deep merge objects | Custom logic / dict merge |
## π‘ Examples [#-examples]
**Concatenate and unique (order preserved):**
```python
a = [1, 2, 3, 2]
b = [3, 4, 5]
merged = a + b
unique = list(dict.fromkeys(merged))
print(merged) # [1, 2, 3, 2, 3, 4, 5]
print(unique) # [1, 2, 3, 4, 5]
```
**Extend vs new list:**
```python
a = [1, 2]
b = [3, 4]
a.extend(b) # mutates a β [1, 2, 3, 4]
c = [*a, *b] # new list (here a already includes b)
```
**Merge list of dicts by key:**
```python
from typing import Any
def merge_by_id(*groups: list[dict[str, Any]]) -> list[dict[str, Any]]:
by_id: dict[Any, dict[str, Any]] = {}
for group in groups:
for row in group:
key = row["id"]
by_id[key] = {**by_id.get(key, {}), **row}
return list(by_id.values())
left = [{"id": 1, "name": "Ada"}, {"id": 2, "name": "Bob"}]
right = [{"id": 2, "role": "admin"}, {"id": 3, "name": "Cary"}]
print(merge_by_id(left, right))
```
**Flatten nested lists:**
```python
nested = [[1, 2], [3], [4, 5]]
flat = [x for group in nested for x in group]
print(flat) # [1, 2, 3, 4, 5]
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `a += b` mutates `a`; `a = a + b` allocates a new list.
* `set` union drops order and requires hashable elements.
* Aliasing: `c = a; c.extend(b)` also changes `a`.
* Nested mutable elements are shared after shallow merges.
## π Related [#-related]
* [List to dict](/docs/python/examples/list-to-dict)
* [Convert files](/docs/python/examples/convert-files)
* [../loops.md](/docs/python/loops)
* [../dictionaries.md](/docs/python/dictionaries)
* [../types.md](/docs/python/types)
* [../kwags.md](/docs/python/kwags)
---
# Parse JSON API (/docs/python/examples/parse-json-api)
# Parse JSON API [#parse-json-api]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Fetch a JSON API with `urllib` or `httpx`, validate the shape lightly, and extract fields safely.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| ----------------- | -------------------------------- |
| GET + JSON body | Typical REST read |
| `response.json()` | Decode payload |
| Key checks | Avoid `KeyError` on partial data |
| Timeout | Fail fast on hung networks |
## π‘ Examples [#-examples]
**parse\_json\_api.py (stdlib):**
```python
from __future__ import annotations
import json
import urllib.error
import urllib.request
from typing import Any
def fetch_user(user_id: int, timeout: float = 10.0) -> dict[str, Any]:
url = f"https://jsonplaceholder.typicode.com/users/{user_id}"
req = urllib.request.Request(url, headers={"Accept": "application/json"})
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
raw = resp.read().decode("utf-8")
except urllib.error.HTTPError as e:
raise RuntimeError(f"HTTP {e.code} for user {user_id}") from e
except urllib.error.URLError as e:
raise RuntimeError(f"network error: {e.reason}") from e
data = json.loads(raw)
if not isinstance(data, dict):
raise ValueError("expected a JSON object")
if "id" not in data or "email" not in data:
raise ValueError("missing id or email")
return data
def main() -> None:
user = fetch_user(1)
print(f"{user['name']} <{user['email']}>")
if __name__ == "__main__":
main()
```
**With httpx (optional):**
```python
import httpx
def fetch_user_httpx(user_id: int) -> dict:
r = httpx.get(
f"https://jsonplaceholder.typicode.com/users/{user_id}",
timeout=10.0,
)
r.raise_for_status()
return r.json()
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Never assume every key exists β APIs change and error bodies differ.
* Always set a timeout; default can hang forever.
* `json.loads` on HTML error pages raises `JSONDecodeError` β catch it.
## π Related [#-related]
* [HTTP requests](/docs/python/examples/http-requests)
* [../json.md](/docs/python/json)
* [../exceptions.md](/docs/python/exceptions)
---
# Pathlib Walk (/docs/python/examples/pathlib-walk)
# Pathlib Walk [#pathlib-walk]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Walk a directory tree with `pathlib.Path.rglob`, filter by suffix, and collect file sizes without `os.walk`.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| ---------------- | ------------------------------- |
| `Path` | Cross-platform paths |
| `rglob` | Recursive glob |
| `is_file()` | Skip directories / broken links |
| `stat().st_size` | Byte size |
## π‘ Examples [#-examples]
**pathlib\_walk.py:**
```python
from __future__ import annotations
from pathlib import Path
def list_py_files(root: Path) -> list[tuple[Path, int]]:
root = root.resolve()
results: list[tuple[Path, int]] = []
for path in root.rglob("*.py"):
if not path.is_file():
continue
results.append((path.relative_to(root), path.stat().st_size))
return sorted(results, key=lambda t: str(t[0]))
def main() -> None:
root = Path(".")
rows = list_py_files(root)
total = sum(size for _, size in rows)
for rel, size in rows:
print(f"{size:>8} {rel}")
print(f"\n{len(rows)} files, {total} bytes")
if __name__ == "__main__":
main()
```
**Exclude folders:**
```python
SKIP = {".git", ".venv", "node_modules", "__pycache__"}
def list_py_files_filtered(root: Path) -> list[Path]:
out: list[Path] = []
for path in root.rglob("*.py"):
if any(part in SKIP for part in path.parts):
continue
if path.is_file():
out.append(path)
return out
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `rglob` follows symlinks on some platforms β guard against cycles if needed.
* Calling `stat()` on every file can be slow on huge trees; batch or cache.
* Relative vs absolute: resolve once at the start for stable `relative_to`.
## π Related [#-related]
* [Manage files](/docs/python/examples/manage-files)
* [../pathlib.md](/docs/python/pathlib)
* [../files\_io.md](/docs/python/files-io)
---
# Pytest Example (/docs/python/examples/pytest-example)
# Pytest Example [#pytest-example]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
A minimal pytest layout: functions under test, fixtures, parametrize, and `tmp_path`. Run with `python -m pytest -q`.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| ------------- | ------------------- |
| `test_*.py` | Discovery |
| Fixtures | Reusable setup |
| `parametrize` | Many inputs |
| `tmp_path` | Isolated filesystem |
| `capsys` | Capture stdout |
| `raises` | Expected errors |
Keep tests independent; name them after behavior, not implementation.
## π‘ Examples [#-examples]
**mylib.py:**
```python
from pathlib import Path
def normalize(name: str) -> str:
cleaned = name.strip().lower()
if not cleaned:
raise ValueError("empty name")
return cleaned
def save_greeting(path: Path, name: str) -> None:
path.write_text(f"Hello, {normalize(name)}!\n", encoding="utf-8")
```
**test\_mylib.py:**
```python
import pytest
from pathlib import Path
from mylib import normalize, save_greeting
@pytest.mark.parametrize(
"raw,expected",
[(" Ada ", "ada"), ("BOB", "bob")],
)
def test_normalize(raw: str, expected: str) -> None:
assert normalize(raw) == expected
def test_normalize_empty() -> None:
with pytest.raises(ValueError, match="empty"):
normalize(" ")
def test_save_greeting(tmp_path: Path) -> None:
out = tmp_path / "hi.txt"
save_greeting(out, " Ada ")
assert out.read_text(encoding="utf-8") == "Hello, ada!\n"
```
**CLI test with capsys:**
```python
from cli_tool_stub import main # your main(argv) -> int
def test_cli(capsys) -> None:
assert main(["Ada"]) == 0
assert "Hello, Ada!" in capsys.readouterr().out
```
**Run:**
```shellscript
python -m pip install pytest
python -m pytest -q
python -m pytest -k normalize -vv
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Import path issues β use `src` layout or install editable (`pip install -e .`).
* Tests depending on wall-clock time without freezing.
* Writing into the repo instead of `tmp_path`.
* Over-mocking until tests assert nothing real.
* Ignoring warnings that signal API misuse.
## π Related [#-related]
* [CLI tool](/docs/python/examples/cli-tool)
* [Dataclass pipeline](/docs/python/examples/dataclass-pipeline)
* [../testing\_pytest.md](/docs/python/testing-pytest)
* [../exceptions.md](/docs/python/exceptions)
* [../pathlib.md](/docs/python/pathlib)
* [../functions.md](/docs/python/functions)
---
# Retry with Backoff (/docs/python/examples/retry-backoff)
# Retry with Backoff [#retry-with-backoff]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Transient failures (network blips, 429/503) often succeed on retry. Use exponential backoff with jitter. Prefer a small helper or `tenacity` over ad-hoc loops copied everywhere.
## π§ Core concepts [#-core-concepts]
| Idea | Detail |
| ------------------- | ----------------------------------------- |
| Retryable | Timeouts, connection errors, 429/5xx |
| Non-retryable | 400/401/403/404 (usually) |
| Exponential backoff | `delay * 2 ** attempt` |
| Jitter | Randomize delay to avoid thundering herds |
| Cap | Max attempts + max delay |
| Idempotency | Safe to retry GET; careful with POST |
## π‘ Examples [#-examples]
**Manual helper:**
```python
from __future__ import annotations
import random
import time
from collections.abc import Callable
from typing import TypeVar
import requests
T = TypeVar("T")
def with_backoff(
fn: Callable[[], T],
*,
attempts: int = 5,
base: float = 0.5,
max_delay: float = 8.0,
) -> T:
last: Exception | None = None
for i in range(attempts):
try:
return fn()
except (requests.Timeout, requests.ConnectionError) as e:
last = e
except requests.HTTPError as e:
status = e.response.status_code if e.response is not None else 0
if status not in {429, 500, 502, 503, 504}:
raise
last = e
delay = min(max_delay, base * (2**i))
delay *= 0.5 + random.random() # jitter
time.sleep(delay)
assert last is not None
raise last
def fetch(url: str) -> dict:
def _call() -> dict:
r = requests.get(url, timeout=10)
r.raise_for_status()
return r.json()
return with_backoff(_call)
```
**tenacity sketch:**
```python
from tenacity import retry, stop_after_attempt, wait_exponential_jitter
import requests
@retry(stop=stop_after_attempt(5), wait=wait_exponential_jitter(initial=0.5, max=8))
def get_json(url: str) -> dict:
r = requests.get(url, timeout=10)
r.raise_for_status()
return r.json()
```
**Async note:** use `asyncio.sleep` in async helpers β see [../async.md](/docs/python/async).
## β οΈ Pitfalls [#οΈ-pitfalls]
* Retrying non-idempotent POSTs can duplicate side effects.
* No jitter β synchronized retries amplify outages.
* Swallowing all exceptions hides permanent bugs.
* Ignoring `Retry-After` headers on 429.
* Busy-loops without sleep burn CPU and get banned.
## π Related [#-related]
* [HTTP requests](/docs/python/examples/http-requests)
* [Scrape HTML](/docs/python/examples/scrape-html)
* [../exceptions.md](/docs/python/exceptions)
* [../concurrency.md](/docs/python/concurrency)
* [../logging.md](/docs/python/logging)
* [../datetime.md](/docs/python/datetime)
---
# Scrape HTML (/docs/python/examples/scrape-html)
# Scrape HTML [#scrape-html]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Fetch HTML over HTTP and parse it with Beautiful Soup (`bs4`). Respect robots.txt, rate limits, and site terms. Prefer official APIs when available.
## π§ Core concepts [#-core-concepts]
| Step | Tool |
| ------ | ---------------------------------------------- |
| Fetch | `requests` + timeout |
| Parse | `BeautifulSoup(html, "html.parser")` or `lxml` |
| Select | CSS via `select` / `select_one` |
| Text | `.get_text(strip=True)` |
| Attrs | `tag["href"]`, `.get("href")` |
Install: `python -m pip install requests beautifulsoup4`. Optional: `lxml` for speed.
## π‘ Examples [#-examples]
**Extract links:**
```python
import requests
from bs4 import BeautifulSoup
def extract_links(url: str) -> list[str]:
resp = requests.get(url, timeout=15, headers={"User-Agent": "docs-bot/1.0"})
resp.raise_for_status()
soup = BeautifulSoup(resp.text, "html.parser")
links: list[str] = []
for a in soup.select("a[href]"):
href = a.get("href")
if href:
links.append(href)
return links
if __name__ == "__main__":
print(extract_links("https://example.com")[:5])
```
**Table-ish content:**
```python
from bs4 import BeautifulSoup
html = """
"""
soup = BeautifulSoup(html, "html.parser")
people = [
{"id": li["data-id"], "name": li.get_text(strip=True)}
for li in soup.select("ul.items li")
]
print(people)
```
**Absolute URLs:**
```python
from urllib.parse import urljoin
import requests
from bs4 import BeautifulSoup
base = "https://example.com"
soup = BeautifulSoup(requests.get(base, timeout=15).text, "html.parser")
abs_links = [urljoin(base, a["href"]) for a in soup.select("a[href]")]
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Scraping JS-rendered pages needs a browser tool β BS4 only sees HTML.
* Fragile CSS selectors break when sites redesign.
* Aggressive crawling can get you blocked β throttle and cache.
* Encoding: use `resp.content` + apparent encoding when text looks wrong.
* Legal/ToS: check before scraping; don't bypass paywalls/auth.
## π Related [#-related]
* [HTTP requests](/docs/python/examples/http-requests)
* [Retry backoff](/docs/python/examples/retry-backoff)
* [../regex.md](/docs/python/regex)
* [../strings.md](/docs/python/strings)
* [../json.md](/docs/python/json)
* [../exceptions.md](/docs/python/exceptions)
---
# HTTP server (/docs/python/examples/server)
# HTTP server [#http-server]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Spin up a small HTTP server with the standard library for local tools, webhooks, and demos. For production APIs prefer FastAPI/Django/Flask with a proper ASGI/WSGI server. Below: `http.server` for static files and a tiny `ThreadingHTTPServer` JSON handler.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| ------------------------------------ | ------------------------------------------------------ |
| `HTTPServer` / `ThreadingHTTPServer` | Socket + request loop |
| `BaseHTTPRequestHandler` | `do_GET` / `do_POST` |
| Static files | `http.server` module CLI or `SimpleHTTPRequestHandler` |
| Bind | `0.0.0.0` for LAN; `127.0.0.1` for local only |
| Port | Check availability; avoid privileged \< 1024 |
## π‘ Examples [#-examples]
**Static file server (CLI):**
```shellscript
python -m http.server 9000 --bind 127.0.0.1
```
**Minimal JSON API:**
```python
import json
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
class Handler(BaseHTTPRequestHandler):
def _json(self, code: int, payload: dict) -> None:
body = json.dumps(payload).encode("utf-8")
self.send_response(code)
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_GET(self) -> None:
if self.path == "/health":
self._json(200, {"status": "ok"})
return
self._json(404, {"error": "not found"})
def do_POST(self) -> None:
if self.path != "/echo":
self._json(404, {"error": "not found"})
return
length = int(self.headers.get("Content-Length", "0"))
raw = self.rfile.read(length)
data = json.loads(raw.decode("utf-8") or "{}")
self._json(200, {"echo": data})
def log_message(self, fmt: str, *args) -> None:
print(f"{self.address_string()} - {fmt % args}")
def main() -> None:
host, port = "127.0.0.1", 9000
server = ThreadingHTTPServer((host, port), Handler)
print(f"listening on http://{host}:{port}")
server.serve_forever()
if __name__ == "__main__":
main()
```
**Async alternative sketch (aiohttp-style idea with asyncio):**
```python
# Prefer a real framework for async HTTP; stdlib has limited ASGI support.
# Example entry with FastAPI would be: uvicorn app:app --port 9000
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `http.server` is not hardened for public internet exposure.
* Blocking work in `do_*` stalls that worker threadβoffload heavy jobs.
* Forgetting `Content-Length` / correct headers breaks some clients.
* Binding `0.0.0.0` without auth exposes the service on the LAN.
## π Related [#-related]
* [Telegram bot](/docs/python/examples/tg-bot)
* [Encrypt](/docs/python/examples/encrypt)
* [Manage files](/docs/python/examples/manage-files)
* [../async.md](/docs/python/async)
* [../import\_export.md](/docs/python/import-export)
* [../f-string.md](/docs/python/f-string)
---
# SQLite (/docs/python/examples/sqlite)
# SQLite [#sqlite]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
`sqlite3` is a zero-config SQL database in the stdlib. Great for local apps, caches, and prototypes. Use parameterized queries always; never format SQL with f-strings and user input.
## π§ Core concepts [#-core-concepts]
| Task | API |
| ------------ | ------------------------------- |
| Connect | `sqlite3.connect(path)` |
| Cursor | `conn.execute` / `cursor` |
| Commit | `conn.commit()` or `with conn:` |
| Params | `?` placeholders |
| Rows | `sqlite3.Row` factory |
| Transactions | Context manager on connection |
`:memory:` creates an in-memory DB. WAL mode helps concurrent readers.
## π‘ Examples [#-examples]
**Create and query:**
```python
import sqlite3
from pathlib import Path
db = Path("app.db")
with sqlite3.connect(db) as conn:
conn.row_factory = sqlite3.Row
conn.execute(
"""
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL UNIQUE
)
"""
)
conn.execute("INSERT OR IGNORE INTO users (name) VALUES (?)", ("Ada",))
row = conn.execute(
"SELECT id, name FROM users WHERE name = ?", ("Ada",)
).fetchone()
print(dict(row) if row else None)
```
**Many inserts:**
```python
import sqlite3
rows = [("Ada",), ("Bob",), ("Cy",)]
with sqlite3.connect("app.db") as conn:
conn.executemany("INSERT OR IGNORE INTO users (name) VALUES (?)", rows)
```
**Context and cleanup:**
```python
import sqlite3
def list_names(path: str) -> list[str]:
with sqlite3.connect(path) as conn:
cur = conn.execute("SELECT name FROM users ORDER BY name")
return [r[0] for r in cur]
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* SQL injection: never interpolate user strings into SQL.
* Forgetting `commit` (use `with conn:` which auto-commits if no error).
* Sharing connections across threads β check `check_same_thread`.
* `LIKE` wildcards from users need escaping.
* Schema migrations: plan `ALTER` carefully; SQLite has limits.
## π Related [#-related]
* [CSV and Excel](/docs/python/examples/csv-excel)
* [Dataclass pipeline](/docs/python/examples/dataclass-pipeline)
* [CLI tool](/docs/python/examples/cli-tool)
* [../context\_managers.md](/docs/python/context-managers)
* [../exceptions.md](/docs/python/exceptions)
* [../pathlib.md](/docs/python/pathlib)
---
# Telegram bot (/docs/python/examples/tg-bot)
# Telegram bot [#telegram-bot]
*Python Β· Example / how-to*
***
## π Overview [#-overview]
Build a simple Telegram bot with long polling using `httpx`/`requests` against Bot API, or the popular `python-telegram-bot` library. Store the bot token in an environment variable. Start with echo/commands; add webhooks when you deploy behind HTTPS.
## π§ Core concepts [#-core-concepts]
| Piece | Detail |
| ----------- | --------------------------------------------------------------- |
| Token | From [@BotFather](https://t.me/BotFather); `TELEGRAM_BOT_TOKEN` |
| getUpdates | Long polling for local/dev |
| sendMessage | Reply to `chat.id` |
| Commands | Messages starting with `/` |
| Webhook | Production: HTTPS endpoint receives updates |
| Library | `pip install python-telegram-bot` |
## π‘ Examples [#-examples]
**Minimal echo with `python-telegram-bot` (v21+ async):**
```python
import os
from telegram import Update
from telegram.ext import Application, CommandHandler, ContextTypes, MessageHandler, filters
async def start(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
assert update.effective_message
await update.effective_message.reply_text("Hi! Send me any text.")
async def echo(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
assert update.effective_message and update.effective_message.text
await update.effective_message.reply_text(update.effective_message.text)
def main() -> None:
token = os.environ["TELEGRAM_BOT_TOKEN"]
app = Application.builder().token(token).build()
app.add_handler(CommandHandler("start", start))
app.add_handler(MessageHandler(filters.TEXT & ~filters.COMMAND, echo))
app.run_polling(allowed_updates=Update.ALL_TYPES)
if __name__ == "__main__":
main()
```
**Raw Bot API long poll (stdlib + urllib):**
```python
import json
import os
import urllib.parse
import urllib.request
TOKEN = os.environ["TELEGRAM_BOT_TOKEN"]
API = f"https://api.telegram.org/bot{TOKEN}"
def call(method: str, **params):
data = urllib.parse.urlencode(params).encode()
with urllib.request.urlopen(f"{API}/{method}", data=data, timeout=60) as resp:
return json.loads(resp.read().decode())
def main() -> None:
offset = 0
while True:
payload = call("getUpdates", offset=offset, timeout=30)
for upd in payload.get("result", []):
offset = upd["update_id"] + 1
msg = upd.get("message") or {}
text = msg.get("text")
chat = msg.get("chat") or {}
chat_id = chat.get("id")
if text and chat_id:
call("sendMessage", chat_id=chat_id, text=f"echo: {text}")
if __name__ == "__main__":
main()
```
**Run:**
```shellscript
set TELEGRAM_BOT_TOKEN=123456:ABC...
python bot.py
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Committing tokens to gitβrotate immediately if leaked.
* Two pollers with the same token fight over updates.
* Network timeouts: use reasonable `timeout` and retry with backoff.
* Webhooks require valid TLS; local dev usually uses polling.
## π Related [#-related]
* [Server](/docs/python/examples/server)
* [Encrypt](/docs/python/examples/encrypt)
* [Manage files](/docs/python/examples/manage-files)
* [../async.md](/docs/python/async)
* [../import\_export.md](/docs/python/import-export)
* [../kwags.md](/docs/python/kwags)
---
# Exceptions (/docs/python/exceptions)
# Exceptions [#exceptions]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Exceptions signal error conditions. Use `try` / `except` / `else` / `finally`, raise with `raise`, and chain with `raise ... from`. Prefer built-in types or small custom hierarchies over bare `except:`.
## π§ Core concepts [#-core-concepts]
| Construct | Role |
| ----------------- | ------------------------ |
| `try` | Guarded block |
| `except Exc as e` | Handle specific type(s) |
| `except* ` | Exception groups (3.11+) |
| `else` | Runs if no exception |
| `finally` | Always runs (cleanup) |
| `raise` | Raise / re-raise |
| `raise X from e` | Explicit chaining |
| `ExceptionGroup` | Multiple errors (3.11+) |
Catch narrow types first. `BaseException` includes `KeyboardInterrupt` / `SystemExit` β usually do not catch those.
## π‘ Examples [#-examples]
**Try / except / else / finally:**
```python
def load(path: str) -> str:
try:
with open(path, encoding="utf-8") as f:
data = f.read()
except FileNotFoundError:
return ""
except OSError as e:
raise RuntimeError(f"read failed: {path}") from e
else:
return data.strip()
finally:
pass # close handled by `with`
```
**Custom exception:**
```python
class AppError(Exception):
"""Base app error."""
class NotFoundError(AppError):
def __init__(self, entity: str, key: object) -> None:
super().__init__(f"{entity} not found: {key!r}")
self.entity = entity
self.key = key
raise NotFoundError("user", 42)
```
**Exception groups (3.11+):**
```python
def run_all() -> None:
errors: list[Exception] = []
for i in (0, 1):
try:
if i == 0:
raise ValueError("bad")
raise TypeError("wrong")
except Exception as e:
errors.append(e)
if errors:
raise ExceptionGroup("batch failed", errors)
try:
run_all()
except* ValueError as eg:
print("values", eg.exceptions)
except* TypeError as eg:
print("types", eg.exceptions)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Bare `except:` or `except Exception` can hide bugs β be specific.
* Catching and ignoring without logging loses diagnostics.
* `finally` with `return` can swallow exceptions β avoid returning there.
* Use `raise ... from e` (or `from None`) intentionally for traceback clarity.
* Validating with exceptions in hot loops can be slower than checks.
## π Related [#-related]
* [Context managers](/docs/python/context-managers)
* [Classes](/docs/python/classes)
* [Logging](/docs/python/logging)
* [Files I/O](/docs/python/files-io)
* [Testing with pytest](/docs/python/testing-pytest)
* [Conditionals](/docs/python/conditionals)
---
# F-strings (/docs/python/f-string)
# F-strings [#f-strings]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Formatted string literals (`f"..."` / `f'...'`) embed expressions inside `\{\}`. They are the preferred way to build strings in modern Python (3.6+). Use them for logging messages, paths, and readable interpolationβavoid for SQL/HTML without escaping.
## π§ Core concepts [#-core-concepts]
| Feature | Syntax |
| ------------- | ---------------------------------------- |
| Basic | `f"Hello, \{name\}"` |
| Expression | `f"\{x + 1\}"`, `f"\{obj.attr\}"` |
| Format spec | `f"\{value:.2f\}"`, `f"\{n:04d\}"` |
| Debug (3.8+) | `f"\{x=\}"` β `x=42` |
| Conversion | `!s` str, `!r` repr, `!a` ascii |
| Nested quotes | Prefer different quote styles or escapes |
| Multiline | `f"""..."""` |
Format mini-language: alignment (`<^>`), width, precision, types (`d`, `f`, `x`, `%`).
## π‘ Examples [#-examples]
**Formatting numbers and dates:**
```python
from datetime import datetime
price = 19.5
n = 42
now = datetime(2026, 7, 10, 13, 30)
print(f"${price:.2f}") # $19.50
print(f"{n:04d}") # 0042
print(f"{now:%Y-%m-%d %H:%M}")
```
**Debug and repr:**
```python
user = "Ada"
print(f"{user=}") # user='Ada'
print(f"{user!r}") # 'Ada'
```
**Alignment and tables:**
```python
rows = [("id", "name"), (1, "Ada"), (2, "Bob")]
for a, b in rows:
print(f"{a!s:>4} | {b:<10}")
```
**Walrus in f-string (careful):**
```python
data = [1, 2, 3]
print(f"{(n := len(data))} items") # 3 items
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Backslashes inside `\{\}` expressions are limited; compute outside the braces if needed.
* Nested f-strings and complex expressions hurt readabilityβextract variables.
* Never build SQL/shell commands with raw f-strings; use parameterized APIs.
* Mixing `str.format` / `%` / f-strings inconsistently makes style noisy.
* Lazy logging: prefer `logger.info("x=%s", x)` over `f"..."` if the message may be skipped (minor; clarity often wins).
## π Related [#-related]
* [Strings](/docs/python/strings)
* [Types](/docs/python/types)
* [Docstrings](/docs/python/docstrings)
* [Dictionaries](/docs/python/dictionaries)
* [Loops](/docs/python/loops)
---
# Files I/O (/docs/python/files-io)
# Files I/O [#files-io]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
File I/O uses `open()` / `Path.open()` with modes for text or binary. Always use a context manager. Prefer `pathlib` helpers for simple whole-file reads/writes; use streaming for large files.
## π§ Core concepts [#-core-concepts]
| Mode | Meaning |
| --------------------- | ------------------------------------- |
| `"r"` / `"w"` / `"a"` | Read / write truncate / append (text) |
| `"rb"` / `"wb"` | Binary |
| `"x"` | Exclusive create |
| `"r+"` | Read + write |
| `encoding=` | Text mode charset (use `utf-8`) |
| `newline=` | Universal newlines control |
| `buffered` | Default buffering; `flush` / `fsync` |
Text mode yields `str`; binary yields `bytes`. Line iteration is memory-friendly.
## π‘ Examples [#-examples]
**Text read/write:**
```python
from pathlib import Path
path = Path("log.txt")
with path.open("w", encoding="utf-8") as f:
f.write("hello\n")
f.writelines(["line2\n", "line3\n"])
with path.open("r", encoding="utf-8") as f:
for line in f:
print(line.rstrip())
```
**Binary and buffered copy:**
```python
from pathlib import Path
src, dst = Path("in.bin"), Path("out.bin")
with src.open("rb") as r, dst.open("wb") as w:
while chunk := r.read(1024 * 64):
w.write(chunk)
```
**CSV-ish streaming:**
```python
from pathlib import Path
def count_rows(path: Path) -> int:
with path.open(encoding="utf-8") as f:
return sum(1 for _ in f)
print(count_rows(Path("data.csv")))
```
**Temporary files:**
```python
import tempfile
from pathlib import Path
with tempfile.TemporaryDirectory() as td:
p = Path(td) / "x.txt"
p.write_text("scratch", encoding="utf-8")
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Forgetting `with` leaks file handles.
* Writing text without `encoding` uses locale-dependent defaults.
* `"w"` truncates immediately on open β easy data loss.
* Mixing str/bytes in the wrong mode raises `TypeError`.
* On Windows, text mode translates newlines β use binary for exact bytes.
## π Related [#-related]
* [Pathlib](/docs/python/pathlib)
* [Context managers](/docs/python/context-managers)
* [Bytes / bytearray](/docs/python/bytes-bytearray)
* [JSON](/docs/python/json)
* [Examples: manage files](/docs/python/examples/manage-files)
* [Walrus](/docs/python/walrus)
---
# Functions (/docs/python/functions)
# Functions [#functions]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Functions are first-class callables defined with `def`. They encapsulate behavior, take parameters, and optionally return a value (`None` if omitted). Prefer small, typed functions with clear names over deep nesting.
## π§ Core concepts [#-core-concepts]
| Topic | Notes |
| -------------------- | ---------------------------------------------------------- |
| Definition | `def name(params) -> ReturnType:` |
| Positional / keyword | `f(1, b=2)` |
| Defaults | Evaluated once at definition time |
| `*args` / `**kwargs` | Variable positional / keyword |
| Keyword-only | After `*` : `def f(a, *, b)` |
| Positional-only | Before `/` (3.8+): `def f(a, /, b)` |
| Annotations | Hints for tools; not enforced at runtime |
| Docstrings | First statement; see [docstrings](/docs/python/docstrings) |
Callables include functions, methods, lambdas, and objects with `__call__`.
## π‘ Examples [#-examples]
**Typed function with keyword-only args:**
```python
def greet(name: str, /, *, shout: bool = False) -> str:
msg = f"Hello, {name}"
return msg.upper() if shout else msg
print(greet("Ada"))
print(greet("Ada", shout=True))
```
**Defaults and mutable trap fix:**
```python
def append_item(item: str, bucket: list[str] | None = None) -> list[str]:
if bucket is None:
bucket = []
bucket.append(item)
return bucket
```
**Early return and multiple values:**
```python
def parse_int(text: str) -> int | None:
if not text.strip():
return None
try:
return int(text)
except ValueError:
return None
code, label = (200, "ok") # tuple return pattern
```
**Higher-order:**
```python
from collections.abc import Callable
def twice(fn: Callable[[int], int], x: int) -> int:
return fn(fn(x))
print(twice(lambda n: n + 1, 3)) # 5
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Mutable default arguments (`def f(xs=[])`) share one list across calls.
* Forgetting `return` yields `None` β easy bug in conditionals.
* Annotations are not runtime checks unless you add a validator.
* Closures capture variables by name β see [closures](/docs/python/closures).
* Excessive `*args/**kwargs` hides the real API; prefer explicit params.
## π Related [#-related]
* [\*args and \*\*kwargs](/docs/python/kwags)
* [Lambda](/docs/python/lambda)
* [Decorators](/docs/python/decorators)
* [Closures](/docs/python/closures)
* [Typing hints](/docs/python/typing-hints)
* [Scope / namespaces](/docs/python/scope-namespaces)
---
# Functools (/docs/python/functools)
# Functools [#functools]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`functools` provides higher-order helpers: freeze arguments with `partial`, memoize with `lru_cache`, fold with `reduce`, and preserve metadata with `wraps` when writing decorators.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ------------------------------ | --------------------------------------- |
| `partial(fn, *a, **kw)` | Pre-fill args β new callable |
| `lru_cache(maxsize=128)` | Memoize pure functions |
| `cache` | Unbounded memoize (3.9+) |
| `reduce(fn, iterable[, init])` | Fold sequence to one value |
| `wraps(fn)` | Copy `__name__`/`__doc__` onto wrappers |
| `singledispatch` | Function overloading by type |
| `total_ordering` | Fill rich comparisons from a few |
## π‘ Examples [#-examples]
**partial:**
```python
from functools import partial
def power(base, exp):
return base**exp
square = partial(power, exp=2)
cube = partial(power, exp=3)
print(square(5), cube(3)) # 25 27
```
**lru\_cache:**
```python
from functools import lru_cache
@lru_cache(maxsize=256)
def fib(n: int) -> int:
if n < 2:
return n
return fib(n - 1) + fib(n - 2)
print(fib(30), fib.cache_info())
fib.cache_clear()
```
**reduce:**
```python
from functools import reduce
import operator
nums = [1, 2, 3, 4]
print(reduce(operator.mul, nums, 1)) # 24
print(reduce(lambda a, b: a + b, nums)) # 10
```
**wraps for decorators:**
```python
from functools import wraps
def trace(fn):
@wraps(fn)
def wrapper(*args, **kwargs):
print("call", fn.__name__)
return fn(*args, **kwargs)
return wrapper
@trace
def greet(name):
"""Say hi."""
return f"hi {name}"
print(greet.__name__, greet.__doc__) # greet, Say hi.
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Cache only pure functions: same args β same result; no hidden I/O/mutation.
* Args to `@lru_cache` must be hashable (no lists/dicts).
* Unbounded `@cache` can grow forever β prefer `lru_cache(maxsize=...)`.
* Without `@wraps`, stack traces and docs point at `wrapper`.
* Prefer comprehensions / explicit loops over `reduce` when clarity matters.
## π Related [#-related]
* [Decorators](/docs/python/decorators)
* [Lambda](/docs/python/lambda)
* [Functions](/docs/python/functions)
* [Closures](/docs/python/closures)
* [Itertools](/docs/python/itertools)
---
# Generators (/docs/python/generators)
# Generators [#generators]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Generators produce values lazily with `yield`. They pause and resume, saving memory for large or infinite streams. Generator functions return generator iterators; generator expressions use `(...)`.
## π§ Core concepts [#-core-concepts]
| Concept | Detail |
| ------------------ | -------------------------------- |
| `yield` | Produce a value; suspend frame |
| `yield from` | Delegate to a sub-iterator |
| `return` | Sets `StopIteration.value` |
| `.send` / `.throw` | Advanced coroutine-style control |
| Gen exp | `(x for x in it)` |
| Exhaustion | One-shot; recreate to reuse |
Generators implement the iterator protocol (`__iter__`, `__next__`).
## π‘ Examples [#-examples]
**Basic generator:**
```python
from collections.abc import Iterator
def countdown(n: int) -> Iterator[int]:
while n > 0:
yield n
n -= 1
print(list(countdown(3))) # [3, 2, 1]
```
**yield from and piping:**
```python
def chain_iters(*iters: Iterator[int]) -> Iterator[int]:
for it in iters:
yield from it
print(list(chain_iters(iter([1, 2]), iter([3]))))
```
**Infinite stream (take n):**
```python
from collections.abc import Iterator
from itertools import islice
def naturals() -> Iterator[int]:
n = 0
while True:
yield n
n += 1
print(list(islice(naturals(), 5))) # [0, 1, 2, 3, 4]
```
**Generator expression:**
```python
paths = ("a.txt", "b.txt", "c.txt")
sizes = (len(p) for p in paths)
print(sum(sizes))
# sizes is exhausted β sum(sizes) again is 0
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Generators are single-use; convert to `list` if you need multiple passes.
* Mixing heavy side effects with `yield` makes control flow hard to follow.
* Forgetting to iterate means the body never runs (lazy).
* `return` inside a generator does not return to the caller as a normal value.
* Prefer async generators (`async def` + `yield`) for async streams β see [async](/docs/python/async).
## π Related [#-related]
* [Iterators](/docs/python/iterators)
* [Comprehensions](/docs/python/comprehensions)
* [Itertools](/docs/python/itertools)
* [Loops](/docs/python/loops)
* [Async / await](/docs/python/async)
* [Context managers](/docs/python/context-managers)
---
# Getting Started with Python (/docs/python/getting-started)
# Getting Started with Python [#getting-started-with-python]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Python is a readable, general-purpose language used for scripting, web backends, data work, and automation. You write `.py` files (or use the interactive REPL) and run them with the `python` (or `python3`) interpreter.
## π§ Core concepts [#-core-concepts]
| Idea | Meaning |
| ----------- | ---------------------------------------------- |
| Interpreter | Runs your code line by line |
| REPL | Interactive prompt (`>>>`) for trying snippets |
| Script | A `.py` file you run from the terminal |
| Module | A reusable `.py` file you can `import` |
| Package | A folder of modules (often with `__init__.py`) |
**Install check:** open a terminal and run `python --version` (or `python3 --version`). Use a virtual environment (`venv`) so project packages stay isolated.
**Typical first workflow:**
1. Install Python 3 from [python.org](https://www.python.org/) (or your OS package manager).
2. Create a folder for practice files.
3. Write `hello.py`, then run `python hello.py`.
## π‘ Examples [#-examples]
**Check the version:**
```shellscript
python --version
# Python 3.12.x
```
**Start the REPL:**
```shellscript
python
```
```python
>>> 2 + 2
4
>>> print("hi")
hi
>>> exit()
```
**Create and run a script:**
```python
# hello.py
print("Hello from a file!")
```
```shellscript
python hello.py
```
**Create a virtual environment (recommended):**
```shellscript
python -m venv .venv
# Windows PowerShell:
.\.venv\Scripts\Activate.ps1
# macOS/Linux:
source .venv/bin/activate
pip install requests
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* On some systems `python` is missing and you must use `python3`.
* Mixing system Python and project packages without `venv` causes version conflicts.
* Indentation is syntax in Python β tabs vs spaces can break files.
* Saving as `.txt` or wrong encoding can confuse beginners; use UTF-8 `.py` files.
## π Related [#-related]
* [hello\_world.md](/docs/python/hello-world)
* [variables.md](/docs/python/variables)
* [comments.md](/docs/python/comments)
* [indentation.md](/docs/python/indentation)
* [venv.md](/docs/python/venv) Β· [pip.md](/docs/python/pip)
---
# Glob (/docs/python/glob-module)
# Glob [#glob]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
The `glob` module expands Unix-style filename patterns into matching paths. Use `glob.glob` for a list, `glob.iglob` for a lazy iterator. For OO APIs, prefer `pathlib.Path.glob` / `.rglob`.
## π§ Core concepts [#-core-concepts]
| API | Role |
| -------------------------- | --------------------------------- |
| `glob.glob(pattern)` | List of matching path strings |
| `glob.iglob(pattern)` | Iterator (memory-friendly) |
| `*` | Any chars in one segment |
| `?` | Single char |
| `**` | Recursive dirs (`recursive=True`) |
| `[abc]` / `[0-9]` | Character class |
| `glob.escape(s)` | Escape literal `*?[` in names |
| `Path.glob` / `Path.rglob` | pathlib equivalents |
Patterns are relative to the current working directory unless absolute.
## π‘ Examples [#-examples]
**Basic glob:**
```python
import glob
py_files = glob.glob("src/*.py")
print(sorted(py_files))
```
\*\*Recursive **:**
```python
import glob
for path in glob.iglob("**/*.md", recursive=True):
print(path)
```
**Character classes and escape:**
```python
import glob
# match report1.csv β¦ report9.csv
print(glob.glob("report[1-9].csv"))
needle = "file[1].txt"
print(glob.glob(glob.escape(needle))) # literal brackets
```
**pathlib alternative:**
```python
from pathlib import Path
root = Path("src")
for p in sorted(root.rglob("*.py")):
if p.is_file():
print(p.relative_to(root))
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `**` only recurses when `recursive=True` (stdlib `glob`).
* Results are unordered β sort if you need stable output.
* Hidden files (`.` prefix) may be skipped depending on pattern/platform.
* Huge trees: prefer `iglob` / pathlib over building giant lists.
* Patterns follow cwd β anchor with absolute paths when scripts `chdir`.
## π Related [#-related]
* [Pathlib](/docs/python/pathlib)
* [Files I/O](/docs/python/files-io)
* [Os](/docs/python/os)
* [Shutil](/docs/python/shutil)
* [Regex](/docs/python/regex)
---
# Glossary (/docs/python/glossary)
# Glossary [#glossary]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Alphabetical glossary of core Python terms covering types, syntax, OOP, packaging, and runtime concepts you meet daily.
## π§ Core concepts [#-core-concepts]
| Term | Definition |
| ------------------- | -------------------------------------------------------------------------------------- |
| Annotation | Optional type hint attached to a variable, parameter, or return value. |
| Argument | A value passed into a function when it is called. |
| Bytecode | Intermediate instructions CPython compiles source into before the interpreter runs it. |
| Class | A blueprint that defines attributes and methods for creating objects. |
| Closure | A nested function that remembers variables from its enclosing scope. |
| Comprehension | Compact syntax that builds a list, set, or dict from an iterable expression. |
| Context manager | An object usable with `with` that sets up and tears down a resource. |
| Coroutine | An `async def` function that can pause and resume with `await`. |
| Decorator | A callable that wraps another function or class to modify behavior. |
| Descriptor | An object defining `__get__`/`__set__`/`__delete__` that customizes attribute access. |
| Dict | A mutable mapping of unique hashable keys to values. |
| Dunder | A βdouble underscoreβ special method or attribute such as `__init__`. |
| Exception | An error object raised to interrupt normal control flow. |
| Generator | A function using `yield` that produces values lazily as an iterator. |
| GIL | Global Interpreter Lock that allows only one CPython bytecode thread at a time. |
| Immutable | A value that cannot change after creation, such as `str`, `tuple`, or `frozenset`. |
| Import | The mechanism that loads a module or package into the current namespace. |
| Iterator | An object with `__next__` that yields items until `StopIteration`. |
| Lambda | A small anonymous function expressed as `lambda args: expression`. |
| List | An ordered, mutable sequence of elements. |
| Method | A function defined on a class and bound to an instance when accessed. |
| Module | A single `.py` file that can be imported as a namespace. |
| Mutable | An object whose contents can change in place, such as `list` or `dict`. |
| Namespace | A mapping from names to objects, such as locals, globals, or a module dict. |
| Package | A directory of modules, typically with `__init__.py`, importable as a unit. |
| Parameter | A named slot in a function definition that receives an argument. |
| Property | A managed attribute exposed via getter/setter methods using `@property`. |
| Set | An unordered collection of unique hashable elements. |
| Slice | A range of indices written as `start:stop:step` used to extract subsequences. |
| Statement | A complete instruction that performs an action, such as assignment or `return`. |
| Tuple | An ordered, immutable sequence often used for fixed groupings. |
| Virtual environment | An isolated Python install directory for project-specific packages. |
| Walrus operator | The `:=` operator that assigns a value inside an expression. |
## π‘ Examples [#-examples]
**Comprehension and slice:**
```python
nums = [1, 2, 3, 4, 5]
evens = [n for n in nums if n % 2 == 0]
print(nums[1:4]) # [2, 3, 4]
```
**Decorator and context manager:**
```python
from functools import wraps
from pathlib import Path
def once(fn):
@wraps(fn)
def wrapper(*args, **kwargs):
return fn(*args, **kwargs)
return wrapper
with Path("notes.txt").open() as f:
text = f.read()
```
**Walrus operator:**
```python
while (line := input(">> ")) != "quit":
print(line.upper())
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Confusing **parameter** (definition) with **argument** (call-site value).
* Mixing **list** (mutable) with **tuple** (immutable) when APIs expect one or the other.
* Treating **iterator** and **iterable** as the same β an iterable can be re-looped; many iterators cannot.
* Assuming **annotation** enforces types at runtime β they are hints unless a checker or validator runs.
* Equating **module** and **package** β a package is a collection of modules.
## π Related [#-related]
* [lists](/docs/python/lists)
* [dictionaries](/docs/python/dictionaries)
* [decorators](/docs/python/decorators)
* [generators](/docs/python/generators)
* [exceptions](/docs/python/exceptions)
* [venv](/docs/python/venv)
---
# Hashlib (/docs/python/hashlib)
# Hashlib [#hashlib]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`hashlib` computes cryptographic digests (SHA-256, SHA-512, BLAKE2, β¦). Use it for integrity checks and content addressing β not for password storage (use `hashlib.scrypt` / dedicated libs with salt).
## π§ Core concepts [#-core-concepts]
| API | Role |
| ---------------------------------- | ---------------------------------------- |
| `hashlib.sha256(data)` | SHA-256 hasher (or empty then `.update`) |
| `h.update(chunk)` | Feed more bytes |
| `h.digest()` | Raw bytes |
| `h.hexdigest()` | Hex string |
| `hashlib.file_digest(f, "sha256")` | Hash open file (3.11+) |
| `hashlib.algorithms_guaranteed` | Always-available algos |
| `hashlib.blake2b` / `sha3_256` | Other strong hashes |
Always hash **bytes**, not `str` β encode text first.
## π‘ Examples [#-examples]
**String / bytes:**
```python
import hashlib
msg = b"hello"
print(hashlib.sha256(msg).hexdigest())
h = hashlib.sha256()
h.update(b"hel")
h.update(b"lo")
print(h.hexdigest()) # same as above
```
**File hashing (streaming):**
```python
import hashlib
from pathlib import Path
def sha256_file(path: Path, chunk=1024 * 1024) -> str:
h = hashlib.sha256()
with path.open("rb") as f:
while True:
block = f.read(chunk)
if not block:
break
h.update(block)
return h.hexdigest()
print(sha256_file(Path("archive.zip")))
```
**Python 3.11+ file\_digest:**
```python
import hashlib
from pathlib import Path
with Path("archive.zip").open("rb") as f:
digest = hashlib.file_digest(f, "sha256")
print(digest.hexdigest())
```
**Compare digests safely:**
```python
import hashlib
import hmac
expected = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"
actual = hashlib.sha256(b"hello").hexdigest()
print(hmac.compare_digest(expected, actual))
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Donβt use MD5/SHA-1 for security β fine for non-crypto checksums only.
* Donβt hash passwords with plain SHA-256; use salted KDFs (`scrypt`, Argon2).
* Reading whole files into memory is wasteful β stream with `.update`.
* Hex vs bytes: APIs often expect hex strings; keep types consistent.
* Use `hmac.compare_digest` to avoid timing leaks when verifying.
## π Related [#-related]
* [Files I/O](/docs/python/files-io)
* [Pathlib](/docs/python/pathlib)
* [Bytes / bytearray](/docs/python/bytes-bytearray)
* [Tempfile](/docs/python/tempfile)
---
# Hello World (/docs/python/hello-world)
# Hello World [#hello-world]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
A βHello Worldβ program is the smallest useful program: it prints a message so you know the language toolchain works. In Python that is usually one `print` call.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| ------------ | ------------------------------------------------- |
| `print(...)` | Sends text to standard output (the terminal) |
| String | Text in quotes: `"Hello"` or `'Hello'` |
| Script entry | Top-level code runs when you execute the file |
| `__name__` | Equals `"__main__"` when the file is run directly |
You can run code in the REPL or save it in a `.py` file. Files are better for anything you want to keep.
## π‘ Examples [#-examples]
**One-liner in the REPL:**
```python
print("Hello, World!")
```
**Minimal script (`hello.py`):**
```python
print("Hello, World!")
```
```shellscript
python hello.py
```
**With a main guard (good habit):**
```python
def main():
print("Hello, World!")
if __name__ == "__main__":
main()
```
**Slightly richer greeting:**
```python
name = "Ada"
print(f"Hello, {name}!")
print("Welcome to Python.")
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Forgetting quotes: `print(Hello)` looks for a variable named `Hello`.
* Using smart/curly quotes from a word processor β use straight `"` or `'`.
* Expecting `print` to return a value; it returns `None`.
* On Windows, double-clicking a `.py` may flash a window and close; run from a terminal instead.
## π Related [#-related]
* [getting\_started.md](/docs/python/getting-started)
* [variables.md](/docs/python/variables)
* [input\_output\_basics.md](/docs/python/input-output-basics)
* [print\_input.md](/docs/python/print-input)
---
# Import and export (/docs/python/import-export)
# Import and export [#import-and-export]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Python modules are `.py` files (or packages with `__init__.py`). `import` loads and binds names; there is no separate `export` keywordβpublic API is whatever you define and optionally list in `__all__`. Prefer absolute imports; use relative imports inside packages.
## π§ Core concepts [#-core-concepts]
| Form | Meaning |
| ---------------------------- | ------------------------------------- |
| `import math` | Bind module name |
| `from math import sqrt` | Bind selected names |
| `from math import sqrt as s` | Alias |
| `from pkg import module` | Submodule |
| `from . import sibling` | Relative (package only) |
| `from .sibling import x` | Relative name |
| `__all__` | Names exported by `from pkg import *` |
| Entry point | `if __name__ == "__main__":` |
Search path: current package, `PYTHONPATH`, site-packages. Circular imports usually mean you should restructure or lazy-import inside functions.
## π‘ Examples [#-examples]
**Package layout:**
```plaintext
mypkg/
__init__.py
utils.py
api.py
```
```python
# mypkg/__init__.py
from .api import Client
__all__ = ["Client"]
```
```python
# mypkg/api.py
from .utils import normalize
class Client:
def get(self, key: str) -> str:
return normalize(key)
```
```python
# app.py
from mypkg import Client
# or: from mypkg.api import Client
```
**Selective import and alias:**
```python
import json
from pathlib import Path
from collections import defaultdict as dd
data = json.loads(Path("data.json").read_text(encoding="utf-8"))
```
**Script vs library:**
```python
# tools/cli.py
def main() -> None:
print("run")
if __name__ == "__main__":
main()
```
**Lazy import (avoid cycles / heavy deps):**
```python
def dump(obj: object) -> str:
import json # local import
return json.dumps(obj)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `from module import *` pollutes namespaces; avoid outside REPL/`__init__` re-exports.
* Shadowing stdlib names (`json.py`, `random.py`) breaks imports.
* Relative imports fail when a file is run as a script (`python pkg/mod.py`); use `-m`.
* Mutating `sys.path` in app code is fragileβfix packaging instead.
* Side effects at import time (DB connect, network) make testing and startup painful.
## π Related [#-related]
* [Types](/docs/python/types)
* [Docstrings](/docs/python/docstrings)
* [Async / await](/docs/python/async)
* [\*args and \*\*kwargs](/docs/python/kwags)
* [Examples: manage files](/docs/python/examples/manage-files)
---
# Indentation (/docs/python/indentation)
# Indentation [#indentation]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Python uses indentation (leading spaces) to define code blocks instead of braces `\{\}`. Consistent indentation is required syntax. The community standard is **4 spaces** per level (PEP 8).
## π§ Core concepts [#-core-concepts]
| Idea | Meaning |
| ------ | ------------------------------------------------------------------------ |
| Block | Indented suite under `if`, `for`, `while`, `def`, `class`, `with`, `try` |
| Indent | Move right to enter a block |
| Dedent | Move left to leave a block |
| Colon | Headers end with `:` then an indented body |
| Suite | One or more indented statements (or `pass`) |
Empty blocks need `pass`. Mixing tabs and spaces in one file causes `TabError` or confusing bugs.
## π‘ Examples [#-examples]
**if / else blocks:**
```python
score = 85
if score >= 90:
print("A")
elif score >= 80:
print("B")
else:
print("keep practicing")
```
**Nested indentation:**
```python
for n in range(1, 4):
print(f"n={n}")
if n % 2 == 0:
print(" even")
else:
print(" odd")
print("done") # back at top level
```
**Functions and `pass`:**
```python
def todo_later():
pass # placeholder body
def add(a, b):
total = a + b
return total
```
**Wrong vs right (conceptually):**
```python
# Wrong β body not indented (IndentationError)
# if True:
# print("hi")
# Right
if True:
print("hi")
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Mixing tabs and spaces β configure the editor to insert spaces.
* Copy-pasting from the web can bring inconsistent indent widths.
* Only indent when a block is required; random indent is a syntax error.
* `IndentationError` / `TabError` messages point near the problem line β check the line *above* too.
## π Related [#-related]
* [getting\_started.md](/docs/python/getting-started)
* [comments.md](/docs/python/comments)
* [boolean\_logic.md](/docs/python/boolean-logic)
* [conditionals](/docs/python/conditionals) (if present in your notes set)
---
# Inheritance (/docs/python/inheritance)
# Inheritance [#inheritance]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Inheritance reuses and specializes behavior via subclasses. Python uses C3 MRO for multiple inheritance. Prefer shallow hierarchies, explicit `super()`, and protocols/ABCs for shared interfaces.
## π§ Core concepts [#-core-concepts]
| Topic | Notes |
| --------------------------- | ---------------------------------- |
| Subclass | `class Child(Parent):` |
| Override | Redefine method in child |
| `super()` | Cooperative call to next MRO class |
| MRO | `Class.__mro__`, `Class.mro()` |
| `isinstance` / `issubclass` | Runtime type checks |
| Multiple inheritance | `class C(A, B):` β order matters |
| Mixin | Small reusable behavior class |
`super()` without args works inside methods (3.x); binds using the defining class and instance.
## π‘ Examples [#-examples]
**Single inheritance + super:**
```python
class Animal:
def __init__(self, name: str) -> None:
self.name = name
def speak(self) -> str:
return "..."
class Dog(Animal):
def __init__(self, name: str, breed: str) -> None:
super().__init__(name)
self.breed = breed
def speak(self) -> str:
return "woof"
```
**Mixin:**
```python
class JsonMixin:
def to_json(self) -> str:
import json
return json.dumps(self.__dict__)
class Point(JsonMixin):
def __init__(self, x: int, y: int) -> None:
self.x = x
self.y = y
print(Point(1, 2).to_json())
```
**MRO peek:**
```python
class A:
def ping(self) -> str:
return "A"
class B(A):
def ping(self) -> str:
return "B>" + super().ping()
class C(A):
def ping(self) -> str:
return "C>" + super().ping()
class D(B, C):
def ping(self) -> str:
return "D>" + super().ping()
print(D().ping()) # D>B>C>A
print(D.__mro__)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Diamond problems: always use `super()` consistently in cooperative hierarchies.
* Parent `__init__` not called β missing attributes; call `super().__init__`.
* Overusing inheritance for code reuse β prefer composition / mixins carefully.
* Changing base method signatures breaks Liskov expectations for callers.
* `isinstance` checks can fight duck typing; prefer protocols when flexible.
## π Related [#-related]
* [Classes](/docs/python/classes)
* [ABC / protocols](/docs/python/abc-protocols)
* [Magic methods](/docs/python/magic-methods)
* [Dataclasses](/docs/python/dataclasses)
* [Exceptions](/docs/python/exceptions)
* [Typing hints](/docs/python/typing-hints)
---
# Input and Output Basics (/docs/python/input-output-basics)
# Input and Output Basics [#input-and-output-basics]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Programs talk to the outside world through **output** (showing results) and **input** (reading data). Beginners usually start with `print` for output and `input` for reading a line of text from the keyboard.
## π§ Core concepts [#-core-concepts]
| Tool | Direction | Notes |
| ---------------- | --------- | ----------------------------------------------------- |
| `print(*values)` | Out | Converts values to strings; default ends with newline |
| `input(prompt)` | In | Always returns a `str` (even for numbers you type) |
| f-strings | Out | `f"Hello \{name\}"` embeds expressions |
| `sep` / `end` | Out | Control separators and line endings |
| Casting | In | Use `int(...)`, `float(...)` after reading text |
Standard streams: **stdout** (normal output), **stderr** (errors), **stdin** (input).
## π‘ Examples [#-examples]
**Basic print and input:**
```python
name = input("What is your name? ")
print("Hello,", name)
print(f"Welcome, {name}!")
```
**Convert typed text to numbers:**
```python
raw = input("Enter a whole number: ")
n = int(raw)
print(n, "squared is", n * n)
```
**print options:**
```python
print("a", "b", "c", sep="-")
print("Loading", end="...")
print(" done")
```
**Safe parse with a try/except:**
```python
raw = input("Price: ").strip()
try:
price = float(raw)
except ValueError:
print("Please enter a number like 9.99")
else:
print(f"You entered {price:.2f}")
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `input` returns strings: `"5" + "1"` is `"51"`, not `6`.
* `int("3.14")` fails β use `float` first or validate.
* EOF in redirected input can raise `EOFError`.
* For real apps, prefer `logging` over scattered `print` debugging.
## π Related [#-related]
* [hello\_world.md](/docs/python/hello-world)
* [variables.md](/docs/python/variables)
* [print\_input.md](/docs/python/print-input)
* [none\_and\_truthy.md](/docs/python/none-and-truthy)
---
# Iterators (/docs/python/iterators)
# Iterators [#iterators]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
An iterable can produce an iterator via `iter(obj)`. An iterator yields values with `next(it)` until `StopIteration`. For-loops, comprehensions, and unpacking all consume iterators.
## π§ Core concepts [#-core-concepts]
| Protocol | Methods |
| ------------------- | --------------------------------------- |
| Iterable | `__iter__` β iterator |
| Iterator | `__iter__` (returns self) + `__next__` |
| `iter(x)` | Get iterator |
| `next(it, default)` | Next value or default |
| Sequence | `__len__` + `__getitem__` also iterable |
Many built-ins are iterable: `list`, `dict`, `range`, files, generators. Custom classes can implement the protocols or use generators.
## π‘ Examples [#-examples]
**Manual iteration:**
```python
it = iter([10, 20, 30])
print(next(it))
print(next(it))
print(next(it, None))
print(next(it, None)) # None β exhausted
```
**Custom iterator:**
```python
from collections.abc import Iterator
class CountDown:
def __init__(self, start: int) -> None:
self.current = start
def __iter__(self) -> Iterator[int]:
return self
def __next__(self) -> int:
if self.current <= 0:
raise StopIteration
self.current -= 1
return self.current + 1
print(list(CountDown(3))) # [3, 2, 1]
```
**Iterable (fresh iterator each time):**
```python
class Repeat:
def __init__(self, value: str, times: int) -> None:
self.value = value
self.times = times
def __iter__(self):
for _ in range(self.times):
yield self.value
r = Repeat("x", 3)
print(list(r), list(r)) # both work
```
**tee / consume carefully:**
```python
from itertools import tee
a, b = tee(range(3), 2)
print(list(a), list(b))
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Iterators are exhausted after one pass; iterables like `list` can re-iterate.
* Calling `iter` on an iterator returns itself β no rewind.
* Mutating a collection while iterating is unsafe.
* `StopIteration` inside generators becomes runtime errors in some contexts (PEP 479).
* Infinite iterators need explicit stopping (`islice`, `break`).
## π Related [#-related]
* [Generators](/docs/python/generators)
* [Loops](/docs/python/loops)
* [Itertools](/docs/python/itertools)
* [Magic methods](/docs/python/magic-methods)
* [Comprehensions](/docs/python/comprehensions)
* [Zip / enumerate / map / filter](/docs/python/zip-enumerate-map-filter)
---
# Itertools (/docs/python/itertools)
# Itertools [#itertools]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`itertools` offers fast, memory-efficient building blocks for iterators: infinite streams, combinatorics, and chaining. Compose small tools instead of materializing huge lists.
## π§ Core concepts [#-core-concepts]
| Function | Role |
| --------------------------------------- | ---------------------- |
| `count` `cycle` `repeat` | Infinite / repeated |
| `chain` `chain.from_iterable` | Concatenate |
| `islice` | Slice an iterator |
| `tee` | Split into n iterators |
| `groupby` | Group consecutive keys |
| `accumulate` | Running totals |
| `product` `permutations` `combinations` | Combinatorics |
| `zip_longest` | Zip with fill |
| `batched` | Chunks (3.12+) |
All return iterators β consume with `for`, `list`, `next`, or `islice`.
## π‘ Examples [#-examples]
**islice and count:**
```python
from itertools import count, islice
print(list(islice(count(10, 2), 5))) # [10, 12, 14, 16, 18]
```
**chain and groupby:**
```python
from itertools import chain, groupby
print(list(chain([1, 2], (3, 4))))
rows = [("a", 1), ("a", 2), ("b", 3)]
for key, group in groupby(rows, key=lambda r: r[0]):
print(key, list(group))
```
**Combinatorics:**
```python
from itertools import combinations, product
print(list(combinations("ABC", 2)))
print(list(product([0, 1], repeat=2)))
```
**batched (3.12+) / chunk recipe:**
```python
from itertools import batched, islice
from collections.abc import Iterator
print(list(batched(range(10), 3)))
def chunked(it: Iterator[int], n: int) -> Iterator[list[int]]:
while True:
block = list(islice(it, n))
if not block:
return
yield block
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `groupby` requires **sorted** (or already consecutive) keys.
* `tee` can buffer a lot if iterators diverge β prefer re-iterating sources.
* Infinite iterators need a stop condition.
* Combinatorial functions explode in size β bound inputs.
* Exhausting one `tee` branch while another lags costs memory.
## π Related [#-related]
* [Iterators](/docs/python/iterators)
* [Generators](/docs/python/generators)
* [Loops](/docs/python/loops)
* [Comprehensions](/docs/python/comprehensions)
* [Collections module](/docs/python/collections-module)
* [Zip / enumerate / map / filter](/docs/python/zip-enumerate-map-filter)
---
# JSON (/docs/python/json)
# JSON [#json]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
The `json` module serializes Python objects to JSON text/bytes and back. Use for APIs, configs, and storage. Prefer `json.loads`/`dumps` for strings and `load`/`dump` for files.
## π§ Core concepts [#-core-concepts]
| Function | Role |
| ----------------- | ----------------------------------------------- |
| `loads` / `dumps` | str β object |
| `load` / `dump` | file β object |
| Types | `dict` `list` `str` `int` `float` `bool` `None` |
| `indent` | Pretty-print |
| `ensure_ascii` | Escape non-ASCII when True |
| `default=` | Hook for non-JSON types |
| `object_hook=` | Custom dict β object |
JSON objects become `dict` with `str` keys. Duplicate keys: last wins on parse.
## π‘ Examples [#-examples]
**Round-trip:**
```python
import json
data = {"name": "Ada", "ok": True, "n": None, "tags": ["py"]}
text = json.dumps(data, indent=2, ensure_ascii=False)
print(text)
obj = json.loads(text)
```
**Files with pathlib:**
```python
import json
from pathlib import Path
path = Path("config.json")
path.write_text(json.dumps({"port": 8000}, indent=2), encoding="utf-8")
cfg = json.loads(path.read_text(encoding="utf-8"))
```
**Custom types:**
```python
import json
from datetime import datetime, timezone
def default(o: object) -> object:
if isinstance(o, datetime):
return o.isoformat()
raise TypeError(f"not JSON serializable: {type(o)}")
payload = {"ts": datetime.now(timezone.utc)}
print(json.dumps(payload, default=default))
```
**Typed parse helper:**
```python
import json
from typing import Any
def load_object(text: str) -> dict[str, Any]:
data = json.loads(text)
if not isinstance(data, dict):
raise TypeError("expected JSON object")
return data
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `tuple` becomes JSON array β comes back as `list`.
* Keys must be strings; non-str keys are coerced in `dumps` (surprising).
* `NaN`/`Infinity` are allowed by default but invalid in strict JSON β use `allow_nan=False`.
* Huge `loads` into memory β stream with `ijson` or line-delimited JSON for big data.
* Never `json.loads` untrusted data into `pickle`-like execution β JSON is data-only (safe) but still validate schema.
## π Related [#-related]
* [Dictionaries](/docs/python/dictionaries)
* [Files I/O](/docs/python/files-io)
* [Dataclasses](/docs/python/dataclasses)
* [Datetime](/docs/python/datetime)
* [Typing hints](/docs/python/typing-hints)
* [Examples: http requests](/docs/python/examples/http-requests)
---
# *args and **kwargs (/docs/python/kwags)
# \*args and \*\*kwargs [#args-and-kwargs]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`*args` collects extra positional arguments into a tuple; `**kwargs` collects extra keyword arguments into a dict. Use them for wrappers, decorators, forward-compatible APIs, and when calling through to another function. Prefer explicit parameters when the API is stable and small.
## π§ Core concepts [#-core-concepts]
| Syntax | Role |
| ------------ | ----------------------------------------- |
| `*args` | Extra positional β `tuple` |
| `**kwargs` | Extra keywords β `dict` |
| `*` in def | Keyword-only parameters after `*` |
| `/` in def | Positional-only before `/` (3.8+) |
| Call unpack | `f(*list)`, `f(**dict)` |
| Order in def | params, `*args`, keyword-only, `**kwargs` |
Names `args`/`kwargs` are convention only; `*rest` / `**options` are fine.
## π‘ Examples [#-examples]
**Collect and forward:**
```python
def logged(fn):
def wrapper(*args, **kwargs):
print(f"call {fn.__name__}")
return fn(*args, **kwargs)
return wrapper
@logged
def add(a: int, b: int) -> int:
return a + b
add(1, 2)
```
**Mix with explicit params:**
```python
def connect(host: str, port: int = 5432, /, *, timeout: float = 5.0, **opts):
print(host, port, timeout, opts)
connect("localhost", 5432, timeout=2.0, ssl=True)
```
**Unpacking into calls:**
```python
def greet(name: str, excited: bool = False) -> str:
return f"Hi {name}{'!' if excited else '.'}"
args = ("Ada",)
kwargs = {"excited": True}
print(greet(*args, **kwargs))
```
**Merge kwargs safely:**
```python
def create_user(name: str, **kwargs):
defaults = {"active": True, "role": "user"}
data = {**defaults, **kwargs, "name": name}
return data
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Overusing `**kwargs` hides required fields and breaks autocomplete/type checkersβannotate with `TypedDict` or explicit params when possible.
* Duplicate keywords: `f(1, a=1, **\{"a": 2\})` raises `TypeError`.
* Mutating the received `kwargs` dict can surprise callers if you store it; copy when retaining.
* `*args` typing is limited; prefer `*args: int` (homogeneous) or redesign.
* Forgetting to forward `*args, **kwargs` in wrappers drops arguments silently until call time.
## π Related [#-related]
* [Types](/docs/python/types)
* [Dictionaries](/docs/python/dictionaries)
* [Docstrings](/docs/python/docstrings)
* [Import / export](/docs/python/import-export)
* [Loops](/docs/python/loops)
---
# Lambda (/docs/python/lambda)
# Lambda [#lambda]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
A `lambda` creates a small anonymous function from a single expression. Use for short callbacks (`sort` keys, `map`/`filter`) when a full `def` would be noise. Prefer named `def` for anything non-trivial or multi-statement.
## π§ Core concepts [#-core-concepts]
| Form | Meaning |
| ------------------- | ----------------------------------------- |
| `lambda args: expr` | Returns `expr`; no statements |
| Parameters | Same rules as `def` (defaults, `*`, `**`) |
| Scope | Closure over enclosing names (LEGB) |
| Type | Ordinary function object |
Lambdas cannot contain `return`, assignments (except walrus in 3.8+ inside expr), or multiple statements. Annotations on lambdas are awkward β prefer `def`.
## π‘ Examples [#-examples]
**Sort and group keys:**
```python
rows = [("ada", 3), ("bob", 1), ("cy", 2)]
rows.sort(key=lambda pair: pair[1])
# [("bob", 1), ("cy", 2), ("ada", 3)]
```
**With map / filter (often better as comprehension):**
```python
nums = [1, 2, 3, 4]
doubled = list(map(lambda n: n * 2, nums))
evens = list(filter(lambda n: n % 2 == 0, nums))
# Prefer: [n * 2 for n in nums], [n for n in nums if n % 2 == 0]
```
**Callable default / partial-like:**
```python
ops = {
"add": lambda a, b: a + b,
"mul": lambda a, b: a * b,
}
print(ops["add"](2, 3))
```
**Capture loop variable correctly:**
```python
# Wrong: all lambdas see final i
bad = [lambda: i for i in range(3)]
# Right: default-arg bind
good = [lambda i=i: i for i in range(3)]
print([f() for f in good]) # [0, 1, 2]
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Do not stuff complex logic into lambdas β readability collapses.
* Late binding in closures: loop variables need default-arg capture.
* `lambda` has no good docstring / name in stack traces (``).
* Prefer comprehensions over `map`/`filter` + lambda in most Python code.
* Cannot assign annotations cleanly; use `def` for public APIs.
## π Related [#-related]
* [Functions](/docs/python/functions)
* [Closures](/docs/python/closures)
* [Decorators](/docs/python/decorators)
* [Zip / enumerate / map / filter](/docs/python/zip-enumerate-map-filter)
* [Comprehensions](/docs/python/comprehensions)
* [Walrus](/docs/python/walrus)
---
# Lists (/docs/python/lists)
# Lists [#lists]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
A `list` is an ordered, mutable sequence. Use for collections that grow, shrink, or change in place. Prefer list methods and comprehensions over manual index loops. Heterogeneous types are allowed; typed code usually keeps one element type.
## π§ Core concepts [#-core-concepts]
**Create / access**
| Operation | Example |
| ------------- | ----------------------------------------- |
| Create | `[1, 2]`, `list(iterable)`, `[]` |
| Index / slice | `xs[0]`, `xs[-1]`, `xs[1:3]`, `xs[i:j:k]` |
| Assign slice | `xs[1:3] = [9, 8]`; can change length |
| Unpack | `a, *mid, z = xs` |
**Instance methods**
| Method | Mutates? | Notes |
| ---------------------------------- | -------- | -------------------------------------------- |
| `append(x)` | yes | Add one at end β O(1) amort. |
| `extend(it)` | yes | Add all from iterable; `xs += it` |
| `insert(i, x)` | yes | Insert before index `i` β O(n) |
| `pop([i])` | yes | Remove/return at `i` (default last) |
| `remove(x)` | yes | Remove first `== x`; `ValueError` if missing |
| `clear()` | yes | Remove all |
| `index(x[, start[, stop]])` | no | First index; `ValueError` if missing |
| `count(x)` | no | Occurrences of `x` |
| `sort(*, key=None, reverse=False)` | yes | In-place; returns `None` |
| `reverse()` | yes | In-place reverse |
| `copy()` | no | Shallow copy (`xs[:]` / `list(xs)`) |
**Builtins that operate on lists / sequences**
| Builtin | Notes |
| ------------------------------------------------ | ----------------------------------- |
| `len(xs)` | Length |
| `x in xs` / `x not in xs` | Membership β O(n) |
| `min` / `max` / `sum` | Over elements; `sum` needs numbers |
| `sorted(xs, *, key=β¦, reverse=β¦)` | New sorted list |
| `reversed(xs)` | Reverse iterator |
| `enumerate(xs, start=0)` | `(i, x)` pairs |
| `zip(*seqs)` | Parallel tuples (stops at shortest) |
| `all` / `any` | Truth of all / any elements |
| `map(fn, xs)` / `filter(fn, xs)` | Lazy; wrap with `list(...)` |
| `slice` assignment / `del xs[i]` / `del xs[a:b]` | Delete or replace ranges |
Lists are dynamic arrays; append/pop at the end is amortized O(1). Insert/delete at the front is O(n).
## π‘ Examples [#-examples]
**Build and mutate:**
```python
nums: list[int] = [3, 1, 4]
nums.append(1)
nums.extend([5, 9])
nums.insert(0, 0)
last = nums.pop() # 9
nums.remove(1) # removes first 1
print(nums) # [0, 3, 4, 1, 5]
```
**Sort with key:**
```python
people = [{"name": "bob", "age": 30}, {"name": "ada", "age": 25}]
people.sort(key=lambda p: p["age"])
names = sorted(people, key=lambda p: p["name"])
```
**Comprehension and unpacking:**
```python
squares = [n * n for n in range(6) if n % 2 == 0]
a, *mid, z = [10, 20, 30, 40]
# a=10, mid=[20, 30], z=40
```
**Stack / queue patterns:**
```python
stack: list[str] = []
stack.append("a")
stack.pop() # LIFO
from collections import deque
q: deque[str] = deque()
q.append("a")
q.popleft() # FIFO β prefer deque over list.pop(0)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `xs + ys` creates a new list; `xs.extend(ys)` mutates in place.
* `xs *= n` and `[[] ] * n` share references for mutable elements.
* `list.sort` returns `None` β use `sorted` when you need a new list.
* Shallow copy does not deep-copy nested lists β see [copy / deepcopy](/docs/python/copy-deepcopy).
* Prefer `deque` for frequent left pops; lists are poor queues.
## π Related [#-related]
* [Tuples](/docs/python/tuples)
* [Sets](/docs/python/sets)
* [Slicing](/docs/python/slicing)
* [Comprehensions](/docs/python/comprehensions)
* [Unpacking](/docs/python/unpacking)
* [Examples: merge lists](/docs/python/examples/merge-lists)
---
# Logging (/docs/python/logging)
# Logging [#logging]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
The `logging` module is the standard way to record program events. Prefer it over `print` for libraries and services. Configure once at the app entrypoint; modules use `logging.getLogger(__name__)`.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| ----------- | ------------------------------------------- |
| Logger | Named channel (`getLogger`) |
| Level | `DEBUG` `INFO` `WARNING` `ERROR` `CRITICAL` |
| Handler | Destination (stderr, file, syslog) |
| Formatter | Message layout |
| Filter | Extra include/exclude rules |
| Propagation | Child β parent loggers |
Root logger defaults to `WARNING`. Libraries should not call `basicConfig`; apps should.
## π‘ Examples [#-examples]
**Module logger:**
```python
import logging
log = logging.getLogger(__name__)
def work(n: int) -> int:
log.debug("start n=%s", n)
if n < 0:
log.error("invalid n=%s", n)
raise ValueError(n)
log.info("ok")
return n * 2
```
**basicConfig for scripts:**
```python
import logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s [%(name)s] %(message)s",
)
logging.getLogger(__name__).info("boot")
```
**File + stderr handlers:**
```python
import logging
from pathlib import Path
def setup(log_path: Path) -> None:
root = logging.getLogger()
root.setLevel(logging.DEBUG)
fmt = logging.Formatter("%(levelname)s %(message)s")
sh = logging.StreamHandler()
sh.setLevel(logging.INFO)
sh.setFormatter(fmt)
fh = logging.FileHandler(log_path, encoding="utf-8")
fh.setLevel(logging.DEBUG)
fh.setFormatter(fmt)
root.handlers.clear()
root.addHandler(sh)
root.addHandler(fh)
```
**exception helper:**
```python
import logging
log = logging.getLogger("app")
try:
1 / 0
except ZeroDivisionError:
log.exception("divide failed") # includes traceback
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Eager f-strings in log calls allocate even if level is filtered β use `%s` lazy args.
* Multiple `basicConfig` calls are no-ops if handlers already exist.
* Catching exceptions without `log.exception` loses stack traces.
* Don't log secrets (tokens, passwords).
* Avoid configuring logging inside importable libraries.
## π Related [#-related]
* [Exceptions](/docs/python/exceptions)
* [Argparse](/docs/python/argparse)
* [Examples: cli tool](/docs/python/examples/cli-tool)
* [Concurrency](/docs/python/concurrency)
* [Pathlib](/docs/python/pathlib)
* [Testing with pytest](/docs/python/testing-pytest)
---
# Loops (/docs/python/loops)
# Loops [#loops]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Python iterates with `for` over any iterable and `while` for condition-driven loops. Prefer `for` + comprehensions over index arithmetic. Control flow: `break`, `continue`, `else` on loops (runs if no `break`), and `enumerate` / `zip` for paired iteration.
## π§ Core concepts [#-core-concepts]
| Construct | Use |
| --------------------------- | ---------------------------------------------- |
| `for x in iterable` | Iterate values |
| `for i, x in enumerate(xs)` | Index + value |
| `for a, b in zip(xs, ys)` | Parallel iterables |
| `while cond` | Condition loop |
| `break` / `continue` | Exit / next iteration |
| `for`/`while` `else` | Runs if loop did not `break` |
| Comprehensions | `[...]`, `\{...\}`, `(... for ...)` generators |
`range(stop)`, `range(start, stop[, step])` for integer sequences.
## π‘ Examples [#-examples]
**Enumerate, zip, unpacking:**
```python
names = ["Ada", "Bob"]
scores = [10, 7]
for i, name in enumerate(names, start=1):
print(i, name)
for name, score in zip(names, scores, strict=True): # 3.10+
print(f"{name}: {score}")
```
**Loop else (search):**
```python
needle = 4
for n in [1, 2, 3]:
if n == needle:
print("found")
break
else:
print("not found")
```
**Comprehensions and generators:**
```python
squares = [n * n for n in range(5) if n % 2 == 0]
gen = (n * n for n in range(1_000_000)) # lazy
total = sum(gen)
```
**While with sentinel:**
```python
from collections.abc import Iterator
def read_chunks() -> Iterator[str]:
yield "a"
yield "b"
it = iter(read_chunks())
while True:
try:
chunk = next(it)
except StopIteration:
break
print(chunk)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Do not mutate a list you are iterating; iterate a copy or build a new list.
* `for i in range(len(xs))` is usually worse than `for x in xs` / `enumerate`.
* Infinite `while True` needs a clear exit; prefer iterators when possible.
* `zip` truncates to the shortest iterable unless `strict=True`.
* Creating huge lists with comprehensions can exhaust memoryβuse generators.
## π Related [#-related]
* [Dictionaries](/docs/python/dictionaries)
* [Types](/docs/python/types)
* [Strings](/docs/python/strings)
* [\*args and \*\*kwargs](/docs/python/kwags)
* [Async / await](/docs/python/async)
* [Examples: merge lists](/docs/python/examples/merge-lists)
---
# Magic Methods (/docs/python/magic-methods)
# Magic Methods [#magic-methods]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Magic (dunder) methods customize object behavior for operators, iteration, context managers, formatting, and more. Implement only what you need; keep them consistent (`__eq__` with `__hash__`, rich comparisons together).
## π§ Core concepts [#-core-concepts]
| Category | Methods |
| ---------- | ------------------------------------------------------- |
| Lifecycle | `__init__`, `__new__`, `__del__` |
| Repr / str | `__repr__`, `__str__`, `__format__` |
| Compare | `__eq__`, `__lt__`, `__le__`, β¦ |
| Hash | `__hash__` (immutable value types) |
| Container | `__len__`, `__getitem__`, `__setitem__`, `__contains__` |
| Numeric | `__add__`, `__radd__`, `__iadd__`, β¦ |
| Call | `__call__` |
| Context | `__enter__`, `__exit__` |
| Iter | `__iter__`, `__next__`, `__aiter__` |
Returning `NotImplemented` from binary ops lets Python try the reflected method.
## π‘ Examples [#-examples]
**Value object:**
```python
from __future__ import annotations
class Money:
def __init__(self, cents: int) -> None:
self.cents = cents
def __repr__(self) -> str:
return f"Money({self.cents})"
def __str__(self) -> str:
return f"${self.cents / 100:.2f}"
def __eq__(self, other: object) -> bool:
if not isinstance(other, Money):
return NotImplemented
return self.cents == other.cents
def __hash__(self) -> int:
return hash(self.cents)
def __add__(self, other: Money) -> Money:
if not isinstance(other, Money):
return NotImplemented
return Money(self.cents + other.cents)
```
**Container protocol:**
```python
class Bag:
def __init__(self, items: list[str] | None = None) -> None:
self._items = list(items or [])
def __len__(self) -> int:
return len(self._items)
def __getitem__(self, index: int) -> str:
return self._items[index]
def __contains__(self, item: object) -> bool:
return item in self._items
def __iter__(self):
return iter(self._items)
```
**Callable instance:**
```python
class Adder:
def __init__(self, n: int) -> None:
self.n = n
def __call__(self, x: int) -> int:
return x + self.n
add5 = Adder(5)
print(add5(10)) # 15
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Mutable objects should set `__hash__ = None` if they define `__eq__`.
* `__repr__` should be unambiguous; `__str__` can be user-friendly.
* Forgetting `__radd__` breaks `sum()` starting from `0` for custom types.
* `__del__` is not a destructor guarantee β avoid critical logic there.
* Inconsistent `__eq__` / ordering methods confuse sorts and sets.
## π Related [#-related]
* [Classes](/docs/python/classes)
* [Context managers](/docs/python/context-managers)
* [Iterators](/docs/python/iterators)
* [Operators](/docs/python/operators)
* [Dataclasses](/docs/python/dataclasses)
* [Properties / descriptors](/docs/python/properties-descriptors)
---
# None and Truthiness (/docs/python/none-and-truthy)
# None and Truthiness [#none-and-truthiness]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`None` means βno valueβ (Pythonβs null). **Truthiness** means how values behave in `if` / `while` without an explicit comparison: some values count as false, everything else as true.
## π§ Core concepts [#-core-concepts]
| Idea | Detail |
| ------------- | ----------------------------------------------------------- |
| `None` | Singleton object; type is `NoneType` |
| Test for None | Use `x is None` / `x is not None` |
| Falsy values | `None`, `False`, `0`, `0.0`, `""`, `[]`, `\{\}`, `set()`, β¦ |
| Truthy values | Non-empty containers, non-zero numbers, most objects |
| `bool(x)` | Explicit conversion to `True`/`False` |
Functions that βreturn nothingβ actually return `None`.
## π‘ Examples [#-examples]
**None as a missing value:**
```python
user = None
if user is None:
print("Please log in")
else:
print(user)
```
**Default then assign:**
```python
def find_user(user_id):
# pretend lookup failed
return None
result = find_user(42)
name = result if result is not None else "anonymous"
print(name)
```
**Truthiness in conditions:**
```python
items = []
text = ""
count = 0
if not items:
print("list is empty")
if not text:
print("no text")
if count:
print("never runs for 0")
```
**Truthy gotchas with numbers and containers:**
```python
print(bool(0), bool(1), bool(-1)) # False True True
print(bool(""), bool("0")) # False True
print(bool([]), bool([0])) # False True
print(bool(None)) # False
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `if x == None` works but style guides prefer `if x is None`.
* Empty containers are falsy β that is often useful, sometimes surprising.
* The string `"False"` and the number `0` inside a list `[0]` are truthy as containers/strings.
* Returning `None` accidentally from a function that should return data is a common bug.
## π Related [#-related]
* [boolean\_logic.md](/docs/python/boolean-logic)
* [variables.md](/docs/python/variables)
* [truthiness.md](/docs/python/truthiness)
* [input\_output\_basics.md](/docs/python/input-output-basics)
---
# Operators (/docs/python/operators)
# Operators [#operators]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Python operators cover arithmetic, comparison, boolean logic, bitwise ops, membership, identity, and assignment variants. Precedence matters; use parentheses for clarity. Many operators map to magic methods.
## π§ Core concepts [#-core-concepts]
| Group | Operators |
| ------------ | ---------------------------- |
| Arithmetic | `+ - * / // % **` |
| Comparison | `== != < <= > >=` |
| Boolean | `and or not` (short-circuit) |
| Identity | `is`, `is not` |
| Membership | `in`, `not in` |
| Bitwise | `& \| ^ ~ << >>` |
| Assign | `=`, `+=`, `:=` (walrus) |
| Merge (3.9+) | `d1 \| d2` for dicts |
| Matrix | `@` (e.g. NumPy) |
`/` always float division; `//` floor division. `and`/`or` return operands, not strict bools.
## π‘ Examples [#-examples]
**Arithmetic and div:**
```python
print(7 / 2) # 3.5
print(7 // 2) # 3
print(7 % 2) # 1
print(2 ** 10) # 1024
print(-7 // 2) # -4 (floors toward -inf)
```
**Boolean and chaining:**
```python
x = 5
print(1 < x < 10) # True β chained comparisons
print("" or "default") # "default"
print("hi" and "there") # "there"
```
**Identity vs equality:**
```python
a = [1, 2]
b = [1, 2]
print(a == b) # True β equal values
print(a is b) # False β different objects
print(a is not None)
```
**Dict merge and augmented assign:**
```python
left = {"a": 1}
right = {"b": 2}
merged = left | right # {"a": 1, "b": 2}
left |= {"a": 9} # left -> {"a": 9}
n = 3
n *= 2 # 6
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `is` is not for value equality (except `None`, `True`, `False`, enums).
* Mutable `+=` may mutate in place (`list`) or rebind (`tuple`).
* `and`/`or` return last evaluated operand β type may not be `bool`.
* Bitwise `&` / `|` on ints β boolean `and` / `or`.
* Mixed `==` across types is usually `False` (no coercion like JS).
## π Related [#-related]
* [Magic methods](/docs/python/magic-methods)
* [Truthiness](/docs/python/truthiness)
* [Walrus](/docs/python/walrus)
* [Conditionals](/docs/python/conditionals)
* [Slicing](/docs/python/slicing)
* [Unpacking](/docs/python/unpacking)
---
# OS (/docs/python/os)
# OS [#os]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
The `os` module talks to the operating system: env vars, process info, and basic path helpers. Prefer `pathlib` for path work; keep `os` for env, cwd, and platform details.
## π§ Core concepts [#-core-concepts]
| API | Role |
| -------------------------------------- | ------------------------------------ |
| `os.environ` | Mapping of environment variables |
| `os.getenv(k, default)` | Safe read (returns `None` / default) |
| `os.environ["KEY"]` | Read/set; `KeyError` if missing |
| `os.getcwd()` / `os.chdir(path)` | Working directory |
| `os.name` / `os.sep` | Platform hints |
| `os.listdir(path)` | Directory entries (names only) |
| `os.makedirs(path, exist_ok=True)` | Create dirs |
| `os.remove` / `os.rename` | File ops |
| `os.path.join` / `exists` / `splitext` | String path helpers |
| `pathlib.Path` | Preferred OO paths |
**`os.path` vs `pathlib`:** `os.path` returns strings; `Path` objects support `/`, `.read_text()`, `.glob()`, and clearer APIs. Use `pathlib` for new code; `os.path` still appears in older APIs.
## π‘ Examples [#-examples]
**Environment variables:**
```python
import os
home = os.getenv("HOME") or os.getenv("USERPROFILE")
os.environ["APP_ENV"] = "dev"
print(os.environ.get("APP_ENV", "prod"))
# del os.environ["APP_ENV"] # unset
```
**Cwd and listing:**
```python
import os
from pathlib import Path
print(os.getcwd())
for name in os.listdir("."):
print(name, Path(name).is_file())
```
**`os.path` vs `pathlib`:**
```python
import os
from pathlib import Path
s = os.path.join("data", "a.csv")
print(os.path.exists(s), os.path.splitext(s))
p = Path("data") / "a.csv"
print(p.exists(), p.suffix, p.read_text(encoding="utf-8") if p.exists() else "")
```
**Create dirs (os vs pathlib):**
```python
import os
from pathlib import Path
os.makedirs("out/logs", exist_ok=True)
Path("out/cache").mkdir(parents=True, exist_ok=True)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `os.environ["MISSING"]` raises `KeyError` β prefer `getenv` / `.get`.
* Env values are always strings; cast (`int(os.getenv("PORT", "8000"))`).
* `chdir` is process-global and surprises libraries β avoid in libraries.
* Mixing `str` and `Path` is fine at the boundary; stick to one style inside a module.
* `listdir` does not recurse; use `Path.rglob` or `os.walk` for trees.
## π Related [#-related]
* [Pathlib](/docs/python/pathlib)
* [Files I/O](/docs/python/files-io)
* [Dotenv](/docs/python/dotenv)
* [Shutil](/docs/python/shutil)
* [Sys](/docs/python/sys)
* [Venv](/docs/python/venv)
---
# Packaging (/docs/python/packaging)
# Packaging [#packaging]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Packaging turns your code into an installable distribution (wheel/sdist) with metadata. Modern projects use `pyproject.toml` (PEP 517/518/621) with backends like `setuptools`, `hatchling`, or `flit`.
## π§ Core concepts [#-core-concepts]
| Piece | Role |
| ---------------- | ------------------------------- |
| `pyproject.toml` | Build + project metadata |
| Build backend | setuptools / hatchling / β¦ |
| Wheel | Built install artifact (`.whl`) |
| sdist | Source distribution |
| Entry points | Console scripts |
| Extras | Optional deps (`dev`, `docs`) |
| Version | Static or dynamic |
Layout: `src/mypkg/` (recommended) or flat `mypkg/`.
## π‘ Examples [#-examples]
**Minimal pyproject (setuptools):**
```toml
[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "awesome-tool"
version = "0.1.0"
description = "Demo package"
readme = "README.md"
requires-python = ">=3.10"
dependencies = ["requests>=2.31"]
[project.optional-dependencies]
dev = ["pytest>=8"]
[project.scripts]
awesome-tool = "awesome_tool.cli:main"
[tool.setuptools.packages.find]
where = ["src"]
```
**Build and install:**
```shellscript
python -m pip install build
python -m build
python -m pip install dist/awesome_tool-0.1.0-py3-none-any.whl
# or editable
python -m pip install -e ".[dev]"
```
**Console script stub:**
```python
# src/awesome_tool/cli.py
def main() -> None:
print("hello from awesome-tool")
if __name__ == "__main__":
main()
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Forgetting `packages.find` / package data β empty installs.
* Pinning too tightly in libraries frustrates consumers.
* Shipping tests/secrets inside wheels by accident β check includes.
* Multiple tools writing metadata (`setup.cfg` + poetry) β pick one source of truth.
* Entry point module path typos fail only after install.
## π Related [#-related]
* [Pip](/docs/python/pip)
* [Venv](/docs/python/venv)
* [Argparse](/docs/python/argparse)
* [Import / export](/docs/python/import-export)
* [Examples: cli tool](/docs/python/examples/cli-tool)
* [Testing with pytest](/docs/python/testing-pytest)
---
# Pandas (/docs/python/pandas)
# Pandas [#pandas]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Pandas is the go-to tabular data library. Install with `pip install pandas`. Core types: `Series` (1D labeled) and `DataFrame` (2D table). Typical flow: `read_csv` β filter/transform β `groupby` / `merge` β `to_csv`.
## π§ Core concepts [#-core-concepts]
| API | Role |
| --------------------------------------- | ----------------------- |
| `pd.Series(data)` | 1D labeled array |
| `pd.DataFrame(data)` | Table (columns + index) |
| `pd.read_csv(path)` | Load CSV |
| `df[mask]` / `.loc` / `.iloc` | Filter / select |
| `df.groupby(col)` | Split-apply-combine |
| `pd.merge(left, right, on=...)` | Join tables |
| `df.to_csv(path, index=False)` | Export |
| `df.head()` / `.info()` / `.describe()` | Inspect |
## π‘ Examples [#-examples]
**Read and filter:**
```python
import pandas as pd
df = pd.read_csv("sales.csv")
active = df[df["status"] == "active"]
big = df.loc[df["amount"] > 100, ["name", "amount"]]
print(df.head(), active.shape)
```
**Groupby:**
```python
import pandas as pd
df = pd.read_csv("orders.csv")
summary = (
df.groupby("region", as_index=False)["amount"]
.sum()
.sort_values("amount", ascending=False)
)
print(summary)
```
**Merge:**
```python
import pandas as pd
orders = pd.read_csv("orders.csv")
customers = pd.read_csv("customers.csv")
joined = pd.merge(orders, customers, on="customer_id", how="left")
print(joined.head())
```
**Write CSV:**
```python
import pandas as pd
df = pd.DataFrame({"a": [1, 2], "b": ["x", "y"]})
df.to_csv("out.csv", index=False, encoding="utf-8")
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Install first: `pip install pandas` (not in the stdlib).
* Chained indexing (`df[...][...] =`) can fail silently β prefer `.loc`.
* `groupby` default may leave the key as index; use `as_index=False` when you want a flat frame.
* Merges duplicate key columns as `_x` / `_y` if names collide β set `suffixes`.
* Large CSVs: pass `usecols`, `dtype`, or chunk with `chunksize=`.
## π Related [#-related]
* [Csv](/docs/python/csv)
* [Files I/O](/docs/python/files-io)
* [Pip](/docs/python/pip)
* [Venv](/docs/python/venv)
---
# Pathlib (/docs/python/pathlib)
# Pathlib [#pathlib]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`pathlib.Path` is the modern object-oriented path API. Prefer it over `os.path` string munging. Paths use `/` operator, work cross-platform, and integrate with open/read/write helpers.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ---------------------------------- | ------------------- |
| `Path("a/b")` | Create path |
| `p / "c"` | Join |
| `.name` `.stem` `.suffix` | Parts |
| `.parent` `.parents` | Ancestors |
| `.resolve()` | Absolute + symlinks |
| `.exists()` `.is_file()` | Queries |
| `.iterdir()` `.glob()` `.rglob()` | Listing |
| `.read_text()` `.write_text()` | Convenience I/O |
| `.mkdir()` `.unlink()` `.rename()` | FS ops |
Use `PurePath` when you only need parsing without touching the filesystem.
## π‘ Examples [#-examples]
**Build and inspect:**
```python
from pathlib import Path
base = Path(__file__).resolve().parent
data = base / "data" / "input.csv"
print(data.name, data.suffix, data.parent)
print(data.with_suffix(".json"))
```
**Read / write:**
```python
from pathlib import Path
p = Path("notes.txt")
p.write_text("hello\n", encoding="utf-8")
text = p.read_text(encoding="utf-8")
p.write_bytes(b"\x00\x01")
```
**Walk and filter:**
```python
root = Path("src")
for py in sorted(root.rglob("*.py")):
if py.is_file() and "test" not in py.parts:
print(py.relative_to(root))
```
**mkdir / safe delete:**
```python
out = Path("build/out")
out.mkdir(parents=True, exist_ok=True)
tmp = out / "tmp.txt"
tmp.write_text("x", encoding="utf-8")
tmp.unlink(missing_ok=True)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Relative paths depend on process cwd β anchor with `Path(__file__).parent`.
* Always pass `encoding` for text helpers.
* `Path.glob("**/*")` can be huge; narrow patterns.
* Mixing `str` paths and `Path` works often, but keep one style in APIs.
* `resolve(strict=True)` raises if the path does not exist.
## π Related [#-related]
* [Files I/O](/docs/python/files-io)
* [Examples: manage files](/docs/python/examples/manage-files)
* [Context managers](/docs/python/context-managers)
* [Strings](/docs/python/strings)
* [JSON](/docs/python/json)
* [Import / export](/docs/python/import-export)
---
# Pip (/docs/python/pip)
# Pip [#pip]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`pip` installs packages from PyPI (and other indexes) into the current environment. Use `python -m pip` for reliability. Pin versions for apps; keep libraries flexible within SemVer ranges.
## π§ Core concepts [#-core-concepts]
| Command | Role |
| --------------------------------- | ------------------------- |
| `pip install pkg` | Install latest |
| `pip install pkg==1.2.3` | Pin version |
| `pip install -r requirements.txt` | From file |
| `pip install -e .` | Editable project |
| `pip uninstall pkg` | Remove |
| `pip list` / `show` | Inspect |
| `pip freeze` | Export pins |
| `pip index versions pkg` | List versions (newer pip) |
Constraints files (`constraints.txt`) pin transitive deps without listing them as direct requirements.
## π‘ Examples [#-examples]
**Everyday installs:**
```shellscript
python -m pip install -U pip setuptools wheel
python -m pip install 'requests>=2.31'
python -m pip install -r requirements.txt
```
**Editable local package:**
```shellscript
python -m pip install -e .
python -m pip install -e ".[dev]"
```
**requirements vs freeze:**
```shellscript
# hand-maintained direct deps
echo "flask>=3.0" > requirements.txt
# full lock-ish dump of current env
python -m pip freeze > requirements.lock.txt
```
**Extra index / trusted host (careful):**
```shellscript
python -m pip install pkg --index-url https://pypi.org/simple
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Never `curl | sh` unknown installers; prefer `python -m pip`.
* `pip freeze` includes transitive pins β great for apps, noisy for libraries.
* Mixing `--user` installs with venvs causes confusion.
* Compiling wheels may need build tools (Rust/C compilers) for some packages.
* Private indexes: protect tokens; don't commit credentials.
## π Related [#-related]
* [Venv](/docs/python/venv)
* [Packaging](/docs/python/packaging)
* [Import / export](/docs/python/import-export)
* [Testing with pytest](/docs/python/testing-pytest)
* [Examples: http requests](/docs/python/examples/http-requests)
* [Logging](/docs/python/logging)
---
# Print and Input (/docs/python/print-input)
# Print and Input [#print-and-input]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`print` writes to stdout; `input` reads a line from stdin as `str`. Fine for CLIs and debugging; prefer `logging` for applications and `argparse` for structured CLIs.
## π§ Core concepts [#-core-concepts]
| API | Role |
| -------------------------------------------------------- | ---------------------------- |
| `print(*objs, sep=' ', end='\n', file=..., flush=False)` | Output |
| `input(prompt="")` | Read line (no trailing `\n`) |
| `sys.stdout` / `stderr` | Streams |
| `sys.stdin` | Input stream |
`print` converts with `str()`. Soft space separation via `sep`. Use f-strings for formatting.
## π‘ Examples [#-examples]
**print options:**
```python
print("a", "b", "c", sep="-")
print("loading", end="...", flush=True)
print(" done")
import sys
print("error!", file=sys.stderr)
```
**input and parse:**
```python
name = input("Name: ").strip()
raw = input("Count: ").strip()
try:
count = int(raw)
except ValueError:
print("Not an integer", file=sys.stderr)
raise SystemExit(1)
print(f"Hello {name} x{count}")
```
**EOF-safe loop:**
```python
import sys
def main() -> None:
for line in sys.stdin:
text = line.strip()
if not text:
continue
print(text.upper())
if __name__ == "__main__":
main()
```
**Pretty debug:**
```python
from pprint import pprint
data = {"users": [{"id": 1, "name": "Ada"}], "ok": True}
pprint(data, width=40)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `input` always returns `str` β cast explicitly.
* EOF (`Ctrl+D` / `Ctrl+Z`) raises `EOFError` on `input()`.
* Password prompts: use `getpass.getpass` so input is not echoed.
* Leaving debug `print`s in libraries pollutes consumer output β use logging.
* Locale/encoding issues on redirected pipes β prefer UTF-8 aware environments.
## π Related [#-related]
* [F-strings](/docs/python/f-string)
* [Argparse](/docs/python/argparse)
* [Logging](/docs/python/logging)
* [Examples: cli tool](/docs/python/examples/cli-tool)
* [Strings](/docs/python/strings)
* [Exceptions](/docs/python/exceptions)
---
# Properties and Descriptors (/docs/python/properties-descriptors)
# Properties and Descriptors [#properties-and-descriptors]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Properties manage attribute access with getters/setters. Descriptors are the underlying protocol (`__get__` / `__set__` / `__delete__`) used by `property`, `classmethod`, and ORMs. Use properties for derived or validated fields; custom descriptors for reusable attribute behavior.
## π§ Core concepts [#-core-concepts]
| Tool | Role |
| -------------------------- | ----------------------------------------- |
| `@property` | Getter |
| `@x.setter` / `@x.deleter` | Mutate / delete |
| Data descriptor | Defines `__set__` and/or `__delete__` |
| Non-data descriptor | Only `__get__` (e.g. functions) |
| Lookup order | Data desc > instance dict > non-data desc |
`property` is implemented as a data descriptor. Slotted classes still work with descriptors on the class.
## π‘ Examples [#-examples]
**Property with validation:**
```python
class Person:
def __init__(self, name: str, age: int) -> None:
self.name = name
self.age = age
@property
def age(self) -> int:
return self._age
@age.setter
def age(self, value: int) -> None:
if value < 0:
raise ValueError("age must be >= 0")
self._age = value
@property
def label(self) -> str:
return f"{self.name} ({self._age})"
```
**Simple descriptor:**
```python
from typing import Any
class Positive:
def __set_name__(self, owner: type, name: str) -> None:
self.public_name = name
self.private_name = f"_{name}"
def __get__(self, obj: object | None, objtype: type | None = None) -> Any:
if obj is None:
return self
return getattr(obj, self.private_name)
def __set__(self, obj: object, value: int) -> None:
if value <= 0:
raise ValueError(f"{self.public_name} must be > 0")
setattr(obj, self.private_name, value)
class Item:
qty = Positive()
def __init__(self, qty: int) -> None:
self.qty = qty
```
**cached\_property:**
```python
from functools import cached_property
class Report:
def __init__(self, rows: list[int]) -> None:
self.rows = rows
@cached_property
def total(self) -> int:
return sum(self.rows)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Infinite recursion: don't touch `self.age` inside its own getter β use `self._age`.
* Data descriptors hide same-named instance attributes.
* `cached_property` is not thread-lock-safe for all use cases; know the docs.
* Overusing setters instead of immutable design can complicate state.
* Descriptors must live on the **class**, not on instances.
## π Related [#-related]
* [Classes](/docs/python/classes)
* [Decorators](/docs/python/decorators)
* [Magic methods](/docs/python/magic-methods)
* [Dataclasses](/docs/python/dataclasses)
* [ABC / protocols](/docs/python/abc-protocols)
* [Typing hints](/docs/python/typing-hints)
---
# Random (/docs/python/random)
# Random [#random]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
The `random` module provides pseudo-random numbers for simulations, sampling, and shuffling. Do **not** use it for security (passwords, tokens) β use `secrets` instead.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ------------------------------------ | ---------------------- |
| `random()` | Float in `[0.0, 1.0)` |
| `randrange(stop)` / `randint(a, b)` | Integers |
| `choice(seq)` / `choices` / `sample` | Pick items |
| `shuffle(list)` | In-place shuffle |
| `uniform(a, b)` / `gauss` | Distributions |
| `seed(n)` | Reproducible sequences |
| `Random()` | Isolated RNG instance |
## π‘ Examples [#-examples]
```python
import random
random.seed(42)
print(random.randint(1, 6))
print(random.choice(["red", "green", "blue"]))
deck = list(range(1, 53))
random.shuffle(deck)
hand = random.sample(deck, 5)
```
```python
import random
# Weighted picks
colors = ["common", "rare", "epic"]
print(random.choices(colors, weights=[80, 15, 5], k=3))
```
```python
import secrets
token = secrets.token_urlsafe(16)
code = secrets.randbelow(1_000_000)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `random` is not cryptographically secure β use `secrets` for auth material.
* `shuffle` mutates the list; copy first if you need the original order.
* Global RNG is shared β prefer `random.Random()` in libraries/tests.
* Seeding makes runs reproducible but predictable.
## π Related [#-related]
* [hashlib](/docs/python/hashlib)
* [statistics](/docs/python/statistics)
* [lambda](/docs/python/lambda)
* [lists](/docs/python/lists)
---
# Regex (/docs/python/regex)
# Regex [#regex]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
The `re` module provides regular expressions for search, match, split, and substitute. Prefer raw strings (`r"..."`). For complex grammars, consider parsers; for fixed substrings, prefer `str` methods.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ------------------------- | ---------------------------------------------- |
| `re.search` | First match anywhere |
| `re.match` | Match at start |
| `re.fullmatch` | Entire string |
| `re.findall` / `finditer` | All matches |
| `re.split` | Split by pattern |
| `re.sub` | Replace |
| `re.compile` | Reuse pattern |
| Flags | `IGNORECASE`, `MULTILINE`, `DOTALL`, `VERBOSE` |
Groups: `(...)` capturing, `(?:...)` non-capturing, `(?P...)` named. Conditional patterns exist but stay rare.
## π‘ Examples [#-examples]
**Search and groups:**
```python
import re
text = "user=ada id=42"
m = re.search(r"user=(?P\w+)\s+id=(?P\d+)", text)
if m:
print(m.group("user"), int(m.group("id")))
```
**Compile and findall:**
```python
import re
email = re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+")
print(email.findall("a@x.com b@y.org"))
```
**sub and backrefs:**
```python
import re
raw = "2024-07-10"
print(re.sub(r"(\d{4})-(\d{2})-(\d{2})", r"\2/\3/\1", raw))
```
**VERBOSE + flags:**
```python
import re
pattern = re.compile(
r"""
^
(?P\d{4}) -
(?P\d{2}) -
(?P\d{2})
$
""",
re.VERBOSE,
)
print(pattern.fullmatch("2024-07-10").groupdict()) # type: ignore[union-attr]
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Catastrophic backtracking on crafted input β avoid nested ambiguous quantifiers on untrusted data.
* `re.match` only checks the start β use `fullmatch` or `search` intentionally.
* Forgetting raw strings: `"\n"` vs `r"\n"` / `"\\d"` mistakes.
* `str.split` / `replace` is clearer for fixed delimiters.
* Bytes patterns require `bytes` input β don't mix str/bytes.
## π Related [#-related]
* [Strings](/docs/python/strings)
* [F-strings](/docs/python/f-string)
* [JSON](/docs/python/json)
* [Examples: scrape HTML](/docs/python/examples/scrape-html)
* [Files I/O](/docs/python/files-io)
* [Argparse](/docs/python/argparse)
---
# Requests (/docs/python/requests-http)
# Requests [#requests]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`requests` is the popular third-party HTTP client. Install: `pip install requests`. Use it for GET/POST, JSON APIs, headers, and timeouts. For stdlib-only code, see `urllib` β most apps prefer `requests`.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ----------------------------------- | -------------------------------------- |
| `requests.get(url, ...)` | HTTP GET |
| `requests.post(url, ...)` | HTTP POST |
| `params=` | Query string dict |
| `json=` | Encode body as JSON + set content-type |
| `data=` / `files=` | Form body / multipart |
| `headers=` | Custom headers |
| `timeout=` | Connect/read timeout (seconds) |
| `r.raise_for_status()` | Raise on 4xx/5xx |
| `r.json()` / `r.text` / `r.content` | Parse response |
| `requests.Session()` | Reuse cookies/connection |
## π‘ Examples [#-examples]
**GET + JSON:**
```python
import requests
r = requests.get(
"https://httpbin.org/get",
params={"q": "python"},
timeout=10,
)
r.raise_for_status()
print(r.json()["args"])
```
**POST JSON:**
```python
import requests
r = requests.post(
"https://httpbin.org/post",
json={"name": "Ada", "active": True},
headers={"X-Client": "demo"},
timeout=10,
)
r.raise_for_status()
print(r.json()["json"])
```
**Session:**
```python
import requests
with requests.Session() as s:
s.headers.update({"Accept": "application/json"})
r = s.get("https://httpbin.org/headers", timeout=10)
print(r.json()["headers"]["Accept"])
```
**Error handling:**
```python
import requests
try:
r = requests.get("https://httpbin.org/status/404", timeout=5)
r.raise_for_status()
except requests.HTTPError as e:
print("http error", e.response.status_code)
except requests.RequestException as e:
print("network/other", e)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Always set `timeout=` β otherwise calls can hang forever.
* Check status: `raise_for_status()` or inspect `r.status_code`.
* `r.json()` fails on empty/non-JSON bodies β catch `ValueError`.
* Donβt disable TLS verify (`verify=False`) in production.
* Third-party package: pin versions in requirements; not in the stdlib.
## π Related [#-related]
* [Json](/docs/python/json)
* [Urllib parse](/docs/python/urllib-parse)
* [Exceptions](/docs/python/exceptions)
* [Pip](/docs/python/pip)
* [Venv](/docs/python/venv)
---
# Scope and Namespaces (/docs/python/scope-namespaces)
# Scope and Namespaces [#scope-and-namespaces]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Name lookup follows the **LEGB** rule: Local β Enclosing β Global β Built-in. Assignment creates bindings in the current scope unless `global` or `nonlocal` is declared. Each module and function has its own namespace.
## π§ Core concepts [#-core-concepts]
| Scope | Where |
| ------------ | ------------------------------------------------------------- |
| Local | Current function |
| Enclosing | Outer function(s) β closures |
| Global | Module level |
| Built-in | `builtins` module |
| `global x` | Bind / assign module name |
| `nonlocal x` | Rebind enclosing function name |
| Class body | Its own namespace; methods don't see it as enclosing for LEGB |
Comprehensions (3.x) have their own local scope for loop targets.
## π‘ Examples [#-examples]
**LEGB:**
```python
len = 3 # shadows built-in β avoid!
def outer() -> None:
x = "enclosing"
def inner() -> None:
print(x) # enclosing
inner()
outer()
```
**global / nonlocal:**
```python
counter = 0
def bump_global() -> None:
global counter
counter += 1
def make_counter():
n = 0
def inc() -> int:
nonlocal n
n += 1
return n
return inc
```
**Class vs instance:**
```python
class Demo:
kind = "demo"
def label(self) -> str:
# 'kind' is not local; looked up on class via self/type
return self.kind
print(Demo().label())
```
**Inspect:**
```python
import builtins
def f(a: int = 1) -> None:
b = 2
print(locals())
print("sum" in dir(builtins))
f()
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Assignment anywhere in a function makes the name local for the whole function β easy `UnboundLocalError`.
* Shadowing builtins (`list`, `id`, `type`) causes confusing bugs.
* `from module import *` pollutes namespaces β avoid.
* Class attributes vs instance attributes: mutable class attrs are shared.
* Closures late-bind β see [closures](/docs/python/closures).
## π Related [#-related]
* [Closures](/docs/python/closures)
* [Functions](/docs/python/functions)
* [Classes](/docs/python/classes)
* [Import / export](/docs/python/import-export)
* [Walrus](/docs/python/walrus)
* [Lambda](/docs/python/lambda)
---
# Secrets (/docs/python/secrets)
# Secrets [#secrets]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`secrets` generates cryptographically strong random numbers suitable for passwords, tokens, and auth codes. Prefer it over `random` whenever unpredictability matters.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ------------------ | --------------------------------- |
| `token_bytes(n)` | `n` random bytes |
| `token_hex(n)` | Hex string (\~2n chars) |
| `token_urlsafe(n)` | URL-safe base64 |
| `randbelow(n)` | Int in `0 .. n-1` |
| `choice(seq)` | Secure pick from sequence |
| `SystemRandom` | CSPRNG-backed `random.Random` API |
## π‘ Examples [#-examples]
```python
import secrets
import string
alphabet = string.ascii_letters + string.digits
password = "".join(secrets.choice(alphabet) for _ in range(20))
api_key = secrets.token_urlsafe(32)
otp = f"{secrets.randbelow(1_000_000):06d}"
```
```python
import secrets
from hmac import compare_digest
expected = secrets.token_hex(16)
# constant-time compare for tokens
ok = compare_digest(expected, expected)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Still hash/store passwords with a KDF (e.g. `hashlib.scrypt` / Argon2) β donβt store raw secrets.
* `token_hex(16)` is 16 bytes of entropy β 32 hex chars, not β16 hex charsβ.
* Donβt use `random` for session IDs or reset tokens.
## π Related [#-related]
* [random](/docs/python/random)
* [hashlib](/docs/python/hashlib)
* [dotenv](/docs/python/dotenv)
* [hmac note via hashlib](/docs/python/hashlib)
---
# Sets (/docs/python/sets)
# Sets [#sets]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
A `set` is an unordered collection of unique hashable elements. Use for membership tests, deduplication, and set algebra (union, intersection, difference). `frozenset` is the immutable, hashable variant.
## π§ Core concepts [#-core-concepts]
| Operation | Example |
| -------------- | ---------------------------------------------------- |
| Create | `\{1, 2\}`, `set(iterable)`, `set()` (not `\{\}`) |
| Add / remove | `s.add(x)`, `s.discard(x)`, `s.remove(x)`, `s.pop()` |
| Membership | `x in s` β average O(1) |
| Union | `a \| b`, `a.union(b)` |
| Intersection | `a & b`, `a.intersection(b)` |
| Difference | `a - b`, `a.difference(b)` |
| Symmetric diff | `a ^ b` |
| Subset | `a <= b`, `a < b` (proper) |
Elements must be hashable. Insertion order is preserved for iteration in 3.7+ CPython, but do not treat sets as ordered sequences.
## π‘ Examples [#-examples]
**Deduplicate and membership:**
```python
tags = ["py", "web", "py", "api"]
unique = set(tags) # {"py", "web", "api"}
print("web" in unique) # True
```
**Set algebra:**
```python
required = {"auth", "db", "cache"}
installed = {"auth", "db", "logs"}
missing = required - installed # {"cache"}
extra = installed - required # {"logs"}
both = required & installed # {"auth", "db"}
```
**Comprehension and frozenset:**
```python
evens = {n for n in range(10) if n % 2 == 0}
key = frozenset({"a", "b"})
groups: dict[frozenset[str], int] = {key: 1}
```
**Update in place:**
```python
s = {1, 2}
s |= {2, 3} # {1, 2, 3}
s &= {2, 3, 4} # {2, 3}
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `\{\}` creates an empty **dict**, not a set β use `set()`.
* Unhashable elements (`list`, `dict`, `set`) raise `TypeError`.
* `remove` raises `KeyError` if missing; `discard` does not.
* Set equality ignores order; do not rely on iteration order for logic.
* Prefer `frozenset` when you need a set as a dict key or nested in another set.
## π Related [#-related]
* [Lists](/docs/python/lists)
* [Dictionaries](/docs/python/dictionaries)
* [Comprehensions](/docs/python/comprehensions)
* [Types](/docs/python/types)
* [Collections module](/docs/python/collections-module)
* [Truthiness](/docs/python/truthiness)
---
# Shutil (/docs/python/shutil)
# Shutil [#shutil]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`shutil` is high-level file operations: copy, move, delete trees, find executables, and check disk space. Prefer it over hand-rolled recursive delete/copy. Pair with `pathlib` for path construction.
## π§ Core concepts [#-core-concepts]
| API | Role |
| --------------------------- | ----------------------------- |
| `shutil.copy(src, dst)` | Copy file (data + mode) |
| `shutil.copy2(src, dst)` | Copy + metadata |
| `shutil.copytree(src, dst)` | Recursive directory copy |
| `shutil.move(src, dst)` | Move / rename across devices |
| `shutil.rmtree(path)` | Delete directory tree |
| `shutil.which(cmd)` | Resolve executable on `PATH` |
| `shutil.disk_usage(path)` | `total`, `used`, `free` bytes |
| `shutil.make_archive` | Zip/tar helpers |
## π‘ Examples [#-examples]
**Copy and move:**
```python
import shutil
from pathlib import Path
src = Path("report.txt")
shutil.copy(src, "backup/report.txt") # file β file/dir
shutil.copy2(src, "backup/report_meta.txt")
shutil.move("backup/report.txt", "archive/report.txt")
```
**Remove a tree safely:**
```python
import shutil
from pathlib import Path
build = Path("build")
if build.exists():
shutil.rmtree(build)
build.mkdir()
```
**which + disk\_usage:**
```python
import shutil
print(shutil.which("python"), shutil.which("git"))
usage = shutil.disk_usage(".")
print(f"free={usage.free / 1e9:.1f} GB of {usage.total / 1e9:.1f} GB")
```
**copytree with dirs\_exist\_ok (3.8+):**
```python
import shutil
shutil.copytree("templates", "out/templates", dirs_exist_ok=True)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `rmtree` is irreversible β confirm the path; no recycle bin.
* `copy` overwrites existing destination files without asking.
* `copytree` fails if `dst` exists unless `dirs_exist_ok=True`.
* `which` returns `None` if not found β check before `subprocess`.
* On Windows, open files can block `rmtree` / `move`.
## π Related [#-related]
* [Pathlib](/docs/python/pathlib)
* [Files I/O](/docs/python/files-io)
* [Os](/docs/python/os)
* [Tempfile](/docs/python/tempfile)
* [Glob](/docs/python/glob-module)
---
# Slicing (/docs/python/slicing)
# Slicing [#slicing]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Slicing extracts subsequences with `seq[start:stop:step]`. Works on lists, tuples, strings, bytes, and custom sequences implementing `__getitem__`. Slices return a new object for built-in sequences (shallow copy of references).
## π§ Core concepts [#-core-concepts]
| Form | Meaning |
| ---------------- | ----------------------------------- |
| `s[i]` | Single index (negatives from end) |
| `s[a:b]` | From `a` inclusive to `b` exclusive |
| `s[a:b:c]` | Step `c` |
| `s[:]` | Shallow copy (list/tuple/str) |
| `s[::-1]` | Reversed copy |
| `slice(a, b, c)` | Slice object |
| Assign | `xs[a:b] = iterable` (lists) |
Omitted bounds default to ends. Out-of-range slice bounds are clipped; single-index OOB raises `IndexError`.
## π‘ Examples [#-examples]
**Basics:**
```python
xs = ["a", "b", "c", "d", "e"]
print(xs[1:4]) # ['b', 'c', 'd']
print(xs[:2]) # ['a', 'b']
print(xs[::2]) # ['a', 'c', 'e']
print(xs[::-1]) # ['e', 'd', 'c', 'b', 'a']
print(xs[-3:-1]) # ['c', 'd']
```
**Strings and bytes:**
```python
text = "Python"
print(text[1:4]) # "yth"
print(text[::-1]) # "nohtyP"
data = b"\x00\x01\x02\x03"
print(data[1:3]) # b'\x01\x02'
```
**List slice assign / delete:**
```python
nums = [0, 1, 2, 3, 4]
nums[1:3] = [8, 9, 10] # [0, 8, 9, 10, 3, 4]
del nums[2:4] # [0, 8, 3, 4]
nums[::2] = [7, 7] # length must match for extended slice
```
**slice object:**
```python
window = slice(1, 4)
print("abcdef"[window]) # "bcd"
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `xs[:]` is shallow β nested mutables are shared.
* Slice assignment length rules differ for plain vs extended (`step != 1`) slices.
* `s[i:j]` never raises for bad bounds; `s[i]` does.
* Strings/tuples are immutable β cannot slice-assign.
* Copying huge sequences with `[:]` costs memory; prefer views/iterators when possible.
## π Related [#-related]
* [Lists](/docs/python/lists)
* [Strings](/docs/python/strings)
* [Tuples](/docs/python/tuples)
* [Bytes / bytearray](/docs/python/bytes-bytearray)
* [Copy / deepcopy](/docs/python/copy-deepcopy)
* [Comprehensions](/docs/python/comprehensions)
---
# Sqlite3 (/docs/python/sqlite3)
# Sqlite3 [#sqlite3]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`sqlite3` is the stdlib binding to SQLite β a serverless SQL database in a single file (or `:memory:`). Use parameterized queries always. Prefer the connection as a context manager for transactions.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ------------------------------ | ------------------------------------ |
| `sqlite3.connect(path)` | Open DB file / `:memory:` |
| `conn.execute(sql, params)` | Run one statement |
| `conn.executemany(sql, seq)` | Batch inserts/updates |
| `conn.commit()` / `rollback()` | Transactions |
| `conn.cursor()` | Explicit cursor (optional) |
| `with conn:` | Commit on success, rollback on error |
| `row_factory = sqlite3.Row` | Name-based row access |
| `conn.close()` | Release file |
Placeholders: `?` positional or `:name` named β never f-string SQL.
## π‘ Examples [#-examples]
**Connect + create + insert:**
```python
import sqlite3
from pathlib import Path
db = Path("app.db")
with sqlite3.connect(db) as conn:
conn.execute(
"CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT)"
)
conn.execute("INSERT INTO users (name) VALUES (?)", ("Ada",))
# with-block commits on success
```
**Query rows:**
```python
import sqlite3
conn = sqlite3.connect("app.db")
conn.row_factory = sqlite3.Row
try:
rows = conn.execute("SELECT id, name FROM users WHERE name = ?", ("Ada",)).fetchall()
for row in rows:
print(row["id"], row["name"])
finally:
conn.close()
```
**executemany:**
```python
import sqlite3
rows = [("Lin",), ("Grace",), ("Ken",)]
with sqlite3.connect("app.db") as conn:
conn.executemany("INSERT INTO users (name) VALUES (?)", rows)
```
**In-memory + context manager:**
```python
import sqlite3
with sqlite3.connect(":memory:") as conn:
conn.execute("CREATE TABLE t (x INTEGER)")
conn.execute("INSERT INTO t VALUES (1), (2)")
print(conn.execute("SELECT SUM(x) FROM t").fetchone()[0])
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Never interpolate user input into SQL β use `?` / named params.
* Outside `with conn:`, remember `commit()` or changes vanish.
* SQLite locks the DB file; long writes block readers.
* `Row` access needs `row_factory`; default rows are tuples.
* Keep connections short-lived in web apps, or use a pool carefully.
## π Related [#-related]
* [Context managers](/docs/python/context-managers)
* [Pathlib](/docs/python/pathlib)
* [Exceptions](/docs/python/exceptions)
* [Json](/docs/python/json)
---
# Statistics (/docs/python/statistics)
# Statistics [#statistics]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`statistics` is the stdlib module for basic descriptive stats on numeric data. For heavy analysis and DataFrames, prefer [pandas](/docs/python/pandas) / NumPy.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ------------------------ | ------------------------------------------- |
| `mean` / `fmean` | Arithmetic mean (`fmean` faster for floats) |
| `median` / `mode` | Middle value / most common |
| `pstdev` / `stdev` | Population / sample stdev |
| `pvariance` / `variance` | Variance |
| `quantiles` | Cut points |
| `correlation` | Pearson correlation (3.10+) |
## π‘ Examples [#-examples]
```python
from statistics import mean, median, mode, stdev
xs = [2, 4, 4, 4, 5, 5, 7, 9]
print(mean(xs), median(xs), mode(xs))
print(stdev(xs))
```
```python
from statistics import quantiles, correlation
data = [1, 2, 3, 4, 5, 6, 7, 8, 9]
print(quantiles(data, n=4)) # quartiles
print(correlation([1, 2, 3], [2, 4, 6]))
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Empty sequences raise `StatisticsError`.
* `mode` raises if no unique mode (use `multimode`).
* Sample vs population (`stdev` vs `pstdev`) β know which you need.
* Not a substitute for pandas groupby / NumPy vectorized ops on large data.
## π Related [#-related]
* [pandas](/docs/python/pandas)
* [random](/docs/python/random)
* [lists](/docs/python/lists)
* [functools](/docs/python/functools)
---
# Strings (/docs/python/strings)
# Strings [#strings]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`str` is an immutable sequence of Unicode code points. Build strings with f-strings, `join`, and methods like `split`/`strip`/`replace`. Prefer methods and the `str` API over manual index loops. For binary data use `bytes` / `bytearray`.
## π§ Core concepts [#-core-concepts]
**Literals / access**
| Area | Notes |
| ------------- | ------------------------------------------------------------------- |
| Literals | `'...'`, `"..."`, `'''...'''`, `"""..."""`, raw `r"..."`, f-strings |
| Immutability | Methods return **new** strings |
| Index / slice | `s[0]`, `s[-1]`, `s[1:4]`, `s[::-1]` |
| Encode | `s.encode("utf-8")`, `b.decode("utf-8")` |
**Search / test**
| Method | Notes |
| ------------------------------------------------------------------------------ | ---------------------------------- |
| `in` / `find` / `rfind` | Substring; `find` β β1 if missing |
| `index` / `rindex` | Like find; raises `ValueError` |
| `startswith` / `endswith` | Prefix/suffix; tuple of options ok |
| `count(sub[, start[, end]])` | Non-overlapping count |
| `isalnum` / `isalpha` / `isdigit` / `isdecimal` / `isnumeric` | Character class tests |
| `islower` / `isupper` / `istitle` / `isspace` / `isidentifier` / `isprintable` | More predicates |
**Case / strip / pad / align**
| Method | Notes |
| ------------------------------------------------------- | ------------------------------- |
| `lower` / `upper` / `title` / `capitalize` / `swapcase` | Case transforms |
| `casefold()` | Aggressive caseless matching |
| `strip` / `lstrip` / `rstrip` | Trim chars (default whitespace) |
| `removeprefix` / `removesuffix` | Exact affix remove (3.9+) |
| `center` / `ljust` / `rjust` / `zfill` | Pad / zero-pad |
**Split / join / partition / replace**
| Method | Notes |
| --------------------------------------- | --------------------------------- |
| `split(sep=None, maxsplit=-1)` | `None` collapses whitespace |
| `rsplit` / `splitlines(keepends=False)` | From right; by line breaks |
| `partition` / `rpartition` | β `(before, sep, after)` |
| `",".join(parts)` | Join iterable of **str** |
| `replace(old, new, count=-1)` | Replace occurrences |
| `translate(table)` / `maketrans` | Char mapping (fast multi-replace) |
| `format` / `format_map` / f-strings | Interpolation |
Multiline strings keep newlines; use `textwrap.dedent` for indented source blocks.
## π‘ Examples [#-examples]
**Clean and split:**
```python
raw = " Ada, Bob, Cary "
names = [p.strip() for p in raw.split(",")]
print(names) # ['Ada', 'Bob', 'Cary']
print(", ".join(names))
```
**Checks and replace:**
```python
path = "Report.PDF"
print(path.lower().endswith(".pdf"))
print(path.removesuffix(".PDF") + ".pdf") # 3.9+
```
**Slicing and membership:**
```python
s = "python"
print(s[1:4]) # yth
print("th" in s) # True
print(s[::-1]) # nohtyp
```
**Bytes boundary:**
```python
text = "cafΓ©"
data = text.encode("utf-8")
print(data) # b'caf\xc3\xa9'
print(data.decode("utf-8"))
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `str` is immutableβ`s[0] = "X"` is illegal; build a new string.
* `+` in a loop is quadratic for large joinsβuse `"".join(parts)`.
* Default `split()` collapses whitespace; `split(" ")` does not.
* Comparing with `==` is case-sensitive; normalize with `casefold()` for caseless match.
* Mixing `str` and `bytes` raises `TypeError`βdecode/encode explicitly.
## π Related [#-related]
* [F-strings](/docs/python/f-string)
* [Types](/docs/python/types)
* [Loops](/docs/python/loops)
* [Docstrings](/docs/python/docstrings)
* [Dictionaries](/docs/python/dictionaries)
* [Examples: convert files](/docs/python/examples/convert-files)
---
# Subprocess (/docs/python/subprocess)
# Subprocess [#subprocess]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`subprocess` runs external programs. Prefer `subprocess.run(...)` over older `Popen`/`call` helpers for most scripts. Capture output when you need it; raise on failure with `check=True`.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ----------------------------- | ---------------------------------------- |
| `subprocess.run(args, ...)` | Run and wait; returns `CompletedProcess` |
| `capture_output=True` | Capture stdout/stderr as bytes |
| `text=True` / `encoding=` | Decode streams as str |
| `check=True` | Raise `CalledProcessError` on non-zero |
| `cwd=` / `env=` | Working dir / environment |
| `timeout=` | Kill if too slow |
| `shell=True` | Run via shell (usually avoid) |
| `CompletedProcess.returncode` | Exit status |
Pass args as a **list** unless you truly need shell features.
## π‘ Examples [#-examples]
**Basic run:**
```python
import subprocess
r = subprocess.run(["python", "-c", "print(1+1)"], capture_output=True, text=True)
print(r.returncode, r.stdout.strip())
```
**check + capture:**
```python
import subprocess
try:
r = subprocess.run(
["git", "rev-parse", "HEAD"],
capture_output=True,
text=True,
check=True,
)
print(r.stdout.strip())
except subprocess.CalledProcessError as e:
print("failed:", e.returncode, e.stderr)
```
**cwd and env:**
```python
import os
import subprocess
env = {**os.environ, "MY_FLAG": "1"}
subprocess.run(["ls"], cwd="/tmp", env=env, check=True)
```
**Avoid shell when possible:**
```python
import subprocess
# Good: list form, no shell
subprocess.run(["echo", "hello"], check=True)
# Risky: shell=True with untrusted input β injection
# subprocess.run(f"echo {user}", shell=True) # DON'T
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `shell=True` + unsanitized input enables command injection β use argument lists.
* Without `capture_output` / `stdout=PIPE`, output goes to the console.
* Forgotten `text=True` leaves `bytes` in `.stdout`.
* `check=False` (default) hides failures β set `check=True` or inspect `returncode`.
* Deadlocks are rare with `run`; with raw `Popen` pipes, always drain or use `communicate`.
## π Related [#-related]
* [Sys](/docs/python/sys)
* [Os](/docs/python/os)
* [Shutil](/docs/python/shutil)
* [Argparse](/docs/python/argparse)
* [Exceptions](/docs/python/exceptions)
---
# Sys (/docs/python/sys)
# Sys [#sys]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`sys` exposes interpreter and process plumbing: CLI args, module search path, exit codes, and standard streams. Use it for lightweight CLIs; prefer `argparse` when flags grow.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ---------------------------------- | ------------------------------------- |
| `sys.argv` | CLI args: `[script, ...]` |
| `sys.path` | Module import search list |
| `sys.exit(code)` | Exit process (`0` ok, non-zero error) |
| `sys.stdin` / `stdout` / `stderr` | Text streams |
| `sys.version` / `sys.version_info` | Runtime version |
| `sys.platform` | `win32`, `linux`, `darwin`, β¦ |
| `sys.executable` | Path to this Python binary |
| `sys.getsizeof(obj)` | Approx. object size (bytes) |
## π‘ Examples [#-examples]
**argv:**
```python
import sys
# python tool.py a b
script, *args = sys.argv
print("script:", script)
print("args:", args)
if not args:
sys.exit("usage: tool.py ")
```
**path and version:**
```python
import sys
from pathlib import Path
print(sys.version_info[:3])
print(sys.executable)
# Add local package root for scripts (prefer installable packages):
root = Path(__file__).resolve().parent
if str(root) not in sys.path:
sys.path.insert(0, str(root))
```
**stdin / stdout:**
```python
import sys
for line in sys.stdin:
sys.stdout.write(line.upper())
# echo hello | python upper.py
```
**exit codes:**
```python
import sys
ok = True
sys.exit(0 if ok else 1)
# or: raise SystemExit(2)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `sys.argv[0]` is the script path (or `-c` / `-m`), not always a useful name.
* Mutating `sys.path` is a last resort β prefer packages / `PYTHONPATH` / editable installs.
* `sys.exit("msg")` prints to stderr and exits with code `1`.
* Redirected stdout may be block-buffered β flush or use `print(..., flush=True)`.
* Donβt compare versions as strings; use `sys.version_info >= (3, 11)`.
## π Related [#-related]
* [Argparse](/docs/python/argparse)
* [Os](/docs/python/os)
* [Subprocess](/docs/python/subprocess)
* [Print / input](/docs/python/print-input)
* [Import / export](/docs/python/import-export)
---
# Tempfile (/docs/python/tempfile)
# Tempfile [#tempfile]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`tempfile` creates temporary files and directories that are unique and (usually) cleaned up automatically. Prefer context managers (`TemporaryDirectory`, `NamedTemporaryFile`) so cleanup happens even on errors.
## π§ Core concepts [#-core-concepts]
| API | Role |
| ------------------------------ | --------------------------------- |
| `TemporaryDirectory()` | Temp dir; deleted on exit |
| `NamedTemporaryFile()` | Named temp file on disk |
| `TemporaryFile()` | Anon file (may lack usable name) |
| `mkstemp()` / `mkdtemp()` | Low-level create (manual cleanup) |
| `gettempdir()` | System temp directory |
| `suffix=` / `prefix=` / `dir=` | Naming / location |
On Windows, `NamedTemporaryFile(delete=True)` may block reopening by name while still open β use `delete=False` and unlink yourself when needed.
## π‘ Examples [#-examples]
**TemporaryDirectory:**
```python
from pathlib import Path
from tempfile import TemporaryDirectory
with TemporaryDirectory(prefix="job-") as tmp:
root = Path(tmp)
(root / "out.txt").write_text("ok", encoding="utf-8")
print(list(root.iterdir()))
# directory is gone here
```
**NamedTemporaryFile:**
```python
from tempfile import NamedTemporaryFile
with NamedTemporaryFile("w+", suffix=".csv", delete=True, encoding="utf-8") as f:
f.write("a,b\n1,2\n")
f.flush()
print(f.name)
f.seek(0)
print(f.read())
```
**delete=False for external tools:**
```python
from pathlib import Path
from tempfile import NamedTemporaryFile
with NamedTemporaryFile("w", suffix=".txt", delete=False, encoding="utf-8") as f:
path = Path(f.name)
f.write("payload")
try:
print(path.read_text(encoding="utf-8"))
finally:
path.unlink(missing_ok=True)
```
**gettempdir:**
```python
import tempfile
from pathlib import Path
print(Path(tempfile.gettempdir()))
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Temp dirs vanish when the context exits β copy results out first.
* Windows: canβt always reopen a still-open `NamedTemporaryFile` by name.
* `mkstemp` returns an open fd you must close and a path you must delete.
* Secure defaults matter for secrets; avoid world-readable custom `dir=`.
* Donβt assume temp paths persist across reboots or process restarts.
## π Related [#-related]
* [Pathlib](/docs/python/pathlib)
* [Files I/O](/docs/python/files-io)
* [Shutil](/docs/python/shutil)
* [Context managers](/docs/python/context-managers)
* [Os](/docs/python/os)
---
# Testing with pytest (/docs/python/testing-pytest)
# Testing with pytest [#testing-with-pytest]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`pytest` discovers and runs tests with plain `assert`, fixtures, parametrization, and rich failure diffs. Prefer it over `unittest` for new projects. Name files `test_*.py` or `*_test.py`.
## π§ Core concepts [#-core-concepts]
| Feature | Role |
| ------------- | ------------------------------------ |
| Discovery | `test_*` functions/classes |
| `assert` | Rewritten with introspection |
| Fixtures | Setup/teardown via `@pytest.fixture` |
| `parametrize` | Data-driven cases |
| Markers | `@pytest.mark.skip`, `slow`, etc. |
| `raises` | Expect exceptions |
| `tmp_path` | Temp directory fixture |
| Plugins | `pytest-cov`, `pytest-asyncio`, β¦ |
Run with `pytest` or `python -m pytest`. Exit code 0 = all passed.
## π‘ Examples [#-examples]
**Basic tests:**
```python
# test_mathutil.py
def add(a: int, b: int) -> int:
return a + b
def test_add():
assert add(2, 3) == 5
def test_add_negative():
assert add(-1, 1) == 0
```
**Fixtures and tmp\_path:**
```python
import pytest
from pathlib import Path
@pytest.fixture
def sample_file(tmp_path: Path) -> Path:
p = tmp_path / "data.txt"
p.write_text("hello", encoding="utf-8")
return p
def test_read(sample_file: Path):
assert sample_file.read_text(encoding="utf-8") == "hello"
```
**Parametrize and raises:**
```python
import pytest
@pytest.mark.parametrize("n,expected", [(0, 0), (1, 1), (2, 4)])
def test_square(n: int, expected: int):
assert n * n == expected
def test_div_zero():
with pytest.raises(ZeroDivisionError):
_ = 1 / 0
```
**CLI:**
```shellscript
pytest -q
pytest -k "add" --maxfail=1
pytest --cov=mypkg --cov-report=term-missing
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Tests that depend on execution order or shared global state flake.
* Overusing mocks hides real integration bugs β mix unit + integration.
* Mutable fixture state shared across tests when scope is wrong.
* Asserting on exact error strings is brittle β match exception type.
* Don't commit secrets into test fixtures.
## π Related [#-related]
* [Examples: pytest example](/docs/python/examples/pytest-example)
* [Exceptions](/docs/python/exceptions)
* [Venv](/docs/python/venv)
* [Pip](/docs/python/pip)
* [Functions](/docs/python/functions)
* [Packaging](/docs/python/packaging)
---
# Truthiness (/docs/python/truthiness)
# Truthiness [#truthiness]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
In boolean contexts (`if`, `while`, `and`/`or`, `bool(x)`), objects are truthy or falsy. Prefer explicit checks (`is None`, `len`, `==`) when zero/empty must be distinguished from missing.
## π§ Core concepts [#-core-concepts]
| Falsy | Examples |
| ---------------- | ---------------------------------------- |
| Constants | `None`, `False` |
| Numbers | `0`, `0.0`, `0j` |
| Empty containers | `""`, `[]`, `()`, `\{\}`, `set()` |
| Empty binary | `b""`, `bytearray()` |
| Custom | `__bool__` β `False`, or `__len__` β `0` |
Everything else is truthy by default β including `"0"`, `[0]`, and custom objects.
## π‘ Examples [#-examples]
**Typical checks:**
```python
name = ""
items: list[int] = []
if not name:
print("missing name")
if items:
print("has items")
```
**None vs empty:**
```python
def title(value: str | None) -> str:
if value is None:
return "default"
if not value:
return "empty"
return value
print(title(None), title(""), title("Hi"))
```
**and / or return operands:**
```python
print("" or "fallback") # "fallback"
print("hi" or "fallback") # "hi"
print("hi" and "there") # "there"
user = None
print((user or {}).get("id"))
```
**Custom bool:**
```python
class Gate:
def __init__(self, open_: bool) -> None:
self.open = open_
def __bool__(self) -> bool:
return self.open
g = Gate(False)
print(bool(g), "yes" if g else "no")
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `if xs` is False for empty lists β OK for "any items?", wrong for "was provided?".
* `if count:` fails for `0` even when zero is valid.
* NumPy arrays: truth value is ambiguous β use `.any()` / `.all()`.
* Don't use `== True` / `== False`; use `is True` only for actual bools, else rely on truthiness carefully.
* `__bool__` and `__len__` must be consistent if both exist.
## π Related [#-related]
* [Conditionals](/docs/python/conditionals)
* [Operators](/docs/python/operators)
* [Lists](/docs/python/lists)
* [Dictionaries](/docs/python/dictionaries)
* [Magic methods](/docs/python/magic-methods)
* [Types](/docs/python/types)
---
# Tuples (/docs/python/tuples)
# Tuples [#tuples]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
A `tuple` is an ordered, immutable sequence. Use for fixed records, multiple return values, and hashable keys (when all elements are hashable). Prefer tuples over lists when the length and contents should not change.
## π§ Core concepts [#-core-concepts]
| Operation | Example |
| --------------- | --------------------------------------------- |
| Create | `(1, 2)`, `1, 2`, `tuple(iterable)`, `()` |
| Single element | `(1,)` β trailing comma required |
| Index / slice | `t[0]`, `t[1:]` |
| Unpack | `a, b = t`, `a, *rest = t` |
| Concat / repeat | `t + u`, `t * n` (new tuples) |
| Named | `collections.namedtuple`, `typing.NamedTuple` |
Immutability means you cannot assign `t[i] = x` or `append`. Nested mutable objects inside a tuple can still mutate.
## π‘ Examples [#-examples]
**Returns and unpacking:**
```python
def divmod_pair(a: int, b: int) -> tuple[int, int]:
return a // b, a % b
q, r = divmod_pair(17, 5)
```
**As dict keys:**
```python
point = (3, 4)
grid: dict[tuple[int, int], str] = {point: "occupied"}
print(grid[(3, 4)])
```
**NamedTuple (typed record):**
```python
from typing import NamedTuple
class User(NamedTuple):
id: int
name: str
active: bool = True
u = User(1, "Ada")
print(u.name, u._asdict())
```
**Tuple vs list choice:**
```python
coords = (10.0, 20.0) # fixed pair β tuple
history: list[float] = [] # grows over time β list
history.append(1.5)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `(1)` is just `int` 1 β use `(1,)` for a one-element tuple.
* Tuples containing lists/dicts are not hashable.
* `t += (x,)` rebinds `t` to a new tuple (ok for local names; confusing for aliases).
* Prefer `NamedTuple` / `dataclass(frozen=True)` over anonymous long tuples for clarity.
* Slicing a tuple returns a tuple, not a list.
## π Related [#-related]
* [Lists](/docs/python/lists)
* [Sets](/docs/python/sets)
* [Dataclasses](/docs/python/dataclasses)
* [Unpacking](/docs/python/unpacking)
* [Types](/docs/python/types)
* [Dictionaries](/docs/python/dictionaries)
---
# Types (/docs/python/types)
# Types [#types]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Python is dynamically typed at runtime and optionally statically checked via type annotations (PEP 484+). Use built-in generics (`list[str]`, `dict[str, int]`), `typing` utilities, and `|` unions (3.10+). Annotations document APIs and enable mypy/pyrightβthey do not enforce types at runtime unless you add a validator.
## π§ Core concepts [#-core-concepts]
| Kind | Examples |
| ---------------- | ------------------------------------------------ |
| Built-ins | `int`, `float`, `bool`, `str`, `bytes`, `None` |
| Collections | `list[T]`, `dict[K, V]`, `set[T]`, `tuple[A, B]` |
| Union / optional | `int \| None`, `str \| int` |
| Callables | `Callable[[int, str], bool]` |
| Protocols | Structural typing (`typing.Protocol`) |
| Aliases | `type UserId = int` (3.12+) or `TypeAlias` |
| Generics | `class Box[T]: ...` (3.12+) or `TypeVar` |
`isinstance` / `issubclass` for runtime checks; prefer EAFP with care for public boundaries.
## π‘ Examples [#-examples]
**Annotations and unions:**
```python
def parse_id(value: str | int) -> int:
return int(value)
def find(name: str) -> str | None:
return name if name else None
```
**Collections and TypedDict:**
```python
from typing import TypedDict
class User(TypedDict):
id: int
name: str
active: bool
users: list[User] = [{"id": 1, "name": "Ada", "active": True}]
```
**Protocol (duck typing):**
```python
from typing import Protocol
class Closeable(Protocol):
def close(self) -> None: ...
def shutdown(resource: Closeable) -> None:
resource.close()
```
**Runtime check helpers:**
```python
from collections.abc import Iterable
def total(xs: Iterable[int]) -> int:
return sum(xs)
assert isinstance(True, int) # bool is a subclass of int
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Annotations are not runtime-enforced: `def f(x: int): ...` still accepts any object.
* Mutable default arguments (`def f(xs=[])`) share one listβuse `None` + create inside.
* `list` vs `List` from `typing`: prefer built-ins on 3.9+.
* `tuple[int]` is a one-int tuple; `tuple[int, ...]` is variable length.
* Over-annotating trivial locals adds noise; focus on public functions and complex structures.
## π Related [#-related]
* [Dictionaries](/docs/python/dictionaries)
* [Docstrings](/docs/python/docstrings)
* [\*args and \*\*kwargs](/docs/python/kwags)
* [Strings](/docs/python/strings)
* [Import / export](/docs/python/import-export)
* [Async / await](/docs/python/async)
---
# Typing Hints (/docs/python/typing-hints)
# Typing Hints [#typing-hints]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Type hints document and check interfaces with tools like mypy/pyright. They are not enforced at runtime by default. Prefer built-in generics (`list[int]`) on 3.9+ and `|` unions on 3.10+.
## π§ Core concepts [#-core-concepts]
| Feature | Example |
| ------------------- | ------------------------------------ |
| Built-in generics | `list[int]`, `dict[str, int]` |
| Union | `int \| None`, `str \| bytes` |
| Optional | `X \| None` (prefer over `Optional`) |
| Callable | `Callable[[int], str]` |
| TypeAlias | `type Vector = list[float]` (3.12+) |
| TypeVar / ParamSpec | Generics |
| Protocol | Structural typing |
| TypedDict | Dict shapes |
| `typing.cast` | Escape hatch for checkers |
| `typing.Any` | Opt out (use sparingly) |
From 3.11+: `Self`, `LiteralString`, improved error types. From 3.12+: `type` statement, PEP 695 generics.
## π‘ Examples [#-examples]
**Functions and collections:**
```python
from collections.abc import Iterable, Sequence
def average(xs: Sequence[float]) -> float:
return sum(xs) / len(xs)
def flatten(rows: Iterable[Iterable[int]]) -> list[int]:
return [n for row in rows for n in row]
```
**Aliases and TypedDict:**
```python
from typing import TypedDict
type UserId = int # 3.12+; else TypeAlias
class User(TypedDict):
id: UserId
name: str
active: bool
def label(u: User) -> str:
return f'{u["name"]}#{u["id"]}'
```
**Generics (3.12+):**
```python
def first[T](items: Sequence[T]) -> T | None:
return items[0] if items else None
```
**Protocol:**
```python
from typing import Protocol
class Closable(Protocol):
def close(self) -> None: ...
def shutdown(resource: Closable) -> None:
resource.close()
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Hints are ignored at runtime unless you validate (pydantic, beartype, etc.).
* Use `collections.abc` for `Iterable`/`Mapping`/`Sequence`, not concrete lists only.
* `list` is invariant β `list[int]` is not a `list[float | int]` for checkers.
* Avoid `Any` sprawl; prefer `object` or precise unions.
* Forward refs: quote names or `from __future__ import annotations`.
## π Related [#-related]
* [Functions](/docs/python/functions)
* [ABC / protocols](/docs/python/abc-protocols)
* [Dataclasses](/docs/python/dataclasses)
* [Classes](/docs/python/classes)
* [Types](/docs/python/types)
* [Docstrings](/docs/python/docstrings)
---
# Unpacking (/docs/python/unpacking)
# Unpacking [#unpacking]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
Unpacking assigns elements of iterables (and keys/values of mappings with `**`) into names or containers. Covers tuple unpacking, starred targets, function call unpacking, and dict merges. Related to but broader than [\*args/\*\*kwargs](/docs/python/kwags).
## π§ Core concepts [#-core-concepts]
| Form | Role |
| ------------------ | ------------------------------ |
| `a, b = pair` | Sequence unpack |
| `a, *rest, z = xs` | Starred target |
| `*args` in call | Unpack iterable as positionals |
| `**kwargs` in call | Unpack mapping as keywords |
| `\{**a, **b\}` | Dict merge (right wins) |
| `[*a, *b]` | List concat via unpack |
| Nested | ` (a, (b, c)) = ...` |
Length must match unless a starred target absorbs extras. `strict=True` on `zip` helps parallel unpack.
## π‘ Examples [#-examples]
**Basics and star:**
```python
first, second = (10, 20)
head, *mid, tail = [1, 2, 3, 4, 5]
# head=1, mid=[2,3,4], tail=5
```
**Swap and nested:**
```python
a, b = 1, 2
a, b = b, a
row = (1, (2, 3))
i, (j, k) = row
```
**Call site unpacking:**
```python
def greeter(name: str, city: str, *titles: str) -> str:
extra = " ".join(titles)
return f"{name} @ {city} {extra}".strip()
args = ("Ada", "London")
opts = {"titles": ("Dr",)} # won't work as ** for *titles
print(greeter(*args, "Dr"))
vals = [1, 2, 3]
print(max(*vals))
```
**Dict / list merges:**
```python
defaults = {"host": "localhost", "port": 8000}
user = {"port": 9000}
cfg = {**defaults, **user} # port=9000
a, b = [1, 2], [3, 4]
merged = [*a, *b] # [1, 2, 3, 4]
```
**For-loop unpack:**
```python
pairs = [("a", 1), ("b", 2)]
for key, value in pairs:
print(key, value)
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Too many / too few values raises `ValueError`.
* `**` keys must be strings for function calls.
* Later `**` wins on key clashes β order matters.
* Starred expression in assignment: only one `*` target allowed.
* Unpacking generators consumes them.
## π Related [#-related]
* [\*args and \*\*kwargs](/docs/python/kwags)
* [Tuples](/docs/python/tuples)
* [Lists](/docs/python/lists)
* [Dictionaries](/docs/python/dictionaries)
* [Functions](/docs/python/functions)
* [Loops](/docs/python/loops)
---
# Urllib.parse (/docs/python/urllib-parse)
# Urllib.parse [#urllibparse]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`urllib.parse` splits and builds URLs: parse components, encode query strings, and percent-encode path segments. Stdlib β no install. Pair with `urllib.request` or [Requests](/docs/python/requests-http) for HTTP.
## π§ Core concepts [#-core-concepts]
| API | Role |
| -------------------------- | -------------------------------------- |
| `urlparse(url)` | Split into scheme/netloc/path/β¦ |
| `urlunparse(parts)` | Rebuild from parts |
| `urlsplit` / `urlunsplit` | Similar; path keeps params differently |
| `parse_qs` / `parse_qsl` | Query string β dict / list of pairs |
| `urlencode(query)` | Dict/pairs β query string |
| `quote(s)` / `quote_plus` | Percent-encode (path vs form) |
| `unquote` / `unquote_plus` | Decode |
| `urljoin(base, url)` | Resolve relative URLs |
`ParseResult` fields: `scheme`, `netloc`, `path`, `params`, `query`, `fragment`.
## π‘ Examples [#-examples]
**urlparse:**
```python
from urllib.parse import urlparse
u = urlparse("https://example.com:443/a/b?x=1#top")
print(u.scheme, u.hostname, u.port, u.path, u.query, u.fragment)
```
**urlencode + quote:**
```python
from urllib.parse import urlencode, quote, urljoin
qs = urlencode({"q": "hello world", "lang": "en"})
print(qs) # q=hello+world&lang=en
path = "/docs/" + quote("C++ tips")
print(urljoin("https://example.com", path))
```
**parse query:**
```python
from urllib.parse import parse_qs, urlsplit
parts = urlsplit("https://x.test/search?q=a&q=b&page=2")
print(parse_qs(parts.query)) # {'q': ['a', 'b'], 'page': ['2']}
```
**Rebuild URL:**
```python
from urllib.parse import urlparse, urlunparse, urlencode
p = urlparse("https://api.example.com/v1/items")
query = urlencode({"limit": 10, "tag": "py"})
print(urlunparse(p._replace(query=query)))
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `quote` vs `quote_plus`: `+` for form bodies; `%20` often for paths.
* `parse_qs` returns **lists** of values β use `[0]` carefully.
* Donβt assemble URLs with raw f-strings for user input β encode first.
* `urljoin` replaces the path when the second arg starts with `/`.
* IPv6 / unusual URLs: prefer `urlsplit` and inspect `netloc` carefully.
## π Related [#-related]
* [Requests](/docs/python/requests-http)
* [Json](/docs/python/json)
* [Strings](/docs/python/strings)
* [Regex](/docs/python/regex)
---
# Variables (/docs/python/variables)
# Variables [#variables]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
A variable is a name that refers to a value in memory. Python creates the binding when you assign with `=`. You do not declare types up front (though type hints are optional).
## π§ Core concepts [#-core-concepts]
| Idea | Meaning |
| -------------- | ----------------------------------------------- |
| Assignment | `name = value` binds a name to an object |
| Rebinding | `name = other` points the name at a new object |
| Dynamic typing | The same name can later hold a different type |
| Naming | Letters, digits, `_`; cannot start with a digit |
| Convention | `snake_case` for variables and functions |
Common built-in types beginners meet first: `int`, `float`, `str`, `bool`, `list`, `dict`, `None`.
## π‘ Examples [#-examples]
**Create and use variables:**
```python
age = 30
name = "Sam"
height_m = 1.75
is_student = True
print(name, age)
print(f"{name} is {age} years old")
```
**Rebind and swap:**
```python
x = 10
x = x + 5 # now 15
x += 1 # now 16
a, b = 1, 2
a, b = b, a # swap
print(a, b) # 2 1
```
**Multiple assignment and unpacking:**
```python
city, country = "Lisbon", "Portugal"
first, *rest = [10, 20, 30, 40]
print(first, rest) # 10 [20, 30, 40]
```
**Names are references:**
```python
nums = [1, 2, 3]
alias = nums
alias.append(4)
print(nums) # [1, 2, 3, 4] β same list object
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `=` is assignment; `==` is comparison.
* Using a name before assignment raises `NameError`.
* Mutable defaults and shared list/dict aliases surprise beginners.
* Avoid names that shadow builtins: `list`, `str`, `id`, `type`, `input`.
## π Related [#-related]
* [hello\_world.md](/docs/python/hello-world)
* [comments.md](/docs/python/comments)
* [none\_and\_truthy.md](/docs/python/none-and-truthy)
* [scope\_namespaces.md](/docs/python/scope-namespaces)
---
# Venv (/docs/python/venv)
# Venv [#venv]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`venv` creates isolated Python environments so project dependencies do not clash with system packages. Always use a virtualenv per project; activate it before installing with pip.
## π§ Core concepts [#-core-concepts]
| Task | Command |
| ------------------ | ------------------------------------------- |
| Create | `python -m venv .venv` |
| Activate (Windows) | `.venv\Scripts\activate` |
| Activate (Unix) | `source .venv/bin/activate` |
| Deactivate | `deactivate` |
| Install | `python -m pip install -r requirements.txt` |
| Record | `python -m pip freeze > requirements.txt` |
| Remove | Delete the `.venv` folder |
Prefer `python -m pip` so you install into the active interpreter. `.venv` is the common directory name (gitignored).
## π‘ Examples [#-examples]
**Create and use (PowerShell):**
```powershell
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install requests pytest
```
**Create and use (bash):**
```shellscript
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -U pip
python -m pip install -r requirements.txt
```
**Check which Python:**
```shellscript
python -c "import sys; print(sys.executable)"
where python # Windows
which python # Unix
```
**requirements pinning:**
```plaintext
# requirements.txt
requests==2.32.3
pytest>=8.0,<9
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Installing with a global `pip` while thinking the venv is active β verify `sys.executable`.
* Committing `.venv` to git bloats the repo β ignore it.
* Mixing conda and venv accidentally β pick one workflow.
* Nested venvs and path confusion after IDE interpreter switches.
* On Windows, execution policy may block `Activate.ps1` β adjust policy or use `cmd` activate.
## π Related [#-related]
* [Pip](/docs/python/pip)
* [Packaging](/docs/python/packaging)
* [Import / export](/docs/python/import-export)
* [Testing with pytest](/docs/python/testing-pytest)
* [Examples: cli tool](/docs/python/examples/cli-tool)
* [Argparse](/docs/python/argparse)
---
# Walrus Operator (/docs/python/walrus)
# Walrus Operator [#walrus-operator]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
The walrus operator `:=` (PEP 572, 3.8+) assigns inside an expression. Use it to avoid double computation or awkward control-flow temporaries β sparingly, for clarity.
## π§ Core concepts [#-core-concepts]
| Form | Meaning |
| ----------------- | --------------------------------------------- |
| `name := expr` | Assign and yield `expr` |
| In `if` / `while` | Bind then test |
| In comprehensions | Bind in filter/value |
| Scope | Same rules as normal assignment in that block |
Cannot replace plain statement assignment everywhere; illegal in some syntactic positions (e.g. unparenthesized in some tuple contexts historically β prefer parentheses when unsure).
## π‘ Examples [#-examples]
**if / while:**
```python
from pathlib import Path
path = Path("data.txt")
if (text := path.read_text(encoding="utf-8")).strip():
print(text[:20])
while (line := input("cmd> ")) != "quit":
print("got", line)
```
**Reuse match result:**
```python
import re
pattern = re.compile(r"(\d+)")
if m := pattern.search("id=42"):
print(int(m.group(1)))
```
**Comprehension:**
```python
def costly(x: int) -> int:
return x * x
nums = [1, 2, 3, 4]
# compute once per item
big = [y for x in nums if (y := costly(x)) > 5]
print(big) # [9, 16]
```
**Chunked read:**
```python
from pathlib import Path
with Path("blob.bin").open("rb") as f:
while chunk := f.read(4096):
print(len(chunk))
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* Overuse hurts readability β if the name is important, use a normal assignment.
* In comprehensions, the bound name's scope can surprise readers (leaks in some contexts historically; in 3.x comps have isolated scopes for `for` targets, but `:=` targets in the comprehension scope).
* Don't hide side effects inside walrus expressions.
* Not valid as a standalone replacement for all `=` uses.
* Combine carefully with `and`/`or` β parentheses help.
## π Related [#-related]
* [Conditionals](/docs/python/conditionals)
* [Loops](/docs/python/loops)
* [Comprehensions](/docs/python/comprehensions)
* [Files I/O](/docs/python/files-io)
* [Regex](/docs/python/regex)
* [Operators](/docs/python/operators)
---
# Zip, Enumerate, Map, Filter (/docs/python/zip-enumerate-map-filter)
# Zip, Enumerate, Map, Filter [#zip-enumerate-map-filter]
*Python Β· Reference cheat sheet*
***
## π Overview [#-overview]
`enumerate`, `zip`, `map`, and `filter` are built-in iteration helpers. Prefer `enumerate`/`zip` and comprehensions in idiomatic Python; `map`/`filter` shine with existing callables.
## π§ Core concepts [#-core-concepts]
| Built-in | Role |
| ------------------------- | ----------------------------------- |
| `enumerate(it, start=0)` | Index + value pairs |
| `zip(*its, strict=False)` | Parallel iteration (3.10+ `strict`) |
| `map(fn, it, ...)` | Apply fn lazily |
| `filter(pred, it)` | Keep truthy pred |
| `reversed` / `sorted` | Related helpers |
All return iterators (except `sorted` β list). Consume once unless recreated.
## π‘ Examples [#-examples]
**enumerate and zip:**
```python
names = ["ada", "bob", "cy"]
scores = [9, 7, 8]
for i, name in enumerate(names, start=1):
print(i, name)
for name, score in zip(names, scores, strict=True):
print(f"{name}: {score}")
```
**unzip:**
```python
pairs = [("a", 1), ("b", 2)]
letters, nums = zip(*pairs)
print(letters, nums)
```
**map / filter vs comps:**
```python
nums = [1, 2, 3, 4]
print(list(map(str, nums)))
print(list(filter(lambda n: n % 2 == 0, nums)))
# Prefer:
print([str(n) for n in nums])
print([n for n in nums if n % 2 == 0])
```
**dict from parallel lists:**
```python
keys = ["host", "port"]
vals = ["localhost", "8000"]
cfg = dict(zip(keys, vals, strict=True))
```
## β οΈ Pitfalls [#οΈ-pitfalls]
* `zip` stops at the shortest iterable unless `strict=True` (then errors on mismatch).
* `map`/`filter` are lazy β wrap `list()` for reuse/debug.
* `enumerate` on a dict iterates keys β use `.items()` when you need pairs.
* Star-unzip `zip(*xs)` fails on empty `xs`.
* Heavy `lambda` with map/filter is usually clearer as a comprehension.
## π Related [#-related]
* [Loops](/docs/python/loops)
* [Comprehensions](/docs/python/comprehensions)
* [Lambda](/docs/python/lambda)
* [Iterators](/docs/python/iterators)
* [Unpacking](/docs/python/unpacking)
* [Examples: list to dict](/docs/python/examples/list-to-dict)