Pickle

Python's native object serialization format — able to save almost any Python object to bytes and reconstruct it, with a security warning that must never be ignored.

pickle serializes Python objects into a byte stream ("pickling") and reconstructs them later ("unpickling") — including custom class instances, nested structures, and object graphs with shared or circular references, none of which json can handle directly. That power comes with a serious catch: unpickling data is equivalent to running arbitrary code, so pickle must only ever be used with sources you fully trust.

What Is Pickle and Why It Exists

pickle is Python's native, general-purpose object serialization format. Unlike json, which only understands a small fixed set of JSON-compatible types, pickle can serialize nearly any Python object — including custom classes, functions (by reference, not their code), and complex structures with shared references — because it works at the level of Python's own object model rather than a language-neutral text format.

import pickle

data = {"name": "Ada", "scores": [90, 85, 78], "active": True}

blob = pickle.dumps(data)   # bytes, not a human-readable string
print(type(blob), blob[:15])

restored = pickle.loads(blob)
print(restored == data)     # True — full round trip

Basic Syntax: dumps/loads and dump/load

The API mirrors json's naming: pickle.dumps(obj) returns bytes; pickle.loads(data) reconstructs an object from bytes. pickle.dump(obj, fp) and pickle.load(fp) work with an already-open binary file object ("wb"/"rb" mode — pickle output is always binary, never text).

Pickle files must always be opened in binary mode ("wb"/"rb"). Opening in text mode raises TypeError because pickle's output is a byte stream, not decodable text.

import pickle

data = {"name": "Ada", "scores": [90, 85, 78]}

with open("data.pkl", "wb") as f:
    pickle.dump(data, f)

with open("data.pkl", "rb") as f:
    restored = pickle.load(f)

print(restored)

The Security Warning: Never Unpickle Untrusted Data

This is the single most important fact about pickle: unpickling data can execute arbitrary code. The pickle format includes opcodes that can call functions with attacker-controlled arguments during reconstruction — a maliciously crafted pickle byte stream can run any code the attacker wants the moment pickle.load()/pickle.loads() processes it, with no special exploit needed beyond calling load(). Never unpickle data received from an untrusted source: user uploads, network input from outside your control, or any data whose provenance you can't fully vouch for.

For data interchange with anything outside your own fully-trusted process boundary (a web API, a config file, cross-language communication), use json instead — it can only produce plain data structures, with no mechanism to execute code during parsing.

# NEVER do this with data from an untrusted source (uploaded file,
# network request, message queue you don't fully control, etc.):
import pickle
data = pickle.loads(untrusted_bytes_from_the_internet)   # can execute arbitrary code

When Pickle Is the Right Tool

Pickle is appropriate for trusted, internal use cases: caching computed Python objects between runs of your own program, saving/restoring application state you fully control, or passing objects between processes you control (e.g. via multiprocessing, which uses pickle internally to send data between worker processes). It is not appropriate as a data-interchange format across trust boundaries or between different systems/languages.

import pickle
from pathlib import Path

def load_or_compute_cache(cache_path: Path, compute_fn):
    if cache_path.exists():
        with cache_path.open("rb") as f:
            return pickle.load(f)   # trusted: only our own process wrote this file
    result = compute_fn()
    with cache_path.open("wb") as f:
        pickle.dump(result, f)
    return result

Pickling Custom Objects

Instances of user-defined classes pickle by default as long as their class is importable (by module path) from wherever the pickle is later loaded, and their attributes are themselves picklable. Pickle stores the class reference plus the instance's __dict__ (or the result of __getstate__ if defined) — it does not store the class's code itself.

Because pickle stores a reference to the class by its module path, Point must be importable under the same name from the same module when unpickling — pickling an object defined in __main__ (e.g. directly in an interactive session or top-level script) and loading it from a different module will fail with AttributeError or PicklingError.

import pickle

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __repr__(self):
        return f"Point({self.x}, {self.y})"

p = Point(3, 4)
blob = pickle.dumps(p)
restored = pickle.loads(blob)
print(restored)   # Point(3, 4)

Customizing Pickling with getstate/setstate

For objects holding unpicklable state (open file handles, network connections, locks) or requiring custom reconstruction logic, define __getstate__ (returns what to pickle) and __setstate__ (rebuilds the object from that state on unpickling).

import pickle

class Connection:
    def __init__(self, host, port):
        self.host = host
        self.port = port
        self.socket = self._connect()   # not picklable

    def _connect(self):
        return object()   # stand-in for a real, unpicklable connection object

    def __getstate__(self):
        state = self.__dict__.copy()
        del state["socket"]             # exclude the unpicklable part
        return state

    def __setstate__(self, state):
        self.__dict__.update(state)
        self.socket = self._connect()   # reconnect after unpickling

conn = Connection("localhost", 5432)
restored = pickle.loads(pickle.dumps(conn))
print(restored.host, restored.port)     # localhost 5432 — reconnected fresh

What Cannot Be Pickled

Open file handles, database connections, sockets, lambdas (unlike named functions, which pickle by reference), and locally-defined nested functions/classes generally cannot be pickled directly, because pickle needs to be able to look them up by a stable, importable name on the loading side. Attempting to pickle one of these raises pickle.PicklingError (or TypeError/AttributeError depending on the specific case).

import pickle

add_one = lambda x: x + 1
try:
    pickle.dumps(add_one)
except (pickle.PicklingError, AttributeError, TypeError) as e:
    print(f"Cannot pickle a lambda: {e}")

def real_function(x):
    return x + 1

pickle.dumps(real_function)   # works — pickled by reference to its module + name

Protocol Versions

Pickle has several binary protocol versions, each newer one adding features and efficiency but requiring a compatible or newer Python to unpickle. pickle.DEFAULT_PROTOCOL is what dumps/dump use if you don't specify one; pickle.HIGHEST_PROTOCOL is the newest available in the running Python version. Pass protocol= explicitly when you need cross-version compatibility with an older reader.

CPython implementation detail: exact protocol numbers and their default in a given Python version change over releases (e.g. protocol 5, added in Python 3.8, supports out-of-band data for large buffers). Don't hardcode assumptions about which protocol is "current" across Python versions.

import pickle

print(pickle.DEFAULT_PROTOCOL, pickle.HIGHEST_PROTOCOL)

# Use protocol 4 to stay readable by Python 3.4+ readers, if that matters for you
blob = pickle.dumps({"x": 1}, protocol=4)

Portability Considerations

Pickle is a Python-specific format — it cannot be read by other languages, unlike JSON or CSV. It's also not guaranteed to be stable across arbitrary Python or library version changes: if a pickled object's class definition changes shape (renamed attribute, moved module, changed base class) between when it was pickled and when it's unpickled, loading can fail or silently produce a subtly wrong object. Pickle files are therefore best treated as short-to-medium-lived, same-codebase artifacts (a cache, a checkpoint), not as a long-term archival or cross-system storage format.

Good fit for pickle:               Bad fit for pickle:
- in-process object cache           - long-term data archive
- multiprocessing IPC (built-in)    - cross-language data exchange
- ML model checkpoint (same env)    - public API request/response bodies
- your own trusted state file       - data from an untrusted source

Key Takeaways

Pickle can serialize nearly any Python object, including custom classes and complex object graphs — far beyond JSON's small type set.

Never unpickle data from an untrusted source — unpickling can execute arbitrary code, with no separate exploit needed.

Use pickle only for trusted, internal, same-codebase use cases: caching, checkpoints, multiprocessing IPC, your own saved state.

Custom classes pickle by reference to their module + name, plus their __dict__ (or __getstate__ output); the class must be importable when unpickling.

Open resources like sockets and file handles, and lambdas/locally-defined functions, generally can't be pickled directly.

Pickle files are Python-specific and not guaranteed stable across class/version changes — don't treat them as a long-term or cross-language storage format.

data = pickle.loads(request.body) # request.body comes from an external user import json data = json.loads(request.body) # JSON cannot execute code during parsing

Unpickling untrusted input can execute arbitrary code embedded in the byte stream; use json (or another safe, data-only format) for anything from outside your trust boundary.

with open("data.pkl", "w") as f: # text mode pickle.dump(data, f) # TypeError: must be a binary file with open("data.pkl", "wb") as f: # binary mode pickle.dump(data, f)

Pickle output is always a byte stream, never text; the file must be opened in binary mode ('wb'/'rb').

Pickle stores a reference to the class by module path and name, not its code; the class must be importable under that same name wherever the pickle is later loaded.

What is the single most important security rule about pickle, and why?

Never unpickle data from an untrusted source. The pickle format can embed instructions that execute arbitrary code during deserialization, so calling pickle.load()/loads() on attacker-controlled bytes can run arbitrary code with no separate exploit required.

What can pickle serialize that json cannot?

Pickle can serialize nearly any Python object, including custom class instances, object graphs with shared or circular references, and (named) functions by reference — json is limited to dict, list, str, int/float, bool, and None.

Why must files used with pickle.dump/load be opened in binary mode?

Pickle's output is a byte stream, not text — it isn't decodable as a string in any text encoding. Opening in text mode ('w'/'r') raises TypeError because pickle requires a binary-mode file object ('wb'/'rb').

What happens if you try to unpickle an instance of a class that no longer exists at the same module path?

It fails, typically with AttributeError or a pickle-specific error, because pickle stores a reference to the class by module and name rather than the class's actual code — the class must be importable under that same identity wherever the pickle is loaded.

When is pickle an appropriate choice, and when should you reach for json instead?

Pickle is appropriate for trusted, internal, same-codebase uses: caching, checkpoints, multiprocessing IPC, or your own application state. Use json instead whenever data crosses a trust boundary, needs to be read by another language, or needs to be safely handled even if its source might not be fully trusted.

Why might a lambda function fail to pickle while a regular named function pickles successfully?

Pickle serializes functions by reference to their importable module path and name, not their bytecode. A lambda has no stable, importable name to reference, so pickle generally cannot serialize it, while a module-level named function can be looked up by name on unpickling.

Write a function that caches the result of an expensive computation to a pickle file, only recomputing if the cache file doesn't exist. Treat the cache as fully trusted (written only by your own program).

import pickle from pathlib import Path def cached_compute(cache_path: Path, compute_fn): if cache_path.exists(): with cache_path.open("rb") as f: return pickle.load(f) result = compute_fn() with cache_path.open("wb") as f: pickle.dump(result, f) return result

Checking existence first avoids recomputation on subsequent runs; using pickle here is appropriate because the file is only ever written and read by this same trusted program, never by an external or untrusted source.

What does this print?

5. pickle.dumps captured the object's state (count=5) at the moment it was called; mutating the original object afterward has no effect on the already-serialized bytes, and loads() reconstructs a fresh, independent object from that captured state.

Write a class representing a database session that excludes its live connection object from pickling and re-establishes it on unpickling, using __getstate__/__setstate__.

import pickle class DbSession: def __init__(self, dsn: str): self.dsn = dsn self.connection = self._connect() def _connect(self): return object() # placeholder for a real, unpicklable connection def __getstate__(self): state = self.__dict__.copy() del state["connection"] return state def __setstate__(self, state): self.__dict__.update(state) self.connection = self._connect()

__getstate__ returns a copy of __dict__ with the unpicklable 'connection' entry removed, so pickle only serializes the dsn string; __setstate__ restores the remaining attributes and calls _connect() again to rebuild a fresh, valid connection on the unpickling side.

import pickle

class Counter:
    def __init__(self):
        self.count = 0

c = Counter()
c.count = 5
blob = pickle.dumps(c)
c.count = 100
restored = pickle.loads(blob)
print(restored.count)