Foreign Function Interface Concepts

Before you can safely call native code from Python, you need to understand the low-level contract every FFI tool is built on: ABI, calling conventions, data layout, ownership, and how errors cross the language boundary.

A Foreign Function Interface (FFI) is the general concept behind ctypes, cffi, and every other tool that lets one language call functions written in another. All of them are built on top of the same handful of low-level ideas — ABI, calling conventions, data layout, ownership, and error propagation — and understanding these concepts directly explains why interop bugs look the way they do: silent corruption and crashes instead of clean exceptions.

What Is an FFI, Conceptually

An FFI is a mechanism that lets code written in one language invoke a function implemented in another, and exchange data with it, at the level of raw memory and machine calling conventions rather than through either language's own high-level abstractions. Python's various interop tools (ctypes, cffi, the C API) are all specific implementations of this general concept for calling C (or C-ABI-compatible) code.

ABI vs API

An API (Application Programming Interface) is a source-level contract: function names, parameter types as seen in source code, expected behavior. An ABI (Application Binary Interface) is the compiled-level contract: how those same functions actually appear in machine code — how arguments land in registers or on the stack, how a struct's fields are laid out in memory with padding, how a compiled symbol is named. Two libraries can share the same API description yet be ABI-incompatible if compiled with different compilers, flags, or on different platforms.

FFI tools like ctypes and cffi work at the ABI level: you describe types and layouts precisely enough to match the compiled binary, not just its documented interface.

Calling Conventions

A calling convention specifies exactly how a function call is performed at the machine level: which registers hold which arguments, which register holds the return value, who is responsible for cleaning up the stack, and how the call stack is unwound. Common conventions include the C calling convention (cdecl, used almost everywhere on Unix-like systems) and Windows' stdcall (used by much of the Win32 API).

This is why ctypes has both CDLL (cdecl) and WinDLL (stdcall) — calling a stdcall function as if it were cdecl corrupts the stack, typically causing a crash sometime after the call rather than an immediate, obvious failure.

Data Layout and Alignment

Native data types have a fixed size and alignment requirement (an int might be 4 bytes aligned to a 4-byte boundary; a struct's total layout includes padding bytes inserted to keep each field properly aligned). An FFI description of a struct must match the compiler's actual layout exactly — field order, types, and resulting padding — or fields get read from the wrong byte offsets.

struct Example {   // conceptual layout on a common 64-bit platform
    char  flag;     // 1 byte
    // 3 bytes of compiler-inserted padding here
    int   count;    // 4 bytes, must start on a 4-byte boundary
};                  // total size: 8 bytes, not 5

Ownership

Ownership answers the question 'who is responsible for freeing this memory, and when?' C has no automatic memory management, so every allocation crossing an FFI boundary needs an explicit, agreed-upon owner: does the caller free it, does the callee free it, does it live for the whole program? Getting this wrong produces either a memory leak (nobody frees it) or a double-free / use-after-free (more than one side thinks it owns the memory, or it's used after being freed).

Python's own objects are reference-counted and garbage-collected automatically; native memory obtained through an FFI generally is not, unless the binding tool explicitly wraps it in a Python object with a destructor that frees it.

Error Propagation Across the Boundary

C has no exceptions: functions typically signal failure through a special return value (NULL, -1) plus a separate error code (like errno, or a library-specific 'get last error' function), which the caller must check explicitly. An FFI layer has to translate that C-style error signaling into Python's exception-based model — this translation does not happen automatically; the binding code has to check return values and errno itself and raise an appropriate Python exception.

The C 'open' function signals failure by returning -1 and setting errno; nothing about that automatically becomes a Python exception, so the binding code checks it explicitly and raises OSError itself.

import ctypes, os

libc = ctypes.CDLL("libc.so.6", use_errno=True)
libc.open.argtypes = [ctypes.c_char_p, ctypes.c_int]
libc.open.restype = ctypes.c_int

fd = libc.open(b"/nonexistent/path", os.O_RDONLY)
if fd == -1:
    errnum = ctypes.get_errno()
    raise OSError(errnum, os.strerror(errnum))

Native Resource Management

File handles, sockets, native memory buffers, and library-specific handles obtained through an FFI are native resources that Python's garbage collector does not know how to release. Responsible FFI code wraps such resources in a Python object (often a context manager) whose __del__ or __exit__ explicitly calls the matching native 'close' or 'free' function, so cleanup is deterministic instead of relying on chance.

Wrapping a native handle in a context manager ties its lifetime to a 'with' block, giving Python-style deterministic cleanup for a resource Python's own memory management doesn't cover.

class NativeHandle:
    def __init__(self, raw_handle, close_fn):
        self._handle = raw_handle
        self._close_fn = close_fn

    def __enter__(self):
        return self._handle

    def __exit__(self, exc_type, exc, tb):
        if self._handle is not None:
            self._close_fn(self._handle)
            self._handle = None

How Python's FFI Tools Implement These Concepts

ctypes' argtypes/restype describe the ABI-level calling convention and data layout for a function; ctypes.Structure._fields_ describes struct layout; use_errno/get_errno bridges C-style error signaling into something Python code can inspect. cffi's cdef strings serve the same purpose — describing ABI-level types and signatures — with a more declarative, header-like syntax. Every FFI tool is, at its core, a way of stating these five concepts precisely enough for the compiled boundary to work correctly.

Common Pitfalls

Mismatched struct field order or missing padding awareness silently misreads data. Wrong calling convention corrupts the stack. Assuming a native pointer is automatically freed (or automatically kept alive) when Python's garbage collector has no knowledge of it produces leaks or use-after-free bugs. Forgetting to check a C-style error return value lets a failure pass through silently, with corrupted or missing data discovered much later.

Best Practices

Always base type and layout descriptions on the native library's actual header/documentation, matched to the platform and compiler it was built with. Explicitly translate every C-style error signal into a Python exception at the boundary. Wrap every native resource in a Python object that ties its lifetime to Python's own scoping (context managers, explicit close methods) rather than hoping garbage collection happens to handle it.

Mental Model

Think of the FFI boundary as a border crossing with no shared language: ABI and calling conventions are the paperwork format both sides must fill out identically; data layout is how goods must be packed to be recognized; ownership is who signs for custody of each package; error propagation is how a rejected shipment gets reported back, since there's no automatic notification system — someone has to check.

Quick Reference

ABI                  compiled-level contract: registers, layout, symbol names
Calling convention    how args/return values move (cdecl, stdcall, ...)
Data layout            field order, size, and padding matching the compiler
Ownership              who frees native memory, and when
Error propagation      translating C return codes/errno into exceptions
Resource management    tying native handles to Python object lifetimes

Key Takeaways

FFI concepts (ABI, calling convention, data layout, ownership, error propagation, resource management) underlie every Python/C interop tool, not just ctypes.

ABI is the compiled-level contract; getting it precisely right is what separates a working binding from silent memory corruption.

C has no exceptions or automatic memory management — an FFI layer must explicitly translate error codes into Python exceptions and explicitly manage native resource lifetimes.

Interop bugs in this space typically manifest as crashes or corrupted data rather than clean Python errors, because they occur below the level Python's own safety machinery can see.

Field order in the Python description must match the native struct's actual declared order (and its resulting padding), not just contain the same field names — a mismatch silently misreads memory.

result = native_lib.risky_call() # no check of the C-style error signal at all use(result) result = native_lib.risky_call() if result < 0: err = ctypes.get_errno() raise OSError(err, os.strerror(err)) use(result)

C signals failure through return codes, not exceptions; skipping the check means a failed call's garbage or stale result is used as if it were valid data.

What is the difference between an API and an ABI?

An API is the source-level contract (function names, argument types as written in source); an ABI is the compiled-level contract (register usage, memory layout, symbol names) that determines whether compiled code from different sources can actually call each other correctly.

Why can calling a stdcall function using a cdecl calling convention crash a program?

The two conventions disagree about how the stack is cleaned up after a call; using the wrong one leaves the stack in an inconsistent state, which typically corrupts later execution rather than failing immediately at the call site.

Why does the order of fields in a ctypes.Structure matter, not just their types?

Because it determines the memory offset at which each field is read or written; the description must match the compiler's actual layout (including inserted padding) or values are read from the wrong bytes.

Why doesn't C code raise Python-style exceptions on error?

C predates and has no built-in exception mechanism; it signals failure through return values (like NULL or -1) and side-channel error state (like errno), which calling code must check explicitly.

Who is responsible for tracking ownership of memory allocated by a native library and returned to Python?

The FFI binding code — Python's garbage collector has no visibility into native allocations unless the binding explicitly wraps the pointer in a Python object that calls the matching free function when appropriate.

Give an example of tying a native resource's lifetime to Python's own scoping rules.

Wrapping a native handle (file descriptor, library handle, buffer) in a context manager whose __exit__ calls the native close/free function, so a 'with' block guarantees deterministic cleanup.

Write a small context manager class that wraps a raw native handle and a matching close function, ensuring the handle is released even if an exception occurs inside the 'with' block.

class NativeResource: def __init__(self, handle, close_fn): self._handle = handle self._close_fn = close_fn def __enter__(self): return self._handle def __exit__(self, exc_type, exc, tb): if self._handle is not None: self._close_fn(self._handle) self._handle = None return False # don't suppress exceptions

The context manager protocol (__enter__/__exit__) guarantees __exit__ runs even when the block raises, giving deterministic release of a resource Python's garbage collector doesn't know how to manage on its own.

This ctypes Structure's field order doesn't match the C declaration order. What is the practical consequence?

Fields are read at the wrong memory offsets relative to the real struct's layout — 'a' and 'b' would be read from swapped (and likely wrongly-sized/padded) locations, corrupting every value read from or written to instances of this struct.

Explain why a function that returns -1 on failure and sets a global error code (like errno) requires extra binding code to behave like a normal Python function.

Because -1 and errno are C's error-signaling convention, not a Python exception; the binding must explicitly check the return value, read the error code, and raise an appropriate Python exception (such as OSError) — otherwise a failure silently looks like a successful call that happened to return -1.

# C header: struct Pair { short a; long b; };
class Pair(ctypes.Structure):
    _fields_ = [("b", ctypes.c_long), ("a", ctypes.c_short)]