Safe File & Process Handling

Running subprocesses and touching the filesystem safely means argument lists instead of shell strings, validated paths, correctly-created temp files, and deliberate permissions.

Two of the riskiest things a Python program can do are launching another process and touching the filesystem with data it doesn't fully control. This topic covers the standard-library patterns that make both safe: subprocess with argument lists instead of shell strings, careful path validation, the tempfile module for temporary files, and deliberate file permissions.

Subprocess Argument Lists

subprocess.run() accepts either a list of arguments (["ls", "-l", directory]) or a single string. Passing a list means each element is handed to the operating system as one discrete argument — no shell parses or reinterprets it, so spaces, quotes, and shell metacharacters in an argument are never treated as syntax. This is the default and recommended form for nearly all subprocess usage.

Each list element is one literal argument passed directly to the grep executable via the operating system's process-creation API — no shell is involved, so there's nothing for shell metacharacters in any argument to exploit.

import subprocess

result = subprocess.run(
    ["grep", "-n", "TODO", "app.py"],
    capture_output=True,
    text=True,
    check=True,
)
print(result.stdout)

Avoid Unsafe Shell Usage

shell=True runs the command through the system shell (/bin/sh on POSIX), which means the command string is parsed the way you'd type it at a terminal — metacharacters like ;, |, &&, backticks, and $() are all live syntax. If any part of that string comes from untrusted input, this is command injection (see the related Secure Python Coding topic). Even with a fully trusted, hardcoded command, shell=True adds an unnecessary layer of parsing and a small extra process (the shell itself).

If you genuinely need shell features (globbing, pipes, environment-variable expansion) with a *fixed*, non-user-controlled command, shell=True is not inherently forbidden — the danger is specifically interpolating untrusted data into the shell string. Even then, prefer building the pipeline with multiple subprocess.run() calls or subprocess.Popen chains where practical.

import subprocess

filename = get_user_filename()  # untrusted

# DANGEROUS: shell=True interprets ";" as a command separator
# subprocess.run(f"cat {filename}", shell=True)

# SAFE: argument list, no shell, no metacharacter interpretation
subprocess.run(["cat", filename], check=True)

Checking Subprocess Results

By default subprocess.run() does not raise on a nonzero exit code — you have to check result.returncode yourself, or pass check=True to have it raise subprocess.CalledProcessError automatically. Silently ignoring a failed subprocess (treating its absence of output as success) is a common source of bugs that only surface much later.

check=True converts a nonzero exit code into an exception instead of a silently ignored failure; timeout= prevents an unresponsive child process from hanging the parent forever.

import subprocess

try:
    subprocess.run(["git", "clone", repo_url, dest], check=True, timeout=60)
except subprocess.CalledProcessError as exc:
    print(f"git clone failed with exit code {exc.returncode}")
except subprocess.TimeoutExpired:
    print("git clone timed out")

Temporary Files

The tempfile module creates temporary files and directories safely: unique, non-guessable names (avoiding a race where an attacker predicts and pre-creates the path), correct permissions from the start, and automatic cleanup when used as a context manager. Never hand-construct a temp path by concatenating a fixed prefix with, say, the current timestamp — that's both guessable and subject to race conditions between the name being chosen and the file being created.

tempfile creates the file/directory with a securely-random unique name and (on POSIX) restrictive permissions atomically at creation time, closing the window an attacker could otherwise exploit between name selection and creation.

import tempfile
from pathlib import Path

with tempfile.NamedTemporaryFile(mode="w", suffix=".txt", delete=True) as tmp:
    tmp.write("scratch data")
    tmp.flush()
    print(f"wrote to {tmp.name}")
# file is automatically deleted here, on exiting the with-block

with tempfile.TemporaryDirectory() as tmp_dir:
    scratch = Path(tmp_dir) / "work.txt"
    scratch.write_text("intermediate result", encoding="utf-8")
    # ... use scratch ...
# the entire directory and its contents are removed automatically here

Path Validation

Any filesystem path built from untrusted input (a filename from a web upload, a CLI argument, an API field) should be validated before use — most importantly to prevent path traversal, where .. segments let the resolved path escape an intended base directory. Resolve the joined path and confirm it remains inside the trusted base before opening or writing it (full treatment in the Secure Python Coding and Path Traversal material).

from pathlib import Path

def safe_open_for_read(base_dir: Path, user_filename: str):
    base = base_dir.resolve()
    target = (base / user_filename).resolve()
    if not target.is_relative_to(base):  # Python 3.9+
        raise ValueError(f"path escapes base directory: {user_filename!r}")
    if not target.is_file():
        raise FileNotFoundError(f"no such file: {user_filename!r}")
    return target.open(encoding="utf-8")

File Permissions

On POSIX systems, os.chmod() sets a file's permission bits explicitly; when writing sensitive data (credentials, private keys, session tokens), narrow the permissions to owner-only (0o600 for files, 0o700 for directories) rather than relying on the process's default umask, which can vary between environments. Passing mode= to Path.open()/os.open() at creation time is safer than creating the file with default permissions and tightening them afterward, since that avoids a brief window where the file exists with broader-than-intended permissions.

os.open() with an explicit mode argument creates the file already restricted — there's no moment where it briefly exists at the more permissive default mode, unlike write-then-chmod.

0o600/0o700 are POSIX (Linux/macOS) permission bits; Windows uses a different ACL-based permission model, so file-permission code that must be cross-platform generally needs a Windows-specific branch or a portable third-party library — not covered further here.

import os
from pathlib import Path

# create the file with restrictive permissions from the moment it exists
fd = os.open("secret.key", os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
with os.fdopen(fd, "w") as f:
    f.write("super-secret-value")

# tightening after the fact (acceptable, but a brief window existed at default perms)
Path("credentials.json").write_text('{"token": "..."}', encoding="utf-8")
os.chmod("credentials.json", 0o600)

Practical Usage

A realistic workflow: build a subprocess command as a list (never a formatted string), pass check=True and a timeout, write any intermediate scratch data with tempfile so it's automatically cleaned up, validate any filesystem path derived from external input before touching it, and set restrictive permissions on anything containing secrets. None of this is exotic — it's the same handful of standard-library APIs used correctly and consistently.

Edge Cases

A subprocess.run() call without timeout= can hang forever if the child process never exits (e.g. it's waiting on stdin you never provide) — always set a reasonable timeout for any subprocess you don't fully control. Symlinks are a subtle path-validation trap: resolving a path with .resolve() follows symlinks, so a symlink inside an otherwise-valid base directory can still point outside it; .resolve() before the is_relative_to() check (as shown above) correctly catches this, but skipping .resolve() and only checking the un-resolved path would not.

Complete Example

import os
import subprocess
import tempfile
from pathlib import Path

UPLOAD_DIR = Path("uploads").resolve()

def convert_upload_to_pdf(user_filename: str) -> Path:
    # 1. validate the path stays inside the trusted upload directory
    source = (UPLOAD_DIR / user_filename).resolve()
    if not source.is_relative_to(UPLOAD_DIR) or not source.is_file():
        raise ValueError(f"invalid upload filename: {user_filename!r}")

    # 2. do the conversion work in a scratch directory that cleans itself up
    with tempfile.TemporaryDirectory() as scratch_dir:
        out_path = Path(scratch_dir) / "converted.pdf"

        # 3. run the external tool as an argument list, never shell=True
        subprocess.run(
            ["some-converter", str(source), "-o", str(out_path)],
            check=True,
            timeout=30,
        )

        # 4. move the result somewhere permanent with tight permissions
        final_path = UPLOAD_DIR / f"{source.stem}.pdf"
        final_path.write_bytes(out_path.read_bytes())
        os.chmod(final_path, 0o600)
        return final_path

Mental Model

Treat every subprocess call as handing a fixed, pre-built list of ingredients to a chef through a hatch (no room for the ingredients to be reinterpreted as instructions), and treat every filesystem path from outside your program as a delivery address that must be checked against a map before anything gets sent there.

Quick Reference

subprocess.run([...], check=True, timeout=N)  - argument list, no shell
shell=True                                     - avoid; command injection risk with untrusted data
tempfile.NamedTemporaryFile() / TemporaryDirectory() - safe scratch files, auto-cleanup
path.resolve().is_relative_to(base)            - path traversal check (3.9+)
os.open(path, flags, 0o600)                    - create file already restricted
os.chmod(path, 0o600)                          - tighten permissions after the fact

Key Takeaways

Pass subprocess commands as argument lists, not formatted strings; avoid shell=True whenever any part of the command could be untrusted.

Use check=True to catch failed subprocesses and timeout= to prevent a hung child process from blocking forever.

Use tempfile for scratch files/directories — it avoids guessable names and race conditions, and cleans up automatically as a context manager.

Validate any externally-derived path with .resolve() plus .is_relative_to(base) to prevent path traversal, including through symlinks.

Create files containing secrets with restrictive permissions from the start (os.open(..., mode=0o600)) rather than tightening them after the fact.

subprocess.run(f"cp {src} {dst}", shell=True) subprocess.run(["cp", src, dst], check=True)

If src or dst contains shell metacharacters (spaces are usually fine, but ;, |, $(...) are not), shell=True with an interpolated string can execute unintended commands. An argument list passes each value as a single literal argument with no shell parsing.

import time tmp_path = f"/tmp/scratch_{time.time()}.txt" with open(tmp_path, "w") as f: f.write(data) import tempfile with tempfile.NamedTemporaryFile(mode="w", suffix=".txt") as f: f.write(data) tmp_path = f.name

A hand-built temp path based on a timestamp is guessable (an attacker on a shared system could predict and pre-create it) and races between name-selection and file-creation; tempfile picks a securely random, non-guessable name and creates the file atomically.

result = subprocess.run(["risky-tool", arg]) # no check, no timeout -- failure is silently ignored, and a hang blocks forever result = subprocess.run(["risky-tool", arg], check=True, timeout=30)

Without check=True, a nonzero exit code passes silently; without timeout=, a hung child process blocks the parent indefinitely. Both are easy to add and close two common, unrelated failure modes.

Why is passing a command as a list to subprocess.run() safer than passing a single formatted string with shell=True?

A list's elements are passed directly to the operating system's process-creation call as discrete arguments; no shell parses the string, so shell metacharacters embedded in any argument (from untrusted input especially) are never reinterpreted as command syntax.

What does check=True do when passed to subprocess.run(), and what happens without it?

With check=True, a nonzero exit code from the child process raises subprocess.CalledProcessError. Without it, subprocess.run() returns normally regardless of the exit code, and a failed subprocess call can be silently treated as if it succeeded unless you manually inspect result.returncode.

Why does the standard-library tempfile module produce safer temporary files than hand-building a path from a timestamp or counter?

tempfile generates a securely random, non-guessable name and creates the file atomically with appropriately restrictive permissions, closing the window between choosing a name and creating the file that a hand-built path leaves open to guessing or race conditions.

Why must a path-traversal check call .resolve() before checking .is_relative_to(base), rather than checking the unresolved path directly?

.resolve() collapses .. segments and follows symlinks to the real, absolute target; checking an unresolved path could miss a ..-laden path or a symlink that points outside the intended base directory even though the literal (unresolved) path string looks contained.

Why is creating a sensitive file with os.open(path, flags, 0o600) preferable to writing it normally and calling os.chmod(path, 0o600) afterward?

Writing then chmod-ing leaves a brief window where the file exists with the more permissive default mode before it's tightened; creating it with the mode specified up front means it never exists in an over-permissive state.

Why should subprocess.run() calls generally include a timeout= argument?

Without a timeout, a child process that never exits (e.g. waiting on input that's never provided, or genuinely hung) blocks the parent process indefinitely; a timeout raises subprocess.TimeoutExpired instead, bounding how long the parent can be stuck.

Is shell=True always unsafe, even with a completely hardcoded, non-user-controlled command string?

Not inherently — the actual danger is interpolating untrusted data into the shell string, which enables command injection. A fixed, hardcoded command with shell=True is not exploitable that way, but it still adds unnecessary shell parsing and process overhead, so an argument list is still generally preferred when shell-specific features (pipes, globbing) aren't needed.

Write a function run_command_safely(args: list[str], timeout: float = 10.0) -> str that runs a subprocess from an argument list, raises on failure, enforces a timeout, and returns stdout as text.

import subprocess def run_command_safely(args: list[str], timeout: float = 10.0) -> str: result = subprocess.run( args, capture_output=True, text=True, check=True, timeout=timeout, ) return result.stdout # run_command_safely(["echo", "hello"]) -> "hello\n"

Taking args as a list (never building a shell string) prevents command injection; check=True converts a failing exit code into CalledProcessError; timeout= bounds how long a hung subprocess can block the caller; text=True decodes stdout as a string instead of bytes.

What happens when this code runs, and how would you rewrite it safely?

The shell parses ; as a command separator, so this actually runs two commands: touch somefile and then echo pwned — printing pwned even though the intent was only to touch a file named somefile; echo pwned. The safe rewrite is subprocess.run(["touch", name]), which passes the entire string "somefile; echo pwned" as a single literal filename argument to touch, with no shell involved to reinterpret the semicolon.

Write a function write_secret_file(path: str, content: str) -> None that writes content to path on a POSIX system such that the file is never, even momentarily, readable by anyone other than its owner.

import os def write_secret_file(path: str, content: str) -> None: fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600) try: with os.fdopen(fd, "w") as f: f.write(content) except Exception: raise

os.open() with mode=0o600 creates the file already restricted to owner read/write at the moment of creation, closing the window that a write-then-os.chmod() sequence would leave open (where the file briefly exists at the process's default, more permissive umask-derived permissions). os.fdopen wraps the low-level file descriptor in a normal file object for writing.

import subprocess
name = "somefile; echo pwned"
subprocess.run(f"touch {name}", shell=True)