Input/Output
I/O in Python is a three-layer stack, and nearly every trap in the topic is explained by which layer you are currently standing on. At the bottom sits the raw layer, RawIOBase; for a file that is FileIO, a thin wrapper over an operating-system descriptor that can only hand out and take in bytes. Above it sits the buffering layer — BufferedReader / BufferedWriter / BufferedRandom — which accumulates data in memory so a system call is not made per byte. On top sits TextIOWrapper, which decodes bytes into str, encodes them back, and rewrites line endings along the way. open('f.txt') assembles all three, open('f.bin', 'rb') stops at the second layer, and open('f.bin', 'rb', buffering=0) hands you a bare FileIO.
What is Python-specific here is the strict separation of str and bytes. Text mode works only with str and always applies an encoding; binary mode works only with bytes and does not touch a single one. Every other rule of the topic grows out of that split, and none of them is intuitive: buffering=0 is legal only in binary mode, tell() in text mode returns an opaque cookie rather than a byte offset, seek from a non-zero whence is forbidden there, w truncates the file at open time, and a+ writes at the end even immediately after seek(0). Serialization stands apart — json is portable across languages but knows six types, while pickle serializes almost any object and, on untrusted input, executes arbitrary code.
Topic map
- The file object and the io stack — what a stream is, which three layers
open()assembles it from, and why a file need not be a file on disk. - open() modes —
r,w,x,adecide the file's fate,t/bthe data type,+the direction; and what each combination does to the contents. - Text vs binary mode —
strversusbytes, exactly where the encoding is applied, and hownewline=controls the rewriting of\r\n. - Buffering, flush and close — line versus block buffering, why
buffering=0outside binary mode is illegal, and howflush()differs fromos.fsync(). - tell() and seek() — byte offsets in binary mode and opaque cookies in text mode, which is why arithmetic on a position is forbidden there.
- StringIO and BytesIO — in-memory streams with the same file API — for tests, output capture, and building data without touching disk.
- json and pickle — which types survive each format, and why
pickle.loadon untrusted input is the execution of someone else's code.
Common mistakes and traps
| Mistake | Consequence |
|---|---|
| Not closing a file and relying on the garbage collector | close() flushes first, so skipping it loses the tail of the write, not just a descriptor; and collection time is unpredictable |
Treating flush() as a guarantee against a power loss | flush() hands the data to the kernel, not to the medium; os.fsync(f.fileno()) pushes it to the device |
Opening an existing file with w in order to "add to it" | w truncates the file to zero at open time, before the first write |
Expecting seek(0) in a+ mode to make writes land at the start | The O_APPEND flag moves the position to the end before every write — seek only affects reading |
Reading tell() in text mode as a byte offset | It is an opaque marker of decoder state; arithmetic on it and seek from whence=1/2 are forbidden |
| Opening an image or an archive in text mode | Decoding either raises UnicodeDecodeError or silently corrupts the data, and writing additionally rewrites \n as \r\n on Windows |
Loading a pickle that came over the network or from a user | The REDUCE opcode calls an arbitrary object — that is remote code execution, not a "parse error" |
Reading a StringIO right after writing and getting an empty string | The position sits at the end — you need seek(0), or getvalue() |
What interviews check
Everyone gets asked about this topic, from junior to senior, but the depth differs sharply. At junior level the interviewer wants the mode letters and the text/bytes distinction: w truncates, x fails on an existing file, a appends, b yields bytes, and text mode yields str and applies an encoding. Then comes "why must you close a file" — and the right answer is not "so descriptors don't leak" but "because close() flushes the buffer". A candidate who names only descriptors has shown that the buffering layer is not in their mental model.
Middle-level questions are about mechanism. What the buffering argument controls, why buffering=0 is illegal in text mode, how flush() differs from os.fsync(), what tell() and seek() do, and why you cannot move "ten bytes back" in text mode. io.StringIO shows up here too, usually in a testing context where code is handed a stream instead of a file. Senior level moves into serialization — how to teach json your own types via default and object_hook, and why pickle must not touch data that crossed a trust boundary. The classic mistake on that last question is "the format is binary, so no code can run". It is exactly the other way round: pickle is a small virtual machine, and its opcodes can call anything.