Everything Is a File: A Wider View of Data Formats

It Is All Files

Open the file manager on your computer and look around. Photos, music, videos, documents, spreadsheets, saved games, downloaded program installers, the application files of programs you have installed — they are all there, side by side, in folders. They look different in the file manager because of their icons, and they behave differently when you double-click them, but at the level of the operating system, they are the same kind of thing: a sequence of bytes saved on disk, with a name.

There is no special “photo storage” or “video storage” on your computer. There is just the filesystem, and in it are files. What makes a photo a photo and a song a song is not a special kind of storage. It is the format — the agreed-upon way the bytes of the file are arranged and interpreted.

All the data on your computer is in files; the files differ only in their format. Your CSV study log is a file in one format. A JPEG photo is a file in another. A ZIP archive, a PDF document — all files, all formats. The same operating system stores them, the same open(...) call can be pointed at any of them. What changes is what you have to do with the bytes to make them mean something.

Text and Binary

The first useful distinction between formats is text versus binary.

A text file is one where every byte represents a character — a letter, digit, punctuation mark, newline, or space. You can open it in a text editor and read it. For instance, source code files, CSV files, and so on, are all text.

A binary file is one where the bytes are not meant to be read as characters. They are numbers that encode something else: pixel colours, audio samples, video frames, structured records, compressed contents of other files. Open one of these in a text editor and you see strings of nonsense — random punctuation, accented letters, control characters — because the editor is trying to display the bytes as if they were characters, and almost none of them are.

This is why the reading chapter recommended encoding="utf-8" and why the defensive programming chapter will return to it. When you tell Python to open a file in text mode, you are telling Python “interpret the bytes of this file as characters using this encoding.” On a text file, that works. On a binary file, the bytes are not characters, and you might see a UnicodeDecodeError. The error is a hint that the file is not what you might have thought it was.

Similarly, if you try to read a CSV file by treating its bytes as raw binary data, you can do it, but you get back numbers instead of letters and you have to translate them yourself.

The text/binary distinction is mostly about choosing the right tool of reading the files: text mode for files that are text, binary mode for files that are not.

A Short Tour of Common Formats

You do not need to know how every format works internally. What helps is recognising the kind of thing each format is designed for, so that when you encounter one in the wild you know what to expect.

A non-exhaustive tour, organised by what they store:

Structured text — human-readable, often editable in any text editor:

• Plain text — just characters, no particular structure. Good for prose, source code, and anything else that is meant to be read by people.
• CSV — rows of fields. The format from this part. Best for tabular data: spreadsheets, log entries, anything with a fixed set of columns.
• JSON — nested data with brackets and braces. Good for structured data that is not naturally tabular, like a list of sessions where each session has a date, subject, and minutes.
• HTML — nested data with tags. HTML is the format of web pages.

Images — binary, encoded grids of pixels:

• PNG — lossless compression. Good for diagrams, screenshots, anything with sharp edges.
• JPEG — lossy compression. Smaller files, some quality lost. Designed for photos.
• GIF — older format, supports simple animation.

Audio and video — binary, big, heavily engineered for compression:

• WAV — uncompressed audio. Large files.
• MP3 — compressed audio.
• MP4 — compressed video, the most common video container today.

Archives — binary, contain other files inside, often compressed:

• ZIP — bundles many files into one, with compression. The format used for .zip downloads.

Documents — mostly binary, can be very complex:

• PDF — portable documents. Largely binary, with embedded fonts, images, and laid-out text.

Each of these are typically opened with a specific kind of an application. A text editor for text files, an image viewer for images, a media player for audio and video, an archive manager for ZIP files, a PDF reader for PDFs.

Recognising what kind of thing each is — structured text, image, audio, archive, document — is part of what makes a working filesystem feel comprehensible. When you see a file named photo.png, you know it is an image and you know which application to open it with. When you see data.csv, you know it is structured text and that you can open it in a spreadsheet or a text editor.

The Extension is Not a Promise

The last few characters of a file’s name — the extension, the part after the last dot — conventionally indicate the format. .csv for CSV, .png for PNG, .zip for ZIP. Your operating system uses the extension to decide which application to open the file in when you double-click it.

The extension is, however, just a naming convention. It is part of the filename, nothing more. You can rename photo.png to photo.csv, but the file does not change — it is still the same bytes, still a valid PNG, still not a CSV. If you then tell Python to read the renamed file with csv.reader, you will get nonsense or an error.

The other direction is also true. A file named data.txt might be plain text. Or it might be a CSV that someone forgot to rename. Or, occasionally, something else. The extension is what the file claims to be. The format is what the file is. Most of the time they match. When they do not, you have a bug — and the symptom is usually a strange error from whatever tried to read the file.

The practical takeaway is small but useful: when something fails to read, check the file in an editor. If it looks like garbage, it is probably not a text file regardless of what the extension says. If it looks like CSV when you opened it as JSON, the extension lied.

with open("diagram.txt", encoding="utf-8") as f:
    print(f.read())

Check Your Understanding

1. In your own words, what is a file format? Why do photos and music need different formats?
2. What is the difference between a text file and a binary file? Give one example of each from your own computer.
3. If you open a PNG file in a text editor, what do you see, and why?
4. CSV is described in this chapter as “structured text”. What does that mean — what does it have that plain text doesn’t?
5. You rename a PNG file from photo.png to photo.txt. What changes about the file, and what doesn’t?