06 — Data Persistence: Why Container Files Vanish (and How to Stop Them)

A file I saved inside a container was gone the next morning, and I had no vocabulary for why. The one sentence that explained it: containers are ephemeral by default, so anything I want to keep has to live in storage attached from outside the container's writable layer. [1][2]

The framing worth holding onto is that "container storage" is not one thing — it's three modes with different lifetimes, owners, and use cases. The bug I kept hitting (data disappearing) was me using the wrong one by default. Once I could name the three modes, the choice became obvious [1].

Why the default loses data

By default, everything a container writes lands in its writable layer — a thin scratchpad Docker stacks on top of the image's read-only layers [1][2]. That scratchpad exists for the lifetime of the container, not the image. The moment I docker rm the container, the writable layer is discarded, and every file the process wrote disappears with it.

This is the "ephemeral container filesystem" behavior, and it's not a bug — it's the design. Stateless-by-default is what lets me treat containers as disposable: rebuild, replace, scale up, scale down, all without worrying about leftover state. The cost is that anything I do want to keep has to be opted into persistence explicitly [1][2].

The three storage modes

Docker gives me three ways to attach storage, and they differ in who manages the data and where it physically lives.

1. The writable layer (default, ephemeral). Every container gets one. Fast, isolated, and thrown away with the container. Use it only for scratch files I'm happy to lose.

2. Volumes (managed by Docker, persistent). A volume is a directory Docker creates and manages on the host, in its own storage area, and mounts into the container at a path I choose [3]. The container writes to /var/lib/postgresql/data; underneath, Docker is writing to a managed directory on the host. The container can be removed and recreated; the volume — and its data — stays.

docker volume create pgdata
docker run -d -v pgdata:/var/lib/postgresql/data postgres:16

Volumes are the recommended default for persistent app data. They're independent of the host's directory layout (so they work the same on Linux, Mac, and Windows), Docker manages their lifecycle, and they can be shared between containers [3].

3. Bind mounts (a host path, mounted directly). A bind mount wires a specific absolute path on the host into the container. The container is now writing to a real directory I can see from my shell, outside Docker's management [4][5].

docker run -d -v "$PWD:/app" -w /app node:20 node server.js

The trade-off: bind mounts are tethered to the host's filesystem layout (so they don't port across machines cleanly), and the container can write anywhere in that host path, which is a real footgun. Their superpower is live editing — I mount my source directory, edit a file on the host, and the container sees the change instantly. That's the backbone of hot-reloading dev workflows [4].

Choosing between them

The decision collapses to two questions. Who needs to manage the data? and Does it need to be portable?

• Application data that must survive container recreation (database files, uploads, logs I care about) → volume. Docker manages it, it survives removals, and it's portable [3].
• Source code I want to live-edit during development → bind mount. The container sees my host files change instantly, which is what makes dev loops fast [4].
• Throwaway scratch → writable layer. Don't pay the cost of a volume for data I want gone anyway.

Sharing data between containers

One detail I underused at first: volumes aren't tied to a single container. Two containers can mount the same volume and read/write the same files, which is how one container writes data and another reads it. A common pattern is a "data-only" helper container, or just attaching the same named volume to several services in a Compose file. Bind mounts share the same trick — mount the same host path into multiple containers and they're looking at the same files [3][4].

How I use this

The habit I built is to ask, before I docker run anything stateful: "what path holds the data I'd be sad to lose?" Then I attach a volume at exactly that path and nothing else. For databases that's the data directory; for an upload service it's the uploads folder; for a CI runner it's the workspace cache. For dev work where I'm editing code, I reach for a bind mount of my source directory and accept the portability trade-off because the live-reload payoff is worth it. The single question — "is this path ephemeral, and if not, where does the real storage live?" — is what stopped me from losing data to the writable layer.

References

[1] Docker, Inc., "Persisting container data," Docker Docs, 2024. [Online]. Available: https://docs.docker.com/get-started/docker-concepts/running-containers/persisting-container-data/

[2] Docker, Inc., "Concepts — Persisting container data (video)," YouTube, 2024. [Video]. Available: https://www.youtube.com/watch?v=10_2BjqB_Ls

[3] Docker, Inc., "Volumes," Docker Docs, 2024. [Online]. Available: https://docs.docker.com/storage/volumes/

[4] Docker, Inc., "Bind mounts," Docker Docs, 2024. [Online]. Available: https://docs.docker.com/storage/bind-mounts/

[5] Docker, Inc., "Choose the -v or --mount flag," Docker Docs, 2024. [Online]. Available: https://docs.docker.com/storage/bind-mounts/#choose-the--v-or---mount-flag