--- title: "05 — S3: Object Storage, Buckets, and the Access Math" uid: s3-objects tags: ["aws", "s3", "lifecycle", "roadmap:aws", "storage", "objects", "policies"] excerpt: "S3's folders are an illusion — every object is a key in a flat bucket. And every access question is policy math between what the caller's identity allows and what the bucket itself allows." date: 2026-08-13T03:28:31+0000 source: https://www.aveshina.my.id/en/blog/s3-objects --- "A folder in the sky" was my mental model of S3, and it quietly explained nothing — not the bucket names, not the access errors, not the pricing. The two ideas that straightened it out: **S3 is a flat object store where the "folders" are an illusion — every object is just a key in a bucket — and every access question resolves to policy math between what the caller's identity allows and what the bucket itself allows** [1]. Once those two were clear, the rest of S3 (lifecycle rules, presigned URLs, CORS) read like straightforward configuration rather than a maze. ## What S3 actually is Amazon S3 — Simple Storage Service — is object storage [1]. I store arbitrary blobs ("objects") in containers ("buckets"), and each object is identified by a unique key. The unit of retrieval is a whole object: there is no editing in place, no byte-range writes, no filesystem lock. S3 is not a disk; it is a key→blob map with eleven 9s of durability. The mental shift I had to make: the slash in reports/2024/q1.csv is not a directory separator. It is a literal character in the key. S3 has no folders; the console *renders* prefixes as folders for my convenience, but underneath there is one flat keyspace per bucket. Tools that "create a folder" are actually just creating a zero-byte object whose key ends in /. ## Buckets and objects A **bucket** is the top-level container [5]. Three rules worth memorizing: - **Bucket names are globally unique** across all of AWS. my-bucket is taken; I'll be choosing my-company-reports-2024-xyz. - **A bucket lives in one region.** I pick the region at creation; the data stays there unless I replicate it. (Region choice matters for latency, cost, and data residency.) - **There is no limit on the number of objects in a bucket**, and objects can be from 0 bytes to 5 TB. The scale is genuinely vast. An **object** is the blob plus its metadata (content-type, custom tags, encryption settings). Object keys must be unique within a bucket; pushing the same key overwrites the previous object (unless versioning is on, in which case both are retained). ## The access math: identity-based vs resource-based This is where S3 security used to lose me. There are two policy surfaces, and S3 consults *both* [2][3]: - **Identity-based policy** — attached to the IAM user or role making the call. "This caller may s3:GetObject on arn:aws:s3:::my-bucket/*." - **Resource-based policy** — the **bucket policy**, attached to the bucket itself. "Anyone (or these specific principals) may read objects in *me*." For a same-account caller, S3 grants access if *either* policy allows it. For a cross-account caller, *both* must allow it (the caller's identity policy needs to permit the action, *and* the bucket policy needs to trust the caller's account). That "same-account = either, cross-account = both" rule is the part I had to write on a sticky note. A third surface — **ACLs (access control lists)** — exists, and AWS now recommends disabling them entirely in favor of bucket policies. Treat ACLs as legacy. ## Instance profiles and assuming roles, in S3 terms Two IAM concepts keep coming up with S3, and they're worth pinning down here [3][4]: - **Instance profiles** are how an EC2 instance gets a role attached to it. The role's identity-based policy can grant s3:GetObject on a bucket, and code on the instance reads objects without ever holding a long-lived key. - **Assuming roles** is how one identity temporarily takes on another role's permissions via STS — used for cross-account access ("the CI account assumes a role in the prod account to push a build to an S3 bucket"). In both cases the S3 access question collapses to the same identity-vs-resource math above. The delivery mechanism is different; the policy evaluation is identical. ## Bucket and object lifecycle S3 is cheap per GB, but at scale the bill and the clutter add up. **Lifecycle rules** automate transitions and deletions [6]: - **Transition actions** — move objects to a cheaper storage class after N days. "Logs older than 30 days go to S3 Standard-IA; older than 90 days go to Glacier." - **Expiration actions** — delete objects after N days. "Drop everything in tmp/ after 7 days." Lifecycle rules are how a bucket stays maintainable without a cleanup cron. The pattern I reach for: every "log" or "upload" bucket gets a lifecycle rule on creation; every long-term archive bucket gets a transition-to-Glacier rule. The notes on storage classes cover what each class costs and how fast it returns data. ## A real shell snippet Uploading and reading objects is directly observable with the AWS CLI: ``` aws s3 cp ./report.pdf s3://my-bucket/reports/2024/report.pdf aws s3 presign s3://my-bucket/reports/2024/report.pdf --expires-in 3600 ``` The first copies a local file into a bucket. The second generates a **presigned URL** — a time-limited link anyone can use to fetch the object, without holding AWS credentials. Presigning is the clean answer for "give this client a download link valid for an hour." ## How I use this S3 is the storage default for anything that isn't a database. My standing rules: bucket names include a project prefix and a random suffix to survive global uniqueness; every bucket gets versioning and default encryption turned on at creation; public access is blocked by default and only opened deliberately through a bucket policy I can defend; and lifecycle rules are part of the bucket's definition, not an afterthought. For uploads from a browser or mobile client I use presigned URLs (or a pre-signed POST) so the bytes never transit my own servers. The flat-keyspace picture is the part I keep closest — once "folders are an illusion" is internalized, every weird S3 path bug becomes obvious. ## References [1] Amazon Web Services, "Welcome to Amazon Simple Storage Service," S3 User Guide, 2024. [Online]. Available: [https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) [2] Amazon Web Services, "Identity-based policies for S3," S3 User Guide, 2024. [Online]. Available: [https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_identity-vs-resource.html](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_identity-vs-resource.html) [3] Amazon Web Services, "Using instance profiles," IAM User Guide, 2024. [Online]. Available: [https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) [4] Amazon Web Services, "Assuming a role," IAM User Guide, 2024. [Online]. Available: [https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_manage-assume.html](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_manage-assume.html) [5] Amazon Web Services, "Working with buckets," S3 User Guide, 2024. [Online]. Available: [https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html) [6] Amazon Web Services, "Managing your storage lifecycle," S3 User Guide, 2024. [Online]. Available: [https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html) ```quiz Q: In S3, the slash in a key like `reports/2024/q1.csv` is… - a literal character in a flat key — there are no real folders - a directory separator that creates nested folders on disk correct: 0 explain: S3 has a flat keyspace per bucket. The console renders prefixes as folders for convenience, but underneath there is only bucket → key → object. Q: A caller in the SAME account as the bucket needs to read an object. Access is granted if… - either the caller's identity policy OR the bucket policy allows it - both policies must explicitly allow it correct: 0 explain: Same-account access needs only one allow. Cross-account access requires both — the caller's identity policy and the bucket policy must agree. Q: Bucket names in S3 are… - unique within your AWS account - globally unique across all of AWS correct: 1 explain: S3 bucket names share a single global namespace. A name you choose may already be taken by another account, so most names need a project or random suffix. Q: A lifecycle rule that moves objects to Glacier after 90 days and deletes them after 365 days is an example of… - a transition action followed by an expiration action - a CORS configuration correct: 0 explain: Transitions move objects between storage classes; expirations delete them. Together they automate the cleanup-and-tiering that keeps a bucket cheap and uncluttered. Q: You want a client to download an object from a private bucket for the next hour without holding AWS credentials. The clean answer is… - generate a presigned URL with --expires-in 3600 - set the bucket policy to allow s3:GetObject from * correct: 0 explain: A presigned URL is a time-limited, signed link. The client fetches the object directly from S3 without credentials; the bucket stays private. ```