Handbooks / AWS / Chapter 3

S3

7 pages · ~5 min✓ Reviewed

Builds on EC2 and VPC. Next up: RDS and DynamoDB.

AWSPage 1

What This Handbook Covers

1. Buckets and Objects

  • Bucket: globally unique name, one Region
  • Object = key + data (up to 50 TB) + metadata
  • Single PUT max 5 GB; use multipart above
  • 'Folders' are just key prefixes with /

2. Storage Classes and Lifecycle

ClassUseMinimum
Standardfrequentnone
Intelligent-Tieringunknownnone
Standard-IA / One Zone-IAinfrequent30 d
Glacier Instant / Flexiblearchive90 d
Deep Archivearchive180 d
  • Lifecycle = filter + Transition (down only) + Expiration
  • Also abort incomplete multipart uploads
  • Archived objects need a restore before GET

3. Versioning

  • States: unversioned, enabled, suspended; never removable
  • Plain DELETE adds a delete marker
  • DELETE with versionId is permanent
  • Expire noncurrent versions to cut cost

4. Policies, Block Public Access, Presigned URLs

  • Explicit Deny beats any Allow
  • bucket for ListBucket, bucket/* for GetObject
  • Block Public Access overrides public policies and ACLs
  • Presigned URL: time-limited access to one object
bash
aws s3 presign s3://bucket/key --expires-in 3600

CLI makes GET URLs only; PUT URLs need an SDK

AWSPage 2

Buckets, Objects and Strong Consistency

1. Bucket and Object

  • Bucket: globally unique name, lives in one Region
  • Object = key + data (up to 50 TB) + metadata
  • Flat namespace: 'folders' are only key prefixes with /
  • Single PUT capped at 5 GB

2. Uploads and Consistency

  • Over 5 GB: multipart upload; prefer it above 100 MB
  • Incomplete uploads are billed: abort or lifecycle-clean
  • Strong read-after-write: PUT, overwrite, DELETE, LIST
  • No eventual-consistency window
  • Same key, concurrent writers: last writer wins

3. Everyday Commands

bash
aws s3 cp file s3://bucket/key
aws s3 ls s3://bucket/prefix/
aws s3 sync ./dir s3://bucket/dir --delete
aws s3 rm s3://bucket/key --recursive
Careful

sync --delete removes destination objects missing from the source.

4. Interview Point

  • File system? No, a flat key-value object store
  • Eventually consistent? No, strongly consistent today
  • Same key at once? Last writer wins
  • Multipart when? Over 5 GB, ideally over 100 MB
AWSPage 3

Storage Classes and Lifecycle Rules

1. Classes and minimums

ClassUseMinGet
Standardfrequent-ms
Intelligent-Tieringunknown, auto-tiers-ms
Standard-IAinfrequent30d, 128 KBms
One Zone-IAinfrequent, 1 AZ30d, 128 KBms
Glacier Instantarchive90dms
Glacier Flexiblearchive90dmin-hrs
Deep Archivearchive180d~12h / ~48h
  • Early delete is still charged
  • Archived objects need a restore before GET
  • Flexible: Expedited, Standard, Bulk

2. Lifecycle rules

  1. Filterprefix, tag, size
  2. Transitiondays, down only
  3. Expirationdelete
  • Waterfall: Standard → IA → Glacier → Deep Archive
  • Also set AbortIncompleteMultipartUpload, NoncurrentVersionExpiration
  • Rules run async, about daily

3. Interview Point

  • Tiering over IA? Unknown access; no retrieval fee
  • Back to Standard? Transitions only go down; copy instead
  • Tiny objects to Glacier? Per-object costs beat savings
  • Old versions? Expire noncurrent versions
AWSPage 4

Versioning and Delete Behaviour

1. Three States

  • Unversioned, enabled, suspended
  • Enabled can only be suspended, never removed
  • Existing versions are kept

2. Overwrite and Delete

ActionResult
PUT same keyNew version ID, now current
DELETE, no versionIdAdds a delete marker; object looks gone
DELETE with versionIdPermanent removal of that version
Delete the markerObject is restored
MFA DeleteNeeds MFA for permanent deletes, versioning changes
Still billed

Older versions stay after a delete marker and keep costing storage.

3. Enable and Inspect

bash
aws s3api put-bucket-versioning --bucket b --versioning-configuration Status=Enabled
aws s3api list-object-versions --bucket b --prefix key

lists versions and delete markers

4. Interview Point

  • Recover a deleted object? Delete the marker, or copy an older versionId
  • Why does cost grow? Every version is billed; expire noncurrent versions with lifecycle
  • Disable versioning? No, only suspend; old versions stay
  • Current vs noncurrent? Latest version vs all older ones
AWSPage 5

Bucket Policies, Block Public Access and Presigned URLs

1. Bucket Policy

  • Resource-based JSON policy, 20 KB max
  • Needs a Principal; can grant cross-account access
  • Explicit Deny beats any Allow
  • Same account: an Allow in IAM or policy
ARNCoversExample
arn:aws:s3:::bucketbucketListBucket
arn:aws:s3:::bucket/*objectsGetObject
  • One ARN form only gives AccessDenied for the other

2. Block Public Access

  • Four switches: block public ACLs, ignore public ACLs, block public policies, restrict public buckets
  • Account and bucket level; overrides policies and ACLs
  • On by default for new buckets
  • ACLs off by default (BucketOwnerEnforced)

3. Presigned URL

  • Time-limited GET for one object (CLI: GET only; PUT needs SDK)
  • aws s3 presign s3://bucket/key --expires-in 3600
  • Default 3600 s
  • Max 7 days with IAM user keys
  • Role or session credentials expire it earlier

4. Interview Point

  • Share a private object? Send a presigned URL.
  • Early expiry? Role or session credentials end first.
  • Policy vs IAM? They combine; an explicit Deny wins.
  • Org-wide? Enforce Block Public Access on every account.
  • Leaked URL? Use short expiry and a least-privilege signer.
AWSPage 6

S3 Cheat Sheet

1. Model

  • Bucket: globally unique name, one Region
  • Object: key + data up to 50 TB; 5 GB per PUT
  • Flat keys; strong read-after-write; last writer wins

2. Classes and Versioning

ClassMinimum
IA classes30 days, 128 KB
Glacier Instant / Flexible90 days
Deep Archive180 days
  • Intelligent-Tiering: unknown patterns
  • Archived objects need a restore
  • Versioning: suspend only, never removed
  • Delete adds a marker; versionId delete is permanent
  • Lifecycle: expire noncurrent, abort incomplete multipart

3. Access

  • Explicit Deny wins
  • bucket ARN: bucket actions; bucket/*: objects
  • Block Public Access overrides public policies and ACLs
  • Keep ACLs disabled

4. Commands

bash
aws s3 cp f s3://b/k
aws s3 ls s3://b/p/
aws s3 sync ./d s3://b/d --delete
aws s3 rm s3://b/k --recursive
aws s3 presign s3://b/k --expires-in 3600
aws s3api put-bucket-versioning --bucket b --versioning-configuration Status=Enabled
aws s3api put-bucket-lifecycle-configuration --bucket b --lifecycle-configuration file://l.json
aws s3api list-object-versions --bucket b
AWSPage 7

Check yourself

A policy allows s3:GetObject on arn:aws:s3:::my-bucket. Reads still get AccessDenied. Why?
  • GetObject is an object action, so the Resource must be arn:aws:s3:::my-bucket/*.
  • The bare bucket ARN only covers bucket actions like ListBucket.
Versioning is on. You run aws s3 rm s3://b/report.csv and ls shows nothing. Is the data gone?
  • No. A plain delete only adds a delete marker.
  • Older versions remain and are still billed.
  • Delete the marker by its versionId to restore the object.
A 20 KB object moves to Standard-IA and is deleted 10 days later. Did that save money?
  • Probably not. IA bills at least 128 KB.
  • It also bills a 30-day minimum, so the early delete is still charged.