AWS Series — S3 Cheat Sheet
S3 is Object based storage allows you to upload files.
- It is not suitable to install an operating system or run a database on
- File can be from 0 bytes to 5 TB.
- The total volume of data and the number of objects you can store is unlimited.
Files in S3 are stored in buckets. S3 is a universal namespace.
The bucket URL is https://bucket-name.s3.Region.amazonaws.com/key-name
Successful CLI or API uploads will generate an HTTP 200 status code.
Each S3 object has —
- Key — The object name
- Value — The data itself, which is made up of a sequence of bytes
- Version ID — Allows you to store multiple versions of same object
- Metadata — Data about the object you are storing
Securing Your Bucket with S3 —
- Buckets are private by default — When you create an S3 bucket, it is private by default (including all objects within it). You have to allow public access on both the bucket and its objects in order to make the bucket public.
- Object ACLs — You can make individual objects public using object ACLs
- Bucket Policies — You can make entire buckets public using bucket policies
- HTTP Status code — When you upload an object to S3 and it is successful, you will receive an HTTP 200 code.
Hosting a Static Website on S3 —
- Bucket Policies — You can make entire buckets public using bucket policies
- Static Content — You can use S3 to host static content only(not dynamic)
- Automatic Scaling — S3 scales automatically with demand.
Versioning Objects —
- All versions — All versions of object are stored in S3. This includes all writes and even if you delete an object
- Backup — Can be a great backup tool
- Cannot be disabled — Once enabled, versioning cannot be disabled — only suspended
- Lifecycle Rules — Can be integrated with lifecycle rules
- Supports MFA — Can support multi-factor authentication
Storage Classes —
Amazon S3 Glacier Instant Retrieval is a new Archive storage class that delivers the lowest cost storage for long-lived data that is rarely accessed and requires milliseconds retrieval.
Lifecycle Management —
- Automates moving your objects between different storage tiers
- Can be used in conjunction with versioning
- Can be applied to current versions and previous versions
S3 Object Lock and Glacier Vault Lock —
- Use S3 Object Lock to store objects using a write once, read many (WORM) model
- Object Lock can be on individual objects or applied across the bucket as a whole.
- Object Lock comes in two modes: governance mode and compliance mode.
- With Compliance mode, a protected object version cannot be overwritten or deleted by any user, including the root user in your AWS account.
- With Governance mode, users cannot overwrite or delete an object version or alter its lock settings unless they have special permissions.
- S3 Glacier Vault Lock allows you to easily deploy and enforce compliance controls for individual S3 Glacier vaults with a vault lock policy.
- You can specify controls, such as WORM, in a vault lock policy and lock the policy from future edits. Once locked, the policy can no longer be changed.
Encrypting S3 Objects —
- Encryption in Transit — SSL/TLS, HTTPS
- Encryption at Rest: SSE — Server-side encryption, SSE-S3 (AES 256-bit), SSE-KMS, SSE-C
- Client-side Encryption — You encrypt the files yourself before you upload them to S3
- Enforcing Encryption with a Bucket Policy — A bucket policy can deny all PUT requests that don’t include the x-amz-server-side-encryption parameter in the request header.
Optimizing S3 Performance —
- bucket-name/folder1/subfolder1 → /folder1/subfolder1
- You can also achieve a high number of requests: 3500 PUT/POST/DELETE and 5500 GET/HEAD requests per second per prefix.
- You can get better performance by spreading your reads across different prefixes. For example, if you are using 2 prefixes you can achieve 11,000 requests per second.
- If you are using SSE-KMS to encrypt your objects in S3, you must keep in mind the KMS limits.
- Uploading/downloading will count toward the KMS quota
- Currently, you cannot request a quota increase for KMS
- Region-specific. However, its either 5500,10000 or 30000 requests per second.
- Use multipart uploads to increase performance when uploading files to S3.
- Should be used for any files over 100 MB and must be used for any file over 5 GB.
- Use S3 byte-range fetches to increase performance when downloading files to S3.
S3 Replication —
- You can replicate objects from one bucket to another — Versioning must be enabled on both the source and destination buckets
- Objects in an existing bucket are not replicated automatically — Once replication is turned on, all subsequent updated objects will be replicated automatically.
- Delete markers are not replicated by default — Deleting individual versions or delete markers will not be replicated
