- Amazon S3 is probably the most heavily used storage service that is provided by AWS.
- Amazon S3 is a fully managed, object-based storage service that is highly available, highly durable, very cost-effective, and widely accessible.
- The smallest file size that it supports is 0 bytes
- The largest file size is 5 terabytes.
- S3 is a regional service and so when uploading data you as the customer are required to specify the regional location for that data to be placed in.
- Amazon S3 will then store and duplicate your uploaded data multiple times across multiple availability zones within that region to increase both its durability and availability.
- Objects stored in S3 have a durability of 99.99999999999% (eleven 9's) of durability.
- S3 stores multiple copies of the same data in different availability zones.
- The availability of S3 data objects is dependent on the storage class used and this can range from 99.5% to 99.99%.
- AWS ensures that the uptime of Amazon S3 is between 99.5% to 99.99%, depending on the storage class, to enable you to access your stored data.
- The percentage refers to the probability of maintaining your data without it being lost through corruption, degradation of data, or other unknown potential damaging effects.
- Data is stored in a bucket
- bucket name must be completely unique, not just within the region you specify, but globally against all other S3 buckets that exist, of which there are many millions.
- Default bucket limit is 100 (Can be increased)
- Object uploaded to your buckets are given a unique object key to identify it.
- Folders can be created and can provide additional management from a data organization point of view.
- S3 works at the bucket level and not a specific folder level, so unique keys will contain any folders in the bucket as well as the key for the itself.
- Bucket Naming Standards
- They must be at least 3 and no more than 63 characters long.
- They may contain lowercase letters, numbers, periods, and/or hyphens.
- Each label must start and end with a lowercase letter or a number.
- They cannot be formatted as an IP address (for example, 192.168.1.1).
- You can specify encryption at the folder level within a bucket. This can be done at folder creation.
- S3 Standard
- S3 Intelligent Tiering (S3 INT)
- S3 Standard Infrequent Access (S3 S-IA)
- S3 One Zone Infrequent Access (S3 Z-IA)
- S3 Glacier
- S3 Glacier Deep Archive (S3 G-DA)
¶ S3 Standard
- General-purpose storage class.
- Ideal for a range of use cases where you need high throughput with low latency with the added ability of being able to access your data frequently.
- Copies data to multiple availability zones,
- Offers 99.999999999% of durability across multiple availability zones and 99.99% availability across the year
- highest availability that S3 offers.
- Supports SSL, Secure Sockets Layer, for encrypting data in transit in addition to encryption options for when the data is at rest.
- Lifecycle rules allow objects to be automatically be moved to another storage class.
- Ideal for those circumstances where the frequency of access to the object is unknown.
- It can help to optimize your storage costs.
- Depending on your data access patterns of objects in the Intelligent Tiering Class, S3 will move your objects between two different tiers, these being frequent and infrequent access.
- When the objects are moved to Intelligent Tiering, they are placed within the frequent access tier, which is the more expensive of the two tiers.
- If an object is not accessed for 30 days then AWS will automatically move the object to the cheaper tier known as the infrequent access tier.
- Once that same object is accessed again, it will automatically be moved back to the frequent tier.
- Offers 11, 9s of durability across multiple availability zones offering protection against the loss of a single AZ.
- Its availability isn't quite as high as S3 Standard as it set at 99.9%. This storage class also has the added support of SSL for encrypting data in transit in addition to encryption options for when the data is at rest. S3 Intelligent Tiering also supports the lifecycle rules and matches the same performance throughput and low latency as S3 Standard.
¶ S3 Standard Infrequent Access
- This can be seen as the equivalent to the infrequent tier from the Intelligent Tiering class as it is designed for data that does not need to be accessed as frequently as data within the Standard tier, and yet still offers high throughput and low latency access, much like S3 Standard does. As with all other S3 storage classes, it carries that 11, 9s durability across multiple AZs, again by copying your objects to multiple availability zones within a single region to protect against AZ outages. It shares the same availability as Intelligent Tiering of 99.9 percent.
- cheaper cost than S3 Standard.
- Common security features such as SSL for encryption in transit and data at rest encryption is supported as well as management controls such as lifecycle rules to automatically move objects to an alternate storage class based on your requirements.
- It's designed for objects that are unlikely to be accessed frequently.
- It carries the same throughput and low latency.
- durability, although remaining at eleven nines
- only exists in a single availability zone.
- objects will be copied multiple times to different storage locations within the same availability zone instead of across multiple availability zones. This results in a 20% storage cost reduction when compared to S3 Standard. One Zone IA does, however, offer the lowest level of availability which is currently 99.5 percent and this is down to the fact that your data is being stored in a single availability zone. Should the AZ storing your data become unavailable then you will lose access to your data or even worse it may become completely lost should the AZ be destroyed in a catastrophic event. Again, life cycle rules and encryption mechanisms are in place to protect your data both in transit and at rest.
- Not enabled by default
- Can not be turned off once enabled, only suspended
- If suspended, all verions up to suspension are retained
- Only Latest version of an object is shown, but past versions can be viewed/accessed.
- Encours additional costs as it increses your storage amounts.
- Can be turned on during creation of the bucket or on an existing bucket (Found under properties of the bucket)
- When enabling versioning on an existin bucket, bucket items will have a version id of null until a change is made to the object (new version, deleted...) at which time, it will be assigned a version id.
- If you delete a file, the latest file will get a delete marker. When you list the files in the api, it won't show up.
- You can see the file using the version id in the API or the "show All" in the UI.
- To truely delete a version, you must specify the specific version id to delete.
- when server-access logging is enabled on a bucket it captures details of requests that are made to that bucket and its objects
- Could be required for Gov't audit requirements
- Logging all actions is not guarenteed
- Not enabled by default. Can be enabled during bucket creation or on existing buckets
- Requires specific permissions: Log Delivery Group
- Source and Target bucket should be in the same region
- Different buckets should be used for source and target.
- Permissions of the S3 Access Log Group can only be assigned via Access Control List (ACL)
- Manually setting permissions for this via an SDK= Must update the ACL.
- Acess Logs will ony be delivered if this is set to SSE-S3. Encryption with KMS is not supported.
- Log naming - YYYY-MM-DD-HH-MM-SS-uniquestring
- This feature is often used to meet a level of compliance known as WORM, (Write Once Read Many)
- Prevents them from being deleted, either for a set period of time that is defined by you or alternatively prevents it from being deleted until the end of time!
- The ability to add retention periods using Object Lock help S3 to comply with regulations such as FINRA, the Financial Industry Regulatory Authority.
- Can only be enabled if versioning is enabled.
- Once enabled, it can't be disabled.
- Prevents your users from performing a delete or an overwrite of any of the versions of your objects in the bucket throughout the duration set by the retention period.
- If you have very specific permissions, including
s3:BypassGovernanceMode , s3:GetObjectLockConfiguration, and s3:GetObjectRetention, then a user will still be able to delete an object version within the retention period or change any retention settings set on the bucket.
- Must specify a retention period which specifies the number of days when an object will be deleted
- The key difference between Compliance Mode and Governance Mode is that there are NO users that can override the retention periods set or delete an object, and that also includes your AWS root account which has the highest privileges.
- Any object added to a bucket configured for Compliance Mode means that the object will remain for the duration of the retention period.
- Must specify a retention period which specifies the number of days when an object will be deleted
- Legal Hold: Only appears for object versions and not at the bucket level and acts much like a retention period and prevents the object from being deleted, however, legal holds do not have an expiration date.
- The object will remain protected until a user with permissions of
s3:PutObjectLegalHold disables the legal hold on the object.
- If an object is already protected by a retention period, a legal hold can also be placed on the object. When the retention period expires, the object will still be protected by the legal hold regardless of the fact that the retention period has expired.
- Get, Delete, Put API calls against S3 will be logged into CloudTrail.
- Logging is configured in Cloudtrail.
- You must have an existing trail to configure.
- Uses CloudFront (edge locations)
- There is an additional cost for any data sent IN and OUT of S3.
- You will be given a new url to access the bucket via accelerator
- Your bucket name must be DNS compliant and not contain any ".".
- Get Service (List buckets)
- Put Bucket (create bucket)
- Delete Bucket
- Cross region copies using Put object - Copy