Amazon S3 Data Lifecycle Management

Key Takeaways

hello my name is Eric Bowman and I'm a senior storage specialist Solutions architect at AWS in this video I will introduce you to the storage class options within Amazon S3 and how you can use data life cycle management tools to optimize your Amazon S3 storage costs Amazon S3 offers a range of storage classes that you can choose from based on the data access resiliency and cost requirements of your workloads S3 standard is often the best option for active workloads as there is no incremental charges for data retrievals for workloads that are less active the infrequent access storage classes offer data storage cost savings compared to the S3 storage standard storage class this cost savings does come with a tradeoff in the form of a per gigabyte retrieval charge when data is accessed for archive workloads you can choose from three archive storage classes optimized for different access patterns and storage duration the Amazon S3 Glacier storage classes are purpose-built for data archiving providing you with the highest performance most retrieval flexibility and the lowest cost archive storage in the cloud S3 Glacier instant retrieval provides the same low latency access to data as the more active S3 storage classes and offers cost savings when compared to the infrequent access classes a restore operation that can take minutes to hours is necessary to access data stored in the S3 Glacier flexible retrieval and S3 Glacier deep archive storage classes for workloads with known inconsistent access patterns choosing the optimal S3 storage class can provide meaningful cost savings opportunities but often these access patterns change significantly over time this means the most costeffective storage class for a data set will likely change as data ages and becomes less active as data sets stored in grow they may contain millions or even billions of individual objects which can make storage class management a bit more challenging to simplify this task S3 offers life cycle policies that can be configured to automate transitions to infrequent or archive storage classes or expire data based on the age of the object life cycle policies can be configured to transition or expire all data inside a specific S3 bucket or filters can be used to limit the life cycle policy scope prefixes and object tags are often used to logically define unique data sets within an S3 bucket applying these filters to life cycle policies allows unique tiering and retention logic to be applied independently to each of these defined data sets additionally filters can be configured to limit the size of an individual object affected by the life cycle configuration or the number of previous versions to be retained when working with version enabled buckets to help with choosing the optimal S3 storage class for a data set S3 offers a number of features to help you better understand analyze and optimize your storage at scale S3 storage lens delivers organization wide visibility into object storage usage activity Trends and makes actionable recommendations to optimize costs and apply data protection best practices storage lens provides a single view of object storage usage and activity across hundreds or even thousands of accounts in an organization storage lens default dashboards are enabled for all AWS accounts and offer visualizations that help you identify quick ways to optimize S3 storage costs and opportunities to drill down deeper to generate insights at multiple aggregation levels additionally S3 offers S3 inventory S3 server access logs and cloudwatch metrics for fine grained insights into object in user level activities these tools can also be paired with services like Amazon Athena and Amazon quicksite to create custom queries and visualizations to help identify access patterns of data sets furthermore S3 storage class analysis can be configured to monitor data in a bucket prefix or object tag over time it will identify objects that are good candidates for transitioning to infrequent access or archive storage classes earlier we covered optimal storage classes for data with known or predictable access patterns but what if like many customers your data access patterns are unknown and Ever Changing that's what Amazon S3 intelligent tiering is for the Amazon S3 intelligent tiering storage class is designed to optimize storage costs by automatically moving data to the most cost-effective access tier when access patterns change S3 intelligent tiering monitors the access patterns of objects and automatically tiers objects between three access tiers new and active objects reside in the frequent access tier after an object has been inactive for 30 days S3 automatically tiers it to the infrequent access tier and after an additional 60 days to the archive instant access tier if at any point the object be uh becomes active again it is transitioned back to the frequent access tier starting the process again the opt-in archive access tiers are available for additional automated cost savings but will require a restore request which can take minutes to hours before the object is transitioned back to the frequent access tier thank you for taking the time to learn more about S3 storage classes in data life cycle management for information about Amazon S3 please visit our website at aws.amazon.com S3