Amazon S3 security and access management best practices - AWS Online Tech Talks

Key Takeaways

[Music] hello and welcome everyone if you are here you are using S3 or interested in using S3 you are putting your data in Cloud it's important to secure the data and ensure right people have access and nobody else I'm Kavita and later you will hear from shck and we both would be speaking about ways to secure your data in S3 bucket and provide right access increasingly customers are transitioning all their applications to the cloud while many are starting ground up in the cloud as a result of this shift more and more customer workloads are moving to Amazon S3 customers choose Amazon S3 for a lot of different reasons for some it's about high availability High throughput the scale they can reach with Amazon S3 the love NS of durability and the ability to store data across multiple regions with cross region applications for some it's wide range of cost optimization capabilities we provide to store millions and billions of objects within S3 for some customers it's about security and management features that we offer giving you the ability to control access at the object bucket and account level S3 is a powerful service for storing your data but there are situations where you want to automate the manage ment of your data to achieve business goals typical business goals are to understand usage verify security increase efficiency drive down cost or migrate data to other environments selectively S3 provides you with tools and capabilities to manage your data at scale while you continue to expand your storage in the cloud in this session today we will use a case-based approach to learn about these tools and capabilities that will help you manage your data at scale so the first scenario we are going to see is that your organization is hosting thousands of S3 buckets across hundreds of accounts these buckets have data that is often accessed by multiple uses you you as a security engineer or an developer would need to mandate some complaints guidelines so that the objects are not available publicly and all the objects are encrypted in your buckets how can we achieve this let's review some S3 features that will help you simplify Security Management at scale the first one is S3 block Public Access and the second is S3 default encryption we would go into detail about each of these sections so the very first thing the main concept for today is block Public Access if you take nothing else away from the stock take this go after the stock turn on the Block Public Access for all your AWS accounts what is block Public Access Amazon S3 block Public Access is a feature that applies a blanket prodection against public access it guards you against accidental public access this feature can be set at the Bucket Level or at the account level it provides protection against permissions from accles bucket policies or both when you enable block public access to a bucket entities outside the account does not have access unless you explicitly provide access you can provide this access in number of ways one being using the bucket policy or using IM am policies or that if you have a a organizations you can also do it with SCP service control policies as well as well uh we will now go through some examples to see how you can prevent public access to S3 buckets here we can see that there are two buckets one is re a really public website and the other is cross account testing and these both have either the object or the bucket itself has access to entities outside your AWS account now you can go on to block Public Access at the account level turn the setting on you can turn on the setting for all the four different features that we provide and once you enable this setting it gives you overarching protection across all your buckets and overrides your individual bucket setting so now you have turned on the this and let's see what happens so you can see that none of the buckets are public anymore after I turn on block Public Access at the account level all of the buckets are protected the two buckets that were flaged before are now secured and are not publicly accessible anymore more uh apart from block Public Access S3 also provides you some ways on how you can effectively manage public access the first one is a im am called IM IM access analyzer you can use access analyzer to understand how your buckets are being shared the second one is like if you do have public buckets there are very less scenarios for why you need to have public buckets but if you do have one keep those public buckets in a dedicated AWS account so you can ensure that all the other adws accounts that you have you can turn on the BPA block Public Access at the account level and always use bucket policies oracles when you need to provide permissions to your account the second feature that we're going to speak about is Amazon S3 encryption by encryption what do we we mean the very first is that we encrypt on the client side we can do this using AWS SDK or you can use your own application to provide client side encryption the second is that the data is encrypted during Transit and finally we provide several modes to encrypt data address at S3 some of the options are Amazon is three managed Keys AWS Key Management Service and customer provided keys today we have organization wide visibility into your encryption metrics in storage lens while we have this you should also consider setting up default encryption setting for your S3 buckets with default encryption you have the ability to set up S3 buckets where the data is automatically encrypted by default it is a onetime Bucket Level setting where you define your encryption key whether S3 managed key or customer managed KMS key and all your subsequent uploads to your bucket are automatically encrypted we also launched bucket key configuration last year that will help you reduce your KMS cost up to 99% now we are moving on to our second scenario your organization is hosting tens of thousands of S3 buckets across hundreds of accounts these buckets have shared data that is off often accessed by multiple users you are looking for ways to simplify policy management for cross users across the organization so how do you achieve this we can do that using S3 object ownership S3 access points and I am access analyzer before I dive deep into each one of these things I want to go into multiple mechanisms for access management there are multiple ways to provide access to your buckets today we'll be seeing about IM IM policy bucket policy and access control list so for IM am policies or where you define what resources the IM rules in your account can access for bucket policy you define what different who different users or uh rules can access your particular bucket and finally we have accles or Access Control list which predat IM am and was the original way that S3 used to Grant access to bucket object we will dive deep into each one of these in the upcoming slides so IM IM policy IM policy defines what actions can this IM am user perform you attach this IM am policy to an IM am user or an am rule and it has three mandatory elements they are effect which which is either allow or delay and then it's actions you define what actions can be performed by this user and finally where those actions can be performed on which resource in this example we have an IEM policy that allows an IM am user to read and write objects to the bucket called sample bucket who can apply these IM am policies if you are the am administrator for your AWS account you define the policy and attach it to an IM am user or IM am rule within your account you can further allow an IM am user to assume an IM am role with this role assumption the IM am user inherits the permission of that IM Rule and the next is S3 bucket policy bucket policy is something that you attach to a bucket to a resource in AWS there are different types of there are different resource policies that's different from the I am policy that we saw before we have bucket policies we have VPC policies and we have KMS policies and different policies that you can attach it's it's it's a policy that can be attached to a resource so the bucket policy is very similar to IM am policy such that it uses the same Json format the bucket policy defines who can access my bucket and what actions can that user perform the uh the the bucket policy requires one additional statement apart from effect action and resource and that is called as principle this is where you specify who this policy is applicable to here in this example the bucket policy allows the AWS account ID with the 12 mons to read and write objects to the resource sample bucket and finally we have access control list Access Control risks are one of the resource-based options that we have available to manage buckets and objects you can use accles to Grant basic read or write permissions to other AWS accounts a majority of modern use cases in Amazon S3 no longer requires the use of accles and we recommend that you disable accles except in unusual circumstances where you need to control access for each object individually I just want to repeat this again try not to use tles and prer using bucket policies when needed sock will be discussing more about this in the upcoming slides I'm going to hand it over to shaik now to speak further about cross aod management thank you Kavita so I want to start with the concept of cross account rights so cross right is a really common situation for many of our customers with multiple accounts let's use an example to show how cross account rights are done today so you can imagine that you own account a and have a bucket that serves as your data Lake data Lake meaning multiple shared data sets act being accessed by a variety of your users account B for example in that data Lake wants to write data to your bucket now there are three complexities that you need to pay attention to in a cross account right situation whenever account B writes data to your bucket objects are owned by account B now account B needs to give you permission to read via acle or Access Control list third because you don't own these objects because remember account B wrote These objects to your bucket you cannot use bucket policies or IM policies to give other accounts access access you have to do that through Access Control lists or akle now this used to be the old school way of managing granular object level permission way before Amazon or AWS started its own Central IM am or access management policies now this can present a lot of challenges such as enforcing bucket policy to objects owned by the bucket owner in account a and using a bucket policy is a modern best practice approach to sharing your data with other users you use bucket policies to share data with other AWS accounts and then create a security parameter to deny access so as you can see here Ackle assigns ownership that that affects bucket policy applications so objects written by external account are owned by that specific account the result is that the bucket owner cannot further share that object via a bucket policy so we wanted to take a look at this problem long and hard and solve this issue for our customers so I want to talk through one of our recent launch that that that we made just a couple of months ago we want to help customers in that use case and are adding the capability to S3 object ownership a bucket setting then that you can apply to all of your buckets in your account so what is object ownership object ownership is a new feature which is now an enforced setting as a bucket owner we launched it to give customers the right answer for all of them today it does two things things first it makes sure that accles or Access Control lists are disabled and second it makes sure that you as the bucket owner own every single object in your bucket and we have made it as a setting across your Bucket Level it means that bucket policies from here on out will be the sole and the only means of sharing data and ACC control lists are completely out of picture now as a result of bucket ownership setting bucket owner enforced you as now will own every single object in your bucket so now let's look at our previous example when you have a data Lake in your in a bucket in account a and there are multiple writers of data and objects into that bucket so let's take a closer look at how this works so as a part of account a if you have bucket owner enforced as an ownership setting for your bucket regardless of whichever account wrote to that bucket for example that object could be written by account a or by account B they will always be owned by account a and as a result account a will continue to reserve the right of sharing access to this shared data sets shared objects with just using bucket policies so how does this compare to the setup before now compared to before you wouldn't have to deal with Ackle at all for example when account B writes to your bucket the right doesn't need to specify and access control list you own all of the objects you can now use bucket policies to control access for all of these objects regardless of where those objects came from from account B account C account D or account e or hundreds of other accounts you can use bucket policy to give other IM roles or users specific access and granular permissions to all of your objects now so you you might be you ask asking yourself so how do I turn on um this disabling acle and bucket owner enforced we have made it as simple as possible it's a simple Bucket Level setting and we are adding this as the third option to S3 object ownership called bucket owner enforced now you can apply it to a new bucket or any of your existing buckets in your account and if you want to turn it off you can simply do it at any any time and when you do that it will automatically restore the previous accles so you can go back to status quo with just a click of a button now we talked about object ownership kava talk to you about different kinds of policies bucket policies IM roles and block Public Access I want to finish off with a specific feature that is called S3 access points which becomes really important as your shared data sets grow as you grow your data sets and add multiple shared data sets to your buckets with multiple client access your bucket policies get more and more complex Amazon S3 access points helps you simplify access management that contain shared data sets as you can see from the illustration and the diagram here access points help you divide access to your bucket into multiple groups of users each with its own access policies in the same bucket for instance you can divide cross account users into three specific groups Finance sales and developers each group can read data from a designated access point that has its own access point policy so you can think of your entire bucket as a house and access points being specific small doors that can only access specific parts of your house with its own unique key as a result of this you don't have to create each bucket for each of your shared data set you can just use access points to make sure you give granular permissions to different teams within your organization to have specific access to parts of your data sets or parts of your bucket now we have made multiple improvements since launching access point last year we launched S3 access point aliases that makes accessing your access point from other clients significantly easier with access points Alias each access point receives its own unique bucket based Alias that can be used to replace bucket name Uris for data access now you might be thinking why did we build this so when we first launched access points customers told us that they love the ability to have granular access for parts of their data sets to specific users but they also wanted to use access points with other applications AC cross AWS and third-party apps so we added access point Alias so your applications can talk to access points just like they would talk to your bucket this means that you can use these same access point aliases for all of your S3 API requests to talk to services like EMR and they will automatically connect to the access point not requiring any code changes now while we are in the topic of access points and granular permissions and policy I want to talk about policy validation with access analyzer now access analyzer guides you to author and validate secure and functional policies with more than 100 policy checks with automated reasoning you can use these checks while creating new policies or to validate existing policies you can use the policy validation to confidently check that your policies provide intended access before they're attached to IM am principles or your S3 buckets now this saves you time and gives you a preview of your security posture while at the same time simplifying permission management at scale now this becomes especially important for our customers as Kavita mentioned earlier that are using S3 buckets for their data Lake use cases with hundreds of thousands of data sets with hundreds of thousands of users with one click with policy validation you would be able to check your entire security posture across your different data sets now policy validation this is a feature that contains over 100 encoded validation based on AWS policy writing best practices it analyzes the policy to dynamically generate findings that help developers and storage administrators identify issues during authoring findings are categorized into errors warnings and suggestions based on their impact error find ings are generated when a policy statement is nonfunctional for example it will generate an error when developers wrongly use S3 list buckets which is an invalid action here instead of S3 list all my buckets which is the correct Action Now warnings are also generated when a statement contains constructs that may make the policy susceptible to security risks for example the policy validation generates a warning when a policy con construct allows an IM am principle to pass any role to any service which may result in an escalation of privileges finally suggestions are generated to recommend stylistic improvements that don't impact the access Allowed by the policy for example as you can see here it generates a suggestion to remove redundant actions and help author well structured policies within the policy size limits that are easy to understand and simple to maintain policy validation is turned on by default in the Json policy editor in the IM am and S3 console in the AM console you can view findings generated by the policy validation when creating a managed or inline policy for your IM principles meaning your roles and your users in the S3 console you can view findings generated by the policy validation in the S3 policy editor or when creating a bucket or an access point policy the policy validation dynamically gener Ates and updates its findings as you edit your policy these findings Guide You by providing actionable steps to fix the issue in a policy statement along with links to AWS documentation for easy reference and user guides you can click on a specific finding to identify the location of the associated statement within that policy these findings get resolved once the once the policy is fixed automatically now to wrap up we looked at two specific scenarios today we talked about how to meet compliance requirements and improve your security posture with features like S3 default encryption and S3 block Public Access Kavita also walked you through how different policies work with the S3 for you to be confident about the permissions that you're generating and granting to different users in your organization we also discussed how to simplify Access Control even further how you can remove Access Control lists from your bucket by turning on bucket owner enforced setting in S3 object ownership as a result your permission models become exceedingly simple so that regardless of whichever accounts are writing to your different data sets and creating new objects as an as the bucket owner you have automatic ownership to all of those objects we also discussed S3 access points which opens a specific secured window to your bucket that you can assign permissions to to different users and then finally we discussed policy validations and access analyzer to audit all of your permission controls and make sure that you are doing the right things by your security posture I I'm so excited to finish this Tech talk with you and give you more thoughts into how AWS and Amazon S3 are continuing to help you improve your security and access permissions for your SD buckets and data sets Kavita and I wanted to thank you for your time and attention and we can't wait to talk to you again [Music]