Building serverless applications with infrastructure as code - AWS Online Tech Talks

Key Takeaways

[Music] hi everyone thanks for joining me today for this session which is about building serverless applications with infrastructure as code if you don't know what infrastructure is code is don't worry we're going to get everyone up to speed in just a little bit so my name is talia i'm a senior developer advocate for serverless here at aws and my job is just to build serverless things and then teach the world about them so before becoming a dev advocate i was a test engineer i did things like end-to-end testing and automation and all of that testing stuff that you developers hate doing that was my job this is my twitter handle in case you want to contact me so today we're going to talk about a few topics we're going to go through a lot in the time that we have so first we're going to talk about infrastructure as code specifically how it relates to serverless then we'll talk about what it means to be a serverless application and what that looks like we're going to make a sam template and then we're going to make that template reusable and so when i say template that's basically what infrastructure as code is one of the huge benefits of infrastructure as code that we're going to talk about today is its reusability so you create a template and then you can put it in different places use multiple accounts things like that we're then going to prepare for deployment and then we'll actually deploy the application at the end and then finish off with a demo so let's get started so when we say serverless what do we mean we mean it's the removal of server operations and this is an important distinction for customers because it allows you all to focus on the building of the application rather than the management and scaling of the infrastructure to support the application these are the four tenants that define serverless as an operational model firstly there's no infrastructure to provision or manage there's no servers to spin up operator patch serverless applications automatically scale by unit of consumption rather than by server unit and automatic scaling is a really big win for companies because generally you create multiple mvps and you don't want to build all of them out for scalability serverless has a pay for value billing model which means you only pay for what you use not for idle time so if you have an idea that's not taking off some of them can get expensive and this means basically what you can do is you can experiment more so let's say i have an idea for a new feature and bob has a competing idea for a new feature bob and i can a b test our features and regardless of which feature has a higher conversion rate it's not going to cost much because you only pay for what your customers are using and then you can use that data to make better decisions in your product lastly serverless applications have built-in availability and fault tolerance so there's no need to architect for availability because it's built into the service and this is important because things will break unexpected things happen data centers have outages there's earthquakes spaghetti and meatballs can fall from the sky so what do you do if the servers handling your project fall over how do you know that all of your versions are in sync with all of the regions have no fear serverless is here my friends so serverless applications have built in availability and fault tolerance so we've got you covered so let's talk about infrastructure as code and templates so when you're building a serverless applications a lot of times when developers start building whether they're new or whether they just want to get started with serverless they start in the aws console and so what happens is you choose the services you want and then you connect them right so you build your lambda function here you choose what you want as a trigger for your lambda function here you choose what you want as the destination here you can create all the resources you need right there from the console and this is a really really great way to learn you know you can see everything visually in the ui the only problem comes when you want to move your resources to different accounts or you want to move them from one environment to another or just somehow replicate what you've done in the console chances are you're not going to know exactly how you set up the resources in the first place and exactly which configurations you set so to solve this problem we're going to look outside of the console to templates and that brings us to infrastructure as code so templates are a way to store our infrastructure so that it can be reusable with infrastructure as code you are automating the provisioning process so that means you're not going to go to the console click on the services you want create the resources from the console etc you're going to use a tool called sam which we're going to talk about in a minute and you're going to use sam to deploy so we're going to instantiate infrastructure using configuration files basically what this means is you have this template with all the steps and configuration for your application that you can reuse whenever you need this allows you to treat those configuration files as code and what do you do with code you version it you put it in some kind of repo and github or code commit or something like that so you can deploy a version work on it and then merge a new version and you can roll back if you need to so when you when you move to infrastructure as code you can do the same thing with your infrastructure so if something breaks and your infrastructure isn't working well let's roll it back or let's try a new version or let's move it over here to this other environment finally this allows you to eliminate configuration drift through automation this is that classic well hey it works on my machine shenanigans so let's say you get paged one night because there's an incident for your mobile application and you look at the logs and identify the problem in order to fix it you need to update a specific configuration in production so you make the change in production and you go back to sleep because it's really late although you've fixed the issue you've just created an even bigger divide between your staging and production environments because you did not make the same change in your staging environment many times staging environments are not the same as production because of changes made during incident management this is called a configuration drift so with infrastructure as code rather than having that shift in environments you have one central authority that is this template and then you push the infrastructure changes through so now at aws we have several infrastructure as code solutions the primary one is aws cloud formation a lot of the other ones are built on top of that the one we're going to talk about mostly today is aws serverless application model or sam and then there's also the aws cloud development kit or cdk which just allows you to write a little bit more code so you do have a lot of options here again one of the iac tools or infrastructures code tools that aws provides is called sam the serverless application model and the way that i like to think about this is just how like react is a framework for javascript um sam is a framework for building serverless applications it's a good way to think about that this little guy is sam the squirrel he's the sam mascot sam comes in two parts the first is that is sam templates which we've talked about which allows you to version your infrastructure as code and i'm going to show you an example of this the second is the sam cli or command line interface this is a utility that you install on your local machine that helps you with local development debugging builds and deployment for your serverless application here's an example of what a sam template looks like it's written in yaml and you can see that there's a few resources being created here we have an aws lambda function a dynamodb table and an api so i created the lambda function i added the corresponding iam role that lets the lambda function talk to the dynamodb table and there's a lot of options when it comes to policies to add to your services and these sam templates simplify them for you so you can change this to a write policy or whatever you need we're also creating an api from api gateway so your event source for your lambda function here is this http api and you can configure these templates to add any aws resources and create your application and you have a lot of flexibility to build exactly what you want so with these 20 lines of code this on the left becomes this on the right so you have the lambda function that's triggered by api gateway the dynamodb table that stores the data and now what's neat about this is what's already created for you is the permission for api gateway to invoke the lambda function that's what happens automatically for you and sam when you set api gateway as the event source for the lambda function and then you added the policy thing you can only read from the table now we're going to go through the steps to create a serverless application using sam and sam templates the first thing you need to do is install sam and you can do this with this link here and you can use this handy dandy qr code or go to s12d.com sam install once you have sam installed the first command you need to run is sam and knit when you do that you need to fill out a few things first you need to choose what kind of template you want we give you some templates to start from you don't have to build all of this from scratch you can do a sample hello world application like the one shown here you can do a web backend which we have a template for we have the serverless patterns collection that lives in serverless land which i'm going to show you a little bit later these are just a few templates that you can use to start building your application then you have your run time that you get to select so you can choose python ruby go java.net core or maybe you want to use a custom runtime so you have a lot of options here then you can choose a dependency manager if your runtime requires it and the result is something like like this on the right so you get the readme with some instructions you have the events that trigger the lanza function and then the hello world folder with all the information about the hello world function and then you have the template.yaml file which is the sam template that will create the resources that we just saw now the whole goal of infrastructure as code is to make it reusable so what if i want to redo something um i want this process to be simple if i want to duplicate something in another environment or if i want to just not go through all of the configuration steps i want that to be simple i want the template to be reusable now why is this why why use reusable templates in an ideal world everyone would test in production and if you know me or have heard me speak before you know how passionate i am about testing in production but sometimes it's not an option for you and your infrastructure and your team is not set up to test in prod so you have staging environments or test environments in play so when we're dealing with serverless the best practice is to have a developer account for each developer you use the same template to do that also your environments like beta staging and production should should all have their own account there is inherent security in this when you're saying that only folks who get to deploy to production should have access to production same with staging so that's one thing you get out of it by breaking these out and using the same templates you ensure consistency across these environments this is great when you're on boarding a new developer and they're setting up their environment and someone gives them a book like this like a huge book and says this is how you get your machine set up versus if you're using a template you just give them the template and say hey deploy this done you have everything set up you have your environment set up and all of your configurations set up so it makes developer onboarding also just a lot easier so when you make templates reusable you're saying you want to use it regardless of which environment you're in so whether that's staging or beta or production wherever you are now in order to do that you need to parameterize the template you make it have dynamic values that change based on how you're approaching it and there's a lot of ways to do this the first way we can parameterize your template to make it more dynamic is through template parameters and these are parameters declared through the template itself and will be passed into the lambda function these are added or created at update or deploy and you cannot change them they can be typed and validated and they're stored on the deployment machine this isn't where you want to keep your secrets this is where you want to keep your environment information is it dev or is it staging is it prod it's for very simple things but not for secrets or passwords the second is environment parameters these are added at creation or update and stored in an open text on the console so again not a good place to store secrets because everyone has access to the account can see them and they are dynamically referenced in the template so that you can grab them through the code but again you cannot change them the next way is through the aws systems manager parameter store and the aws secrets manager so these are very similar in how they work you can add these at creation update or run time they can be accessed in your code so if it's something you need to grab at every invoke then this might be a good place to store them the systems manager parameter store works with some resources not all and the secrets manager works with a lot more resources and then the final way is through app config so aws app config allows you to set configurations for your lambda functions we also have this really cool thing we just released called extensions and this is good if you want to change something a lot but you don't want to call it every single time a lambda function is invoked another way of parameterizing your templates is through functions and parameters so cloudformation has this idea of intrinsic functions and allows you to insert logic into your templates you can do some conversion like base64 you can substitute strings things like that so there's a lot of ways you can grab dynamic values as needed then we have what's called pseudo parameters where we need to reference what region you're in and you don't want to hard code that because sometimes you don't know so instead you can use a pseudo parameter like aws colon colon region same with account number and all the rest of these listed here okay so now we have our template um and now we want to deploy it we want to actually deploy our application so the sam cli has this handy dandy little command called sam build there's a lot of things that happen when you run sam build so first of all you can build with native runtimes you can also do a custom build using a make file and this gives you a lot of flexibility so just to know a lot of the runtimes are supported by containers so if you do sam build dash dash use container then you'll grab a container that's based on the environment that the lambda function will run in and it'll build that out for you you can also pass a cached or parallel flag the cash flag will say hey cash this build so let's say i have 10 lambda functions and i make a change to just one of them when i do the build nine of them will be cash but only one would need rebuilding and this makes the build much much faster you can also build in parallel now so i can say dash dash parallel and it will build them all at the same time and this makes local development much faster as well so you can also do omission if you want to omit certain files so you have a lot of flexibility here okay so when you run sam builds it creates a separate folder and isolates each of their functions into their own build it takes all the dependencies based on the dependencies manifest and will grab everything you need for that lambda function to work and then it will zip it up and push it out or if you're using a container image it will create a local image and then push it to an amazon ecr repository when you run sam build or sam package later okay so what have we done until now we've created our application we've prepared our template to work across multiple environments and we've learned how to build the application and prepare it for deployment so our application's ready to go so how do we deploy it so all we're going to do is go into the sam cli and run sam deploy so sam deploy deploys or updates serverless applications on the cloud it manages the s3 bucket for deployment artifacts and it saves all of the deployment options to a configuration file and then when it does a deploy it creates a cloud formation change set so all that is it's basically a diff of what what was there and what's there now and it compares the template to what was already out there and if it's a brand new deploy it's just going to deploy all of it if it's not a brand new deploy then it only updates the changes or it'll kick it back if there's a problem and say hey this isn't going to work you know fix these issues so you can use sam to deploy to multiple environments and this creates that consistency across the environments and we just saw all the steps of how to do this now we're going to see this in action i'm going to show you guys a demo we're going to build a quick application so in this scenario anytime a new item is added to a dynamodb table the invoked lambda function logs the event in amazon cloudwatch logs so what we're going to do here is um to build our application we're going to head to serverlesslands.com patterns and use one of the predefined sam templates on the left hand side we're going to filter patterns by dynamodb and then choose the dynamodb to lambda pattern then we're going to click on the download text to copy the commands to the clipboard here you can see that full sam template we're going to be using to deploy let's copy the download text then in your terminal paste the download instructions and all this does is it clones the repo and changes directory into the right place then you run sam deploy dash dash guided remember this deploys your application there are some prompts here that you need to fill in and keeping the responses blank here chooses the default options that are displayed in the brackets and this takes a few minutes here notice that it's creating a change set so it's going to compare what's already out there to what we're trying to deploy and since this is the first time we're deploying this it's going to deploy all of our resources it's going to deploy everything and it's going to take a few minutes but you can see our resources are being deployed we have this um lambda function we have a dynamodb table we have the iam role we have the event source mapping um that that decides that dynamodb is the event source for the lambda function all of these are being created for you with that sam template and like i said before sam is built on top of cloud formation um yeah so okay good once it's done you'll get this success message that your stack has been created and all of the resources have been deployed now we're going to go ahead and we're going to test this we're going to test what we just deployed so we're going to head over to the dynamodb console we're going to choose tables and we're going to select the newly created table that we just deployed we're going to choose the items tab and then choose create item we're going to add an item to our dynamodb table and then we're going to save it now that we've added an item to the to the dynamodb table let's go ahead and um navigate to the lambda console from the lambda console we're going to choose our newly created lambda function and then we're going to scroll down to view logs in cloudwatch so we can see the cloudwatch logs for the lambda function and when we click on this log event we'll be able to see um the logs for what we just did so you can see the new image inserted into the dynamodb table so anytime a new item is added to our dynamodb table this lambda function is invoked and logs the event in amazon cloudwatch logs all right so today we learned about infrastructure as code we learned about templates and how it can make uh your entire um deployment process much easier it creates consistency across your environments which makes testing a lot easier um and we also have a ton of content and learning opportunities including the entire serverless patterns collection on serverlessland.com so be sure to visit if you haven't already um the serverless land patterns collection is a really great way to get started if you have never deployed a serverless application or if you've never used infrastructure as code before if you go to serverlessland.com patterns you'll be able to see sam templates you'll be able to see cdk templates you can just download and use and deploy so we created a bunch of these for you and you can use them in any step of your development process to help you understand um sam and to help you understand infrastructure's code we have a ton of templates on there so be sure to check that out thank you all so much for joining me today in our journey through infrastructure as code my name is talia nasi i hope you all have a wonderful day