Developer Tool Integrations with AI -- The AWS Approach

Key Takeaways

[Music] you're watching the new stack makers a podcast for people who develop deploy and manage at scale software for more information and articles about at scale Technologies please visit the new stack.io now enjoy the show Amazon web services is the world's most comprehensive and broadly adopted Cloud platform offering over 175 fully featured services from data centers globally millions of customers trust AWS to power their infrastructure become more agile and lower costs hey everyone it's great to be here today we're talking about innovation in particular developer Innovation and Amazon web services and I am happy to have Doug seven director of software how are you I'm great your director of software development for Amazon code Whisperer you lead the project that's right very cool and Harry Maurer director AWS devops Services yep that's right over the past several months we've heard so much about generative AI but I'm wondering what you think's really cool about it what is it that you think developers are like intrigued about it what is it that makes it so compelling because it almost seems like a next level Alex what's not cool about it it's generative AI right this is like the coolest thing out there you know from a developer perspective I think this is this is a great evolution of the kind of things we've seen uh in developer tools really over the last you know decade or two in terms of developer tools trying to build capabilities that just make the process of building software easier and the more that they can help with things like oh I type a class name I hit period and then it gives me the methods that are available in the class or the properties are available in class things like that were sort of the simple innovations that happened a long time ago with things like intellisense and now with generative AI the idea that if we were sitting down together to code we're both bringing some knowledge into this situation and I might be doing something and he you know oh don't do that I got a better idea you should do this so the idea that you can apply you know two sets of intelligence to a problem and come up with a solution is really cool and now I can do that with a gender of AI That's always there it's ambient it's ever present and it's just working with me as I were is like super cool is that the conversation you had with developers when you talk about generative AI what do they say to you what is it that you find compelling about it yeah there's the it's going to generate code for me aspect which is really neat because it takes away you know a lot of the mundane stuff that you'd rather not do or you'd rather focus on you know more higher value things but I think in our space particularly when you talk about software development life cycle you know people think about a developer sitting down and writing code and they always just mentally put themselves in the frame oh it's a new project and then I start writing code 90 of the time you're joining a project that's already there and you got to figure out what's going on right and you know you're gonna you're like you have a Wiki or someone who can explain it to you but what I think is really neat about the gender of AI stuff is helping me better understand what I'm already about to work on right so explaining what this last pull request meant or explaining what this code actually does or you know explaining you know why you're seeing this error and it's like having that person right next to you like said you know they would have to go to the team but always having that ability to explain things to you right there is a huge productivity gain in a lot of cases so Doug let's get into Amazon code whisper because I think that's a good segue it's being positioned as an AI coding companion that generates whole line and full line function code recommendations in your IDE to help you get more done what is it that you were thinking about when you're building out code whisper what was it that you thought you saw out there that you think we can differentiate we can we can take this to a Next Level the fundamental problem is that being a software developer today is really difficult Once Upon a Time if you were a c plus developer that's what you were you were a C plus plus developer and that's what you did or you're a web developer and that's what you did but nowadays you're a developer and you work across languages you work across Frameworks one day you might be working in JavaScript the next day you're working in go you're working on different Frameworks and it's hard to keep up it's hard to be good in all of these things it's hard to be up to date in all of these things and so there's a lot of time spent even in something you're familiar with there's a lot of time spent refreshing your memory refreshing yourself going oh yeah I gotta do this I know how to do this but I don't remember let me go look it up that ruins the fundamental thing we want developers to experience which is this natural Flow State of like I've got a problem to solve I'm thinking about that problem I'm writing code and I'm just kind of moving through it fairly fluidly every developer has experienced this this this moment where you're like I'm just going you kind of forget to eat you forget what time it is all sudden it's dark out you don't know what happened but you did a lot you made a lot of progress and so the more that we can help developers cultivate that experience by keeping you where you are keeping you working on the thing you're working on and helping you like bringing that information to you it's like everything you know you you join a project and you have to figure out what's going on there well there's a lot of context in that project and so if I'm going to write some code for that project having that context be present and inform an AI on how it can help you and say okay well if you're trying to do this particular task you're trying to write a particular function let me understand the context of where you're at and let me recommend something for you let me give you a suggestion for the code that we think is going to solve the problem for you and if it were right you hit tab you move on you just saved yourself a whole bunch of time and doing something that would have otherwise been a little bit monotonous and one of the things that we talked about Harry before our discussion is this concept of all of those mechanisms that you really need to have in place so the software can be used in the in the manner that Doug is discussing how does code Catalyst provide that for developers right so take step back so code catalyst is an end-to-end integrated devops tool chain right so we provide developers everything they need to go from planning all the way through to deployment and we're really geared on making it easy for teams to work together to deliver software and we have the saying internally at Amazon that good intentions don't scale meaning if you're relying on someone to check that code for you or if you're relying on the fact that you think the developer was great and there's no bugs that are going out system that's always going to fail and so what you want to do is put these mechanisms in place to enforce what you want to have those good intentions to do and good mechanisms have a way of inspecting and then being able to to stop things going through the system that shouldn't go through the system a way to remediate them and then a way to sort of investigate them and what we wanted to do with code catalyst is bring all those tools together on a common sort of data plane where we could more easily provide those can capabilities to teams so another set of products that my team manages is the code services and those code services are individual sort of point in time solutions for the devops process so code build for building your software code commit for managing your source code so and so forth what they don't have is that connective tissue and that's what we do with code cattless is we provide that connective tissue so that when someone does a commit it'll automatically raise event we can put a rule against that we can stop a bad commit from happening if we see code trying to get into production that hasn't been say a rule applied to it where two people need to approve it before it goes to production we can stop that and then as everything goes through the system we're collecting all that data and then that data can be used to generate a generative AI type models where we can then do predictive things against it I see so that's the intersection isn't it yeah I mean the the you know your develop this is the flow the developer Works through right they have their sort of inner loop where they're writing code testing the code validating their code and then they're bringing that code into their shared environments where they're merging their code with their other developers they're running it in a test environment they're deploying it into production right so that entire workflow that a developer goes through needs help everywhere like they need these tools are great and the more that we can stitch them together the more we can make that process really fluid the better it is for Developers so there's lots of assets that go into that experience that you're talking about part of it is like avoiding the paper cuts is a term I've heard uh discussed this idea of paper cuts means what exactly and you know through for instance using Code Catalyst how you how is it helping you avoid those paper cuts and then how does that then play into what you're talking about with uh generative AI yeah so I can talk a little bit about from a soft you know managing the project aspect and then you know Doug can talk about how it gets into the AI stuff but you know we take a lot of lessons that we learn internally at Amazon and we try to bring them to the tools that we provide to our our customers as well and Amazon's famous for this idea that like if you build it you run it right and so we we expect a lot from our development teams to be able to do a lot of things right they've got to plan their work they've got to execute on the work they've got to operate their services once they're in production and uh so this idea like paper cuts are all the little things that sort of take away from the ability to deliver the value to the customer as opposed to like that sunk cost stuff that you do operating services and the way we overcome that internally is we try to automate everything including like the most mundane things and so like a good good two good examples of of us bringing automation to developers inside of code catalyst is when you start a project and start a catalyst we give you the opportunity to start from what we call blueprint in a blueprint is a way to generate the entire project everything you need from your source code to your tools to the environments into everything that you're going to need to manage that and so with two clicks and you know a couple minutes you've got a fully functioning project that everyone can collaborate on and that would take you normally you know hours if you had to do that manually another simple thing we do is you think about you know we talk about you know developers join projects that don't necessarily always start projects right so we do help you start projects we also help you join them in one of the ways that there's a lot of paper cuts joining a project is just getting your system set up to work on that new project it's like I said like I'm going from like JavaScript on one project Java on another potentially C plus and the other one I've got to make sure I got all the tools and all the setups what we can do inside of code catalyst is automatically spin up a Dev environment for you to connect your IDE to that has all those tools already set up for you so we automate all that those paper cuts away that time you normally have to spend installing tools configuring stuff all that goes away with what click of the button uh whatever branch of the code source code you're working on WE automatically spin up the dev environment for you check out out that code and everything's there for you to start working right again as we talk about like we're collecting all that data too so we better understand how the customers are working with that and can feed back into these models so you talked about the blueprints that speaks to me about hosted Dev environments which you really need and that collaboration and that automation speaks to the devops practices doesn't it well yeah and I think what's really interesting about this is you you start to put all these pieces together and it formulates the whole picture right you get it you get a Dev environment created as part of this blueprint process and you get up and running and now now I've got my IDE connected to my Dev environment and now I've got a generative Ai and code whisper as part of that IDE so not only do they get accelerated in getting going but now as I'm starting to write my code I get that acceleration through the generative AI to be like okay now I've got to do some work I've gotta I've got a problem I've got to solve I've got to create software to build this this thing and I start by you know writing a comment with the intent oh I need a function that does this and then code whisper suggest that function I look and I'm like yep that's right tab move on to the next thing you know and so all of this is just this continuous acceleration of how do we how do we apply our Technologies either by stitching them together or bringing new technologies like AI into this space to help developers just move faster and really part of this too is like so much of the development is repeating things you've done before like you know how how many times are you writing for loops and you know doing some logic reduce evaluation or building a class to represent a data object or something like this and a lot of this is relatively boilerplate stuff it's not the creative problem solving that you're doing that's certainly a big part of software development is the creative problem solving and that's where I'd rather spend my time so if I can say you know put a comment that says I need a I need a class to represent a student for a educational application and all sudden a student class is generated for me I'm like cool move on let me go solve the cool problem now let me go do that I want to get to the joy of work but first I want to talk a little bit about the people who are using this right are they beginners are they more advanced people are they both what team should they on I mean we hear a lot about security now and there's really when it comes down to it it's about the team and how the team can function with the platform architect and the developer and the security people how are you thinking about all these different aspects of that individual users and the teams that they're a part of I mean at the end of the day most of these tools are you would think of these as Pro Dev tools right we're we're a development organization we're responsible for building a line of business hap or a mission critical app or whatever it might be and so I'm coming at this with a knowledge base and some some skill and some tenure in my background uh you know when you think about how ingenerative AI Works we're we're predicting what we think you need so as you're writing code we're looking at the context of what what language are you working in what methods have you already written what classes do you have and then as you're writing your your code or you're writing your intent in the form of comments we're predicting what we think you might need and we're making that a that suggestion now you as the developer have to look at that and determine if it's what in fact you want like you know how good how good are we at predicting that and looking at that and saying yeah that's what I want around that's about 90 of what I want so I'm gonna go ahead and take that and I'll make a couple edits and I can I can move on or in some cases no that's not what I want maybe I didn't express my my self in the comment very well or maybe I'm just doing something different and I'm going to move on past that and so there is a level of like you as a developer still have a responsibility as you're using these tools to use them and get what you need out of them but these are accelerators right these are just moving you along and I think in code Catalyst we're a little our aperture is a little wider but we do interesting thing we want to be uh we want to give you the flexibility we also want to be prescriptive and what I mean by that is um a lot of customers who come to AWS can sometimes be overwhelmed by the number of choices we have right there's a lot of services to use right and we also have a lot of services that sort of you square might be very similar right so like which container platform should I use or you know which database service should I use and we provide a lot of information to developers on the right sort of constellation of services used for a particular type of application but what code Catalyst helps you do is sort of take that next step so a lot of times we provided that information in the past we would do in the form of like a cloud formation template or a cloud or stack right within code Catalyst we take that that us that that cloud formation template and turn it into a blueprint and so we automatically will take those best practices wrap it up with all the tools all the source code that you need and automatically build and deploy that application for you so at the end of the the process of using that blueprint you've got a well architected and designed application based on how we think you should do it for you to start working on right and that's if you choose to use the blueprint if you don't use a blueprint then you can start from scratch and you can be a protab and do what you want to do so we tried it we try to meet a wider spectrum of developers I think with code Catalyst well I'd like to conclude with this idea of the developer experience and one of the themes that comes through when I'm reading about what you're doing and in our discussion is this idea of intent right you have a specific intent and it and you talk about that flow right but I think there's also aspects of the job that may seem tedious but they're just part of the job and they're part of your routine and they're part of just what you do and like it's not that you hate that but it is something that you just have to do how do you fit that all into the culture of what you're trying to build I think to some degree there's the question of there's the things that have to be done and should you be the one that does them is one question so in other words um hey you know there's this boilerplate code it has to be written somebody has to do it well if an AI can do it and you can move on great that's it still has to be done but it's not you having to sit there for 20 minutes riding out some code it's generated for you in a couple seconds and you can move on there's the aspect too of like what we haven't even talked about is you know we're generating code we're getting some suggestions developers are contributing to that so you have this hybrid of code that's been generated by an AI that a human is written and then there's a question of okay well you know have we have we unintentionally in uh pulled into that code anything unwanted did we pull in an open source reference that we didn't want or didn't know that we were probably like this is one of the problems that developers have today right you go off and you copy some code from somewhere and you paste it and it turns out you violated an open source license um and so one aspect of what we do is ensure that when we're generating code if we do have a reference to an open source license we tell you about that we say hey this code has a reference you can go check the reference here's the link to it you can see what you know if you need to attribute it or not that's one aspect of it and the other aspect is the the security of it are we doing anything in the combination of generated code and human written code that would unintentionally create a security vulnerability do we do we hard code an access token did we accidentally code up something that had some SQL injection we do a lot of work on the filtering of the AI generated code to make sure that the code that's coming to the developer is as good as it can be but the hybrid of a of a person working in an AI generating code you really don't know what could happen and so the idea that we've actually introduced some of the things we have on the on the server side that you would be as part of this devops workflow that Harry's talking about where we have tools for doing security scanning you know once you once you commit your code we can do some security scans and incorporate that as part of the process well we move that to the left a little bit bring that into the IDE give the developer the opportunity to say hey I've put some code in here and they asked put some code in here I'm gonna go ahead and run a security scan on that and look for any of these known vulnerabilities so that before I commit my code I can find those things clean them up and get that code in and so that's all part of this process and it kind of partially lands in this bucket of like hey these are things that somebody has to do right can it be an AI can it be a tool if so great let's let's enable that and let's again free the developer up to apply their cognitive load to the creative problem solving uh the unique and new stuff not the not the mundane right there must be some aspect of I think two of like Integrations and you know managing apis I mean what you're talking about is just like this whole flurry of of things you got to be thinking about all the time so it's we talked about you know getting to that flow State and I I had this conversation with other people in the past it's like I think anyone who's been in uh you know software engineering for a while you sort of go through these different stages of uh what gets you excited you know the first time you actually got something to run and the next time is the first time you realize like I've been sitting here for 12 hours and I haven't eaten you know that whole Flow State and I think the next one beyond that is when people start actually using what you build and you know what we're focused on is you've got an application here and it's great that it's working for you but how do we get that into the world where other people are using it how do you get to collaborate with others to build that because it's one thing for you know one two three people to be building a software product together it's a different different when you've got a team of 10 or teams of 10 trying to build something larger and more complicated and um we we want there's joy in that as well right there's joy in seeing all that stuff moves through these pipelines and just watching it flow very easily and that's what we're trying to make we're trying to put that into the product that that joy of seeing it being used and deployed and running for you know hopefully millions of people for to consume so um and there's all different parts of what we're trying to do there whether it's making the CI CD processor easy which can sometimes be a black box for a lot of people making the dev experience easy to spin up the dev environments for us it's like the whole framework around building software as a team well Harry and Doug thank you so much for your time I've really enjoyed it thank you Alex it's been great yeah thank you if you like this video please give us a thumbs up and if you'd like to see more videos like this you can always subscribe to our YouTube channel we're on all the major social media platforms you can always find us at the newestack.io we hope to see you soon [Music]