Migrate mainframe applications to AWS using automated code refactoring - AWS Virtual Workshop

Key Takeaways

[Music] hello everyone welcome to the virtual workshops i am jayanti rajaraman i am a customer delivery architect at aws my specialization is around mainframe migrations and modernizations in this episode we are going to see how to seamlessly convert the cobol applications to java springboot and run it on aws i will guide you through a demonstration of automated conversion using one of our partner tools called blue age so before i get into the demonstration i just want to provide a brief overview of what automated code refactoring is so as you all are probably familiar aware of there are many different patterns to modernize and migrate workloads from mainframe to the cloud and each pattern offers its own unique benefits and each pattern is chosen based on the workload the desired end star end state and the migration time that you have so these are all things to consider when determining the best pattern to use in today's session we are focusing on the legacy automated refactoring pattern so what is this pattern in this pattern the code the data and any associated dependencies are automatically converted to a modern language a data store and a modern framework as well with this pattern your guaranteed functional equivalence uh as compared to what you have on mainframe today so in here what you see is that anything in blue so the mainframe to the new platform anything that you see here in blue stays the same so what you see here is that the business functions that exist on mainframe stays exactly the same what has changed is everything that you see here in yellow so your entire software stack has changed to a modern technology stack so whether it's your operating systems your databases and files any application middlewares the code as well as utilities that may exist on mainframe and your actual data itself so all of this has been transformed into a modern technology with this particular pattern so what really is involved in a mainframe automated code refactoring project so this project involves the conversion of code dependencies and data as i previously stated so for example you can see here on mainframe you may have your code in cobol pl1 assembler jcl and so on so this code can be automatically converted to a java or a.net for example and so the automatic conversion automatically generates the modern code as well as any associated data access layers any data format changes and dependencies all together with minimal manual intervention so the way it's done is it uses certain patterns and rules to transform the mainframe screens and the indexed files as well as any batch stacks to a mod modern target stack so as i stated the this this kind of transformation gives you guaranteed functional equivalence and the result is an object oriented and a service oriented modern application now given that this process is mostly automated it is also very quick and it greatly reduces the risk from a mainframe migration standpoint so how does this process work this process has two steps so the first step involved is the reverse engineering so what happens in the reverse engineering is that the code and any other associated elements that are required from mainframe is ingested into the analyzer tool and this tool analyzes the application modules and it will generate several outputs um the outputs basically will consist of an application model which shows your program and the data dependencies as well so what this output helps with is to detail is to come up with a detailed strategy with the different components data mapping conversion rules and ways to decompose your work packages as well so once you've reviewed this output and decided how you want to break your monolith into macro services and decompose them the analyzer tool generates an intermediary module which can be proprietary to the tool that's being used and that becomes the input into the next part of the process so the next part of the process uses the output from the analyzer and automatically generates code in a technology of your choice so you could for example take a cobol code and automatically convert it to a java spring boot application as i'm about to show you and deploy it on amazon ec2 for compute and the amazon error database for the data storage so this is a completely uh from an architecture standpoint it's an elastic web architecture and this is one of the typical deployments however i do want to point out that the target technology stack can be customizable so for example you can use amazon sqs for messaging you can use amazon elastic cache for in-memory shared data structures you can use ecs or eks for containers or you can even deploy using aws serverless lambda functions another thing also is that this generates a service enabled stack that allows you to expose your services via the api gateway so as you can see here the automated code refactoring is a fast and coherent transformation of your complete application that currently exists on mainframe so before i actually start the lab and i'll show you how this conversion happens i want to review the process so you can follow along as i get into the lab so the first thing that i will be sharing with you and showing is the mainframe application that's going to be converted so i'll actually pull up a mainframe terminal and show you the cscs application as it exists on mainframe so that when you review it when while it's executing on aws you can see that it's exactly functionally equivalent and then the next step of that would be to take the inventory from mainframe and import it into the analyzer tool and here i will show you the output some of the outputs from the analyzer tool and after we have done the analysis part we will get the intermediary modules which is proprietary to this blue age tool which will be imported into the third step which is called a blue age velocity tool the blue age velocity tool uses the intermediary module to actually transform the code and generate a java springboard application and as part of this we are also going to convert the data from vsam into a aurora postgres database so we will i will show you how we are going to take the data and uh migrate the data into the aurora postgres database as well and all of this application will be deployed on a tomcat server on ec2 and finally i will show you a quick test of the application when it's executing on aws so as you can see the source environment i'm using for this particular demo is cobol we have some jcls that are used to load the vsam files currently on mainframe so the cobol becomes java springboard applications the jcl will be converted to groovy the bms maps are converted to angularjs and the vsam files are converted to aurora postgres sql database and the application itself we call it a murak application it consists of four cics screens uh many transactions there's a menu inquiry maintenance and order entry uh and then as far as the components of this application goes we have bms maps we have the cobalt programs associated with these transactions we have batch programs that are currently used in mainframe to load the vsam files and then we have the vsam datasets themselves which are part of this application so this consists the inventory that's going to be part of this demo all right so having said that i'm now going to move into the demo part of it itself so the first part of the demo is showing you the application on mainframe so i'm going to pull up the mainframe application in just a minute okay so this is the mainframe application so i'm just going to log on to the mainframe cscs region so you can see the application so this application consists of a menu transaction from where you can view the customer information you can maintain customer details as well as do order entry so for example i'm able to pull up the customer information here know if i choose to i can add a new customer or i can even change the details of this particular customer that i just pulled up so so this is the mainframe application that i'm going to be using uh in my demo so now i'm going to shut down this connection and show you the actual conversion itself so i'm going to get let me disconnect my mainframe session here all right so as part of this demo what i have done in preparation for the demo is i went ahead and set up um i went ahead and set up my uh an ec2 instance and in that ec2 instance i have already installed the tools required for the demo which would be the blue age analyzer the blue edge velocity tool and i've also installed my tomcat server in this thing in in my ec2 instance so what you see here is my ec2 instance where i've installed the necessary tools so the first thing i would like to show you here is that i have as far as a preparation i also went ahead and downloaded the modules from the application that i just shared with you so you can see here that i have my maps my cics definitions cobalt programs copy books jcls and as well as the vsam definitions for the files that are required so these are all the elements that i'm going to import into the analyzer tool and i got all of these from the mainframe so the first step is the analyzer part so i'm pulling my blue age analyzer tool and it might take just a small second for this to come up so this is an eclipse-based tool so if you're familiar with eclipse-based tool it's very easy to use so this is my analyzer so the i'm going to go ahead and import my assets which is all the stuff that i just showed you that i have downloaded from mainframe and i have it over here so i can call this project anything i'm just calling this murak reverse because my application is a mirac application so it helps me remember that and what i'm going to do here is i'm just going to point to the folder where all the all the necessary elements exist and import it into the tool so that's the murak code so i'm going to import all of those elements into the analyzer tool okay and so what this tool does is the results of the analysis is stored in a internal graph database so what i'm doing here is i'm just creating a graph database where it can store the analysis results [Music] so once i have given it a database name i'm just going to go ahead and let it finish so you can see that see that here it's analyzing the source code and importing all of the elements we'll just give it a few seconds to finish so you can see that it has completely imported all the elements so if i click on the assets here you can see it has in imported the code that i just showed you the bms copy books jcls and so on so this is the code that it has analyzed so one of the things that's critical to the analysis process in a mainframe modernization and migration is understanding the existing dependencies between the programs and the data so that you can decide how to decompose the modules so the tool itself automatically comes with certain graphs built into it which gives you a very good visual of the dependencies so um right here all i'm doing is i just want to make sure i uh the graphs display properly so i'm just kind of doing some um you know preferences to make sure that the displays correctly on my screen and then what i'm going to do is let me for example pull you the inter module graph so i'm going to use a view that i have found to be most useful which is called the smart smart organic view so you can see here this clearly gives you a visual of all the dependencies that exist between the programs and the tran ids so this helps you decompose the module so in my case this is a small application for the purpose of the demo so i'm going to essentially convert all of this and deploy it together but if you were doing this for your real mainframe applications it's probably going to be bigger applications and this gives you a visual to decide how you want to decompose it and then based on the dependencies that you that exist you will have to decide on the integrations and how you're going to keep the data in sync and so on so this visual is really helpful from that aspect another thing you can see here is that it has highlighted any missing elements so in any real you know mainframe effort there is no way you at the very first attempt you know every element that you need for your application so you don't have to worry about it the tool is going to tell you and guide you along on any missing elements so here you see that it is missing an element and um i will go ahead and import it at some point so this is one of the dependencies uh that it displays uh it can also display um you know any data data dependencies as well so again these are all graphs that it automatically generates that gives you a great visual into this so from a data dependency standpoint again you can see that all these red elements here are the data dependencies so you you can see that for example the customer master file all these other programs are dependent on it you can see that the product file is only the order entry system is dependent on it so again this visual is very critical to helping you decompose your workloads when you decide how you're going to migrate migrate them so the other um great thing is this is a visual intuit but there's also many gremlin queries that you can use if you if you rather want to use queries than visuals to understand what's missing and so on so the tool itself comes with several queries built into it so you can just run those queries as well if you choose to so i'll run the query once it loads so what i'm going to do in the meanwhile is obviously there is a missing element that is required before i can proceed further so i'm going to go ahead and import the missing element again you don't need to import everything you can do incremental imports as the tool as you progress along and find other elements that needs to come in so all i'm doing is say incremental imports so at this point you would have gone ahead and found out what the element is and got it from mainframe and saved it into your workspace as well so that's exactly what i'm doing here so i have a folder called missing where i have my saved element so i'm just telling the tool to go ahead and import this thing so again it's going to go ahead and analyze it and include it so now if you see here um if i go back to this intermodule graph that i was sharing earlier you can see this is no longer highlighted in red right so it means that it has successfully imported that module a couple of other things that this really useful information um you know like you can see it has gremlin queries that you can run as well but another important thing i wanted to show you is metrics so you can go ahead and compute the metrics for the applications that you've imported and this is going to provide you useful information again which helps you decide you know how complex is this migration going to be and how are you going to approach it so you can see that for each of the modules it shows you the lines of code it can tell you how many lines of code is commented and not commented uh it gives you complexity view you know the cyclo cyclosomatic complexity essential complexity and so on so based on this you'll know how complex your workload is how big your workload is and so on as well so these are a couple of uh useful metrics and tools that is generated as part of the analysis so in the real world once you have all of this generated and you've decided which parts of the component you want to modernize you can tag them again like i said for the purpose of this demo i'm just going to use all of the elements but if you wanted to only pick certain elements you will be able to just go into the intergraph module which i shared earlier and simply select the elements that you want and tag it and only export them and modernize those workloads so now that we have done this one of the other things this module does is that intermediary module that is generated right so the analyzer has already generated that information so i'm going to go ahead and export it so i can use it in the next part of my project so to do that all i have to do is export assets and i'm going to ask it to export the dsls that have been generated you can save all this on git repositories if you want to but in my case i'm just saving it on my local drive and you can see that it is saving all of this into this folder that's highlighted over here so i'm going to go ahead and ask it to continue and it has successfully exported all the modules so this would be the end of the analyzer part so i'm going to go ahead and close the analyzer so what i want to show you before i get to the next part of it is you can see here that in the analyzer part these are this is just um this is the intermediary module that i just showed you so it has gone ahead and created all the cobol jcl programs and so on in a format that's proprietary to this tool um so what i'm going to do the next step of this process is the actual conversion of the code so so far what we've done is imported the modules and then analyzed them decided how we are going to decompose it and gone ahead and generated that intermediary module so in this step is where you're actually going to convert the code into a java spring boot application and for this i'm using a tool called blue age velocity tool um now this tool is capable of doing other than java spring boot applications it's not just restricted to that but in this case that's what the target stack is going to be so uh in this tool what i'll be doing is i'll be importing that intermediary module that we just generated so i'll just give this a second to launch so while i'm waiting for it to launch i also want to go ahead and show you that one of the other things that i did get from mainframe from a data standpoint is that i have also gone ahead and downloaded all the data required for the four vsam files that we will be using so i'll be using this data and uh for the data migration part of it all right so this tool again is an eclipse-based tool so the same thing um you know it's very if you're used to eclipse it's a very easy to use interface okay so there are a couple of things that need to happen in this tool first is actually importing the intermediary modules the other part of it that we need to do in this tool is also set up the configuration and the rules on what the target stack is so it can convert the code accordingly so first i'm going to just go ahead and import the intermediary modules that i just generated from my analysis so this is what i'm doing i'm going to ask it to import the two modules that i just showed you so those modules exist in my local library so i'm going to go in here so the output from the analyzer is in this folder all i have to do is just select it copy the project into the workflow space and finish so at this point it has imported all the net required information in terms of your source code and jcls and so on so you can see that it has all that information in here so next what i need to do is i need to set up the blue age velocity project so that it's configured uh to convert my code into this java springboard application so in order to do that i'm going to create a project i'm going to call it murak forward again my application is called mirag so i'm just calling it the forward part of it and i all i'm doing here is setting some parameters that's going to help this thing the tool know how what code how to generate the source code um the target code i mean so i've i i'm going to since this is a cscs application i'm going to use enable jigs which is the you know the blue age equivalent of your cics and then um since it's mainframe application i want to make sure i set up my encoding in this case the ebcdic encoding and then i also am going to set up here to use um because i'm converting to a springboard application i just want to make sure i give it a spring profile which i'm calling murak and in this case i'm going to be deploying my application and running it in my in as a local host but you would configure this in the real world to be whatever um you know in domain name that you want it to be and so on so in this case i'm just going to leave it as localhost and at the end of it i just say finish and at this point the forward project is ready so now the next step is to actually ask this tool to go ahead and do the code conversion so to do that i just go in here and i say i want to modernize from my analyzer dsls that i've just imported and when i do that it's going to ask me for some license information and then i'm just asking you to use the configurations and rules that i set up in this murak forward project and i'm also giving it the name of the root package which is again uh this is all related to the tool that i'm using so once i've done that i say okay and you will see that it is started converting my code here this shouldn't take too long it should be done here in just a bit so while that's happening okay so what i want to show you here is there you go the code it has been converted you can see that it's been successfully generated so what you see here is that you've seen several folders have been created so it has created a pom which is basically going to be the packaging part of it but what you see here is that it has created this folder called service so the service uh the this this contains the service package contains the service layer for the spring boot java back end application then you see that there is a web folder the web folder is going to actually have your front end application this is your angularjs application as you can see here and then the tools there is a folder here that's also called tools so there's a service web and then there's a tools folder so the tools folder contains all the utilities for the tasks such as the file access and so on and then of course you have a pom folder the palm folders contains all the information required to link uh and generate a deployable war file later on so now that this has happened the next thing i want to show you is so in a cics application you have a lot of definitions that happens on the mainframe like the ppt the pct and the fct and so on so what we are going to do now is that we are going to create a schema for the gixx which is a cscs equivalent we are going to create a jig schema and the jig schema is going to be loaded in the aurora database so one of the things that i have gone ahead and done is created is gone ahead and created a an arora database here that we can use for our project so you can see that it's a murak database is the name of the database and so i'm going to be using this database and this writer instance to configure my tomcat application as well as to load the cics definitions and later on i'll also be creating different schemas for the vsam files as well and in the case of the demo i'm loading all of this into the same database but in the real world you can have your kicks definitions and your actual data files in separate databases as well so this is the database that i have here and what um so i'm going to access this database using pg admin tool so back here i have connected to my murac database so just logging in so i can connect successfully there you go so that's my murak database that i'm going to be using so you can see here right now if i go in here and if i go into the tables i don't have any of the definitions and so on over here so what i'm going to be doing is i'll start with this clean up clean project so let me delete all of the objects i have here so once uh the way to load these tables is we have also gone ahead and from the mainframe uh we have created a sequel for each of the tables that's required so if you see here for the fct ppt pct there is a corresponding sql that i've done that we have created that's going to create the corresponding tables so all i'm going to do is execute this sql so we can create the corresponding schema files in the pg admin tool and so this is all again going to reside in the murak database so i go to my mirac database i'm going to go to my query tool and execute this query again you can see it has successfully created the schemas so again all i have done at this point is only create the schemas right so i have a schema corresponding to an fct a ppt a pct and so on but i have not actually added the definitions for my murak applications vsam files or programs or transactions into this table yet so what i need to do for that is execute a query so again going back to um one of the other things that i got from mainframe from my csd list from mainframe uh is all the information that i need for these definitions as well so i'm going to execute that query here in the pg admin tool and so this query what it's going to do is again it's going to in the murak database that it's going to insert the definition data into the schema tables that we just created so let me go ahead and execute that so here you can see that it has successfully inserted the data for the definitions into the table for the murak application in particular so that's um we have taken care of the cscs infrastructure and making sure we have copied all the definitions into the jigs application so now um going back to the blue age velocity tool what we are going to do is we are going to create the deployables that we need so we're going to create the war files that we need to deploy into the tomcat application so let me just quickly go in here and make sure i have my jdk so um at this point all i'm trying to do is i'm just going to use maven and go ahead and create the war files so now that i've set up my jdk over here i just come in here to my palm i'm going to go ahead and ask it to do a maven install so again it's going to be doing generating my war files for me so while it's doing this let me go back here so so far we have not done any data migration activities so in order to do the data migration activities i am going to use another tool uh that is called um that's another blue age tool so i'm getting into the blue age administration control so here i have my tomcat uh application manager i have this blue age tool installed here already i have not yet deployed my applications but i'm going to go to this tool so this is a tool that's going to help me actually migrate my data so logging into this tool all right all right so to deploy the data is fairly easy so i'm just going to use the list cat definitions from mainframe to actually load this data so i have all of the list cat that was generated and i'm going to ask you to use that so you can see that there's these four vsam files that we have so i'm creating the schema that's corresponding to each of these vsam files right now so you can see that these are the four files it has a record length and these are fixed record files so at this point we've created the corresponding schema i have not loaded the data so in order to load the data again it's this tool is fairly simple to use you just click roll data [Music] and um you you just simply point it to the file where you have the data so again one of the things i showed you initially was that we already i already had the vsum data downloaded from mainframe so i'm just going to use the data here and load it onto my blue sam server so you can see that it has loaded 20 records i'm going to quickly repeat this procedure for the other three files as well so the next file uh is the invoice control so same thing just takes a few seconds and i'm of course this is small is a fairly small file at this point but so you can see that it's the process to do it is pretty simple and then the last file here is the product file so the product file is 16 and length and i just point it to the product file here click on open door run server load on blue sam alright so now you can see that all of these files and the data records have been loaded in to my blue sam so the data migration has been successfully done so if i go back to my blue age here you can see that it has successfully generated my files for deployment as well and i'll quickly show you these files here so basically what we've done is we've generated two files i'm done with my code conversion and creating the deployment files so the files i've generated are two war files one is for the web front end and one is for the backend so these files um are here in the service so this service one would be the backend file so if i get in here i can see that it has generated a forward service war and similarly for the web it has generated a forward web war so now that the files have been generated i'm going to go ahead and deploy these files into my tomcat server here is my tomcat application so i'm going to just log in here and deploy my files that i just created so let me make sure i get my file names so i can deploy it so here is my file for the forward web and then the war file for my uh for my back end is located in the service target so again i'm just going to copy this so i have this all right so [Music] i'm going to go back into my tomcat server here and finish this deployment first i'll just deploy the forward one okay this is my forward web let me make sure the deployment finishes just give it a second okay so the forward has been deployed now i just need to repeat this for the back end war as well so it would so i'm going to take this war file [Music] right so you can deploy this you know from your eclipse as well i just wanted to show you the deployment here but so this is the back end deployment that i'm doing so i've deployed it deployed the um service so again let's make sure it deploys just give it a second so just to recap what we have done so far is that i showed you how we took the uh application from mainframe and imported into the velo into the analyzer tool we saw the outputs from the analyzer tool then we took the output from the analyzer you imported it into the velocity tool and then from there we had it convert the code to java springboard application and we created the war files and we've deployed it into the tomcat server we've also taken all the definitions from the cics on mainframe and converted it into schemas and added those definition into the postgres database we also took the four vsam files use used the blue sam tool and loaded all of that generated the schema for those vsam files and then loaded the data for those vsam files as well so now we have completed all the steps in this process one other quick thing i want to point out is that i've also configured the um the tomcat server itself so that it's pointing to my aurora database i could probably just quickly show you that um so i just basically i'm pointing here you can see that i've updated the resource type here to point to my database endpoint so the aurora server is configured to connect to the database and then let me go back here so now that i have um you can see that i have my forward and the forward service deployed it's both running and the other thing i wanted to also point out is there is a couple of other applications that you see here so you have a gap walk application so this is a blue edge application that kind of connects the front-end web server and the backend service so that has also been already deployed so now i think we are ready to test the application so to do that my application in this case i'm just running it as a local host so you can see that i'm in the jigs application so this is again the equivalent of the cics on mainframe uh and i can enter my tran id just like i would do in cics and you see the application displayed here it looks just like it did on mainframe when we started off and we looked at the application on mainframe it looks exactly the same i mean there are a few customization options here in terms of how the screen would look in terms of the background and color but other than that the application looks exactly like it is on mainframe so you can um i can just quickly test it as well all right so i can display the customer information and this is exactly what i was doing on mainframe as well you can see that it pulls up the customer information i can go ahead and maintain an info customer or update it so if i want to just update the information i can go ahead and change this customer um and so you can see the messages everything works just like it did on mainframe with the same trend ids and the same interface and the same look so there you have my the cscs application has been deployed on mainframe and you can see that it works just like it did on mainframe so uh so with this what you've seen here is that it's completely possible for you to refactor a mainframe application and deploy it to run on um aws using aws services uh including um in this case it was an easy to tomcat server but you can certainly deploy it um on other tech stacks as well um so with that i think i'm towards the end of my demo all right so what i want to leave you with is that there has never been a better better time than now to modernize your main frames to run on cloud and there are plenty of resources even on our aws website so um you know there is case studies and blogs that can help you guide along and our team is here to help you guide along in this journey as well so looking forward to helping anyone who is interested in migrating their mainframe workloads to cloud please do reach out to us so that we can help you assist in this journey thank you