Procedural Simulation, AI creativity and stochastic modeling. Using Azure Cosmos DB to make games.

Key Takeaways

hi I'm William Harding thank you very much for having me I'm going to be presenting on procedural simulation AI creativity and stochastic modeling and I'm going to talk about how to use Cosmos DB open Ai and other tools to build a interesting and Rich uh digital experiences and some Frameworks on how to do that my name is William Harding and I work for avenot avenot is owned by Accenture and Microsoft we're one of the world's best providers of uh Azure implementations worldwide I work within data and AI which is a global resource pool and I help clients all over the world build projects similar to some of the infrastructure that I'm going to show you here this is in the process of a game but you're going to see a lot of uh industrial application for it as well in the in the workplace this is going to be in two parts so on one side I'm going to be talking about a Ai and how AI can be used to make content and how AI can be used asynchronously in web experiences and that's important for a lot of different kinds of use cases but then I'm also going to be talking about Cosmos DB of course and I'm going to be talking about how uh Cosmos DB uh used in web applications can create realtime simulated experiences uh using the graph API and you can get uh a lot of really good sharp web experiences by uh by by doing that and I'm going to show you some of the infrastructure and kind of the best practices as far as how to do that when you build really complicated scenarios and web experiences uh for your clients so I'm doing in the context of this project which is the game this is a uh procedurally generated uh AI generated tool that uh takes a small amount of input and it creates a universe the universe has planets the planets have people the uh all of that data is stored in a cosmos DB graph the the gremlin graph I'm going to go into some details about that and I'm going to Pivot over here to some of the actual experience you can see uh this is using just open source web tools I'm using babylonjs for the web experience uh each individual planet has its own uh parameters that are associated with it each one has its own series of moons that orbit it they all have their own parameters this is all created stochastically and I'm going to go into that in detail in a second we're uh each individual planet has a procedurally generated name and different attributes about it the planets that you will have any amount of people on it you can actually go to that and also this is AI generated Landscapes uh build these tools on the Fly and each individual population is has its own attributes and components to it uh and you can see these are all Ajax requests that are going to Cosmos DB to get and fetch attributes from the populations on this planet and as you can see as I'm interacting with it uh Cosmos DB is able to fetch all of this data in real time that's not all loaded in one experience that would be too heavy on the system each one is a very small Ajax request and because you can see in the top I already have the root node of My Graph here I'm able to quickly Traverse and get the data that's local to this information in the graph in the in the time that I need it and uh I'm able ble to quickly go from different orientations by getting very fast really clean uh Cosmos DP requests that have high read highr ability uh and the application works really well also scales really well uh and in that note while I'm talking about a game because that's more fun to do in your free time this also has industrial applications you can see that things like the metaverse and digital twins will also use the same framework in order to build their environment the uh uh you're going to use if you have that similar to that planetary hierarchy and population hierarchy you're going to build an equipment hierarchy from your iot data you put some of your Telemetry from adx you put that into a model and you can make a simulation at in your own Workshop from the same tools this is a very normal process and how we would do it at uh at our work so take a second like what is stochastic modeling exactly uh you may already know this because it's very academic stuff but it's just a it has a an amount of Randomness that goes into the model which means each time you run it it will be different you can see here I have an example where uh a flip of the coin determines whether it's going to be heads or tails and if you run it over a large number of times you have a you should get about 50% for uh at the end of that model this is the obviously not the best use case but if you had an industrial application for example uh is this machine likely to fail or how many people will will uh visit this restaurant in a day and will it ever Peak to a certain amount or what kind of problems will you have you can look for bottlenecks with stochastic modeling uh there are all kinds of industrial applications that are used for it for both industrial Finance all kinds of things have can use stochastic modeling and uh as we're going to get into Cosmos DB is a great tool to run and store that data so that you can build models really fast which is an important component of building your model and the same thing if I take that same logic where I'm using a flip of the coin or some Randomness and I make a much more complicated system at it then you're starting to have what is a game and ultimately most games that are developed today are uh procedurally generated models or stochastic models that operate on these large Data Systems and you can interact with them like this because I'm using Azure I can have my application be a game that you can play online on a website and it can interact with Cosmos DB as its data layer and you can uh you can interact with this graph in the real world and thus have an interactive simulated experience that uh users can interact with so that's a cosmos DB component and we talked about that part how to make a website how to make Cosmos DB a part that we're going to talk about Ai and how that integrates as well we're going to talk about some patterns for that you can see for example this is the Json that you get back from your Cosmos DB experience this is a population that lives on this planet we were just looking at them uh and obviously it's not very fun for an end user to look at it in this way uh and building procedural logic which is normally what you do in these games is you have to write some if then statements that's going to build a full sentence out of that that's kind of the status quo thing that you would do but with AI you can make a much more Rich experience very quickly by presenting uh uh that small amount of information and just asking AI to create a narrative for it uh in this case you could see these people who have the the attributes it's able to both create an image for these people which gives you the idea that uh you know what they might look like and things like that but also some description of how they interact so you instead of reading a Json outre uh readout you can read something more descriptive about those people makes your game a little bit more fun and real seeming for your end user uh in the same way you can use that to uh the kind of the classic pitch of the industrial metaverse is you could look at a 3D model of something and say when was the last time this part was serviced and this that kind of components could Traverse a graph get that information render it uh in this kind of way to to a user experience this goes back in the game uh especially the amount it would cost to build all of these assets especially if you want to have thousands of individual randomly generated uh experiences you obviously couldn't even if you wanted to you couldn't pay a team to develop resources that fast to support your game and I can do it much more streamlined so this infrastructure uh that I have in the repository is going to be very similar to what we would do in a production environment with the exceptions that I'm using the development environment of a local host to run the run the experience but you can see here and I'm going to go into these in detail uh so you uh can get a closer look but you can see a a kind of a holistic model of how this infrastructure we run you you've got on one side you've got a series of uh microservices the Azure function and the event Hub that are really just getting and setting little bits of Json to the cosmos DB graph and that as it's doing it is going to be uh continuously placing and setting and getting data that goes into the graph Additionally the user has a layer which they can interact with the graph and they can make decisions that would affect the the simulation and by that also affect logic that would go into it this would be tedious to do in real time especially if AI is in there because there's a few seconds delay in having AI generate content but I'm going to show you some ways in which you can moderate that to make it um more of a streamlined experience for your game so uh I've built uh custom connectors that connect the uh Azure cosos DB Gremlin graph API that load that data into a set and therefore can quickly within my API uh some convenience features that allow my application to quickly like take an object ID fetch It and Fetch all of the elements that are children of that so I can quickly render a scene fast uh that helps the web experience uh and Abstract that so then I can call it really fast and that's in the library that's attached I've also built a similar wrapper for uh very compon rag kind of tools of just being able to uh uh take a bit of Json and say hey can you explain this to the end user or can you can you build a picture or something out of what's in this uh set build some Center rules you can test all of this in a development environment in my process I build all of this out in jupyter notebooks so I can test the data science of it and then I can take the logic of that and put it into the production application because they're all inheriting from the same components in that uh the best way to manage this experience especially if you have a user involved a user cannot wait a long time for these processes to complete uh and shouldn't have to especially with Cosmos DB you should be able to quickly get and set information in the graph in this your in this pregame where the user is interacting with the website they're just making a call to the website they get some data back the JavaScript is going to render it in a cosmetic way the the same experience where if there's something that has to take a couple of seconds or takes time you don't burden the the user to do that you just pass it to an event route and then an asynchronous function will build that and then put it back in the graph so as you can see what you're going to be doing is that event Hub is going to trigger an Azure function that Azure function is then going to interact with open AI or do some more complicated logic about some event that's happening in the universe and then when it's done with that it will spends its response DNA and then place it there and not burden the the user about that at all when the user goes back to their web experience they're going to see that that has been updated and that it's now uh a diff the the the scene has changed which means populations have reproduced or they've starved to death or they have uh um used some resources or they've built new buildings or things like that that's all happening in for the user's experience in relatively real time and then open AI can finally Place uh individual components that are going to be landed in a blob store that can be loaded as data Assets in your game those are loaded by reference to the web experience so Cosmos DB can get a an ID and it knows with that ID where to look it up in the blob store so it can statically go and fetch assets if they exist and then render them in real time that that creates a very slick and very clean user experience to uh an audience and the same if you were using this for a digital twin or some kind of digital twin like an application you could see how if you had this event-based infrastructure you could have that running all the time now the stochastic modeling part if we want to have a genuine model where something is always turning in the background for example machine parts or whatever is in your simulation you're going to uh you're going to have one timer function that is going to be running but it won't have any of the business logic this is a uh an important component it won't do any of the actual business logic itself but it will instead uh query the graph get some information out of it and then just send out a bunch of uh messages to event hubs that will then trigger other asynchronous functions that will enact that logic because there's going to be a limit on how long a timer trigger can run if you'll you'll get into the scale problems if you have all that business logic there so all you do is you just it just get some data from the graph and then for everything that's unresolved you just put out an event Hub notice the event Hub trigger then takes all of those notices and it asynchronously can place all of them in the graph Azure Cosmos DB is generally known as high read High write so it can take it can handle that load and there's no user waiting for the consistency the eventual consistency of the Azure Cosmos DB is going to ensure that uh the model is kind of an always on churning thing uh these can be references to specific IDs for example device IDs uh digital twin IDs they could be uh in this in my case every object has a unique object goodd uh and that way I I know when I can select an item if it has an action that's associated with it I can send off a notice to resolve that action and a different function will pick that up and resolve that action that makes it very easy to have an ongoing churning model that's going to uh present all the data that you need in your simulation so that whole system working together again very very small number of components this is very easy to build easy to run uh your website is the interactive layer but you're really just kind of getting in setting items from the cosmos DB graph anything else that takes more than a web speed amount of time is really going to be hand L by asynchronous functions that are going to run all those jobs in the in the background and the user is oblivious to the time difference of those when they happen uh that means that each of the individual components is creates experience there is a a genuine model that's happening in the back you're using general common random features to create random events or random things that could happen or if you're planning to design a system that's designed for chaos You're Building components that break or fail and then you can have that in simulation you can you can bring that out of course Cosmos Tob can handle a lot so you can not only build one simulation of the same model but you can replicate it to build any number of asynchronous models running at the same time uh that's it thank you very much uh feel free to reach out to us if you are considering running a model like this it's a a personal project of mine to uh build these kinds of systems and um I am eager to connect with people who are building similar experiences either for hobby projects or for industrial applications thank you very much