Make Crazy Art with the NEW OpenAI Dall-e API

Key Takeaways

artificial image generation has been one of the biggest Trends in machine learning over the last year we've seen all kinds of crazy demos like ai's depiction of human evolution and there are now a bunch of different apps out there that allow you to convert text into images like the stable diffusion demo on hugging face however if you want to generate your own images programmatically you'll need to know a thing or two about deep learning and have the hardware to run these compute intensive models well at least that was the case until now because openai just released their image generation API based on their Dolly 2 models that means any developer can now easily generate high quality artificial art and in today's video you'll learn some of the scary things that this API is capable of and I'll throw out some random business ideas for you of apps you might want to create with it before we get started you'll first need an open AI account and you should know that this is a paid API however they give you 18 dollars in credits to get started when you run out of money it will cost you about two pennies per image or 50 images per dollar that's at the max resolution which is currently 1024 pixels it's pretty expensive but as the co-founder of openai Elon Musk would say you get what you pay for go ahead and generate an API key then let's open up vs code and start a new node.js project although you can also easily do this in Python I'll run npm init y to start a new node.js project then use npm install to add the openai SDK for JavaScript now I'm creating three different JavaScript files because I want to show you how to do three different things generate a new image edit an existing image with a mask and create a variation from a source image then finally I'm creating a directory to hold all the image results and you'll also want to go into your package Json and turn the type to module just so we can use es Imports in our code but before we write in the actual code here's a business idea you could build a SAS product that allows a blogger to upload an article then you automatically generate five or six images that go along with the context of that article you could even take an old public domain book like Joseph Conrad's Heart of Darkness create a bunch of AI illustrations and republish it as an illustrated novel now that you have some terrible ideas to get you going let's implement it in code first we'll need to to import the configuration and open API classes from the SDK we create a configuration with the API key and make sure not to expose this API key publicly and then initialize the SDK next I'm creating a variable for the prompt which is a description of the image you want to generate like a ship sailing through a river of fire in deep space and now we can use top level weight to make a call to the create image endpoint on openai it takes the prompt as the first argument as well as the number of images you want to generate and the resolution you can also pass a user ID here which allows open AI to identify abuse like if you have a user who keeps trying to generate boobs and wieners that API call will result in an image URL now that we have that URL we can navigate to it directly or better yet we can save the image to disk in this demo I'm using node 18 which gives me access to the native fetch API in node we can use it to fetch the image URL then we can set the result as a blob we'll need to turn that blob into a buffer which we can do with buffer from by awaiting the blobs array buffer that gives us a format that we can then write to disk with write file sync from the node file system API and now we can generate some art by opening the terminal and use node to run the generate.js file it takes about five to ten seconds to generate an art piece but overall the quality is pretty impressive the next thing I want to show you is how to create an image variation let's go ahead and copy all the code we just wrote and paste it into the variation file now just delete the middle part where we make the call to open AI we're going to replace this with a different API call called create image variation it takes an existing image as a starting point and creates a different result this endpoint doesn't take a prompt but instead takes an image as its input I'm using a picture of the Mona Lisa as that input we pass the image path to create read stream from node.js which is just a fancy way of opening the file then pass Arguments for the number of results and resolution we can now go ahead and run this script from the terminal and it will generate a different version of the Mona Lisa a much uglier one occasionally Dolly produces some really cool stuff but for the most part it's not very aesthetically pleasing I did some testing and recursively generated the Mona Lisa 25 times along with a bunch of other art and found that the algorithm leans towards cartoon characters like The Simpsons and stuff like that if you keep feeding its own results back to it it eventually just devolves into garbage because it's not really making art but rather combining the art made by millions of humans into something that looks like art at first in any case there's one more thing I want to show you and that's how to change a specific part of an existing image I think this aspect of the API has the most potential because you could use it to augment an existing image in more subtle and interesting ways to make something like this we'll first copy our code over to the edit file and this time replace the middle part with a call to the create image edit endpoint this endpoint requires two images one is the source of the full image and the second is a mask or an area of that same image that's transparent that will be replaced with something AI generated I'm using a picture of myself writing some code now the question becomes how do we create a mask there are many ways you could go about it but I'll show you how to do it in figma first we have the original image and now we have a duplicate of it and I'm using the pen tool to draw a shape around the area that I want to mask from there we can select both the image and the shape and then go up here and do subtract selection that should turn the shape white and then we'll also want to go and make sure that the frame does not have a background and then we can export it as a PNG into our project now if we go ahead and run the code we get an image back where only the computer screen has AI generated art that's pretty cool and I think there's a lot of potential for creativity here this concludes our openai Dolly tutorial thanks for watching and I will see you in the next one