Streaming and Caching Speech with Supabase

Key Takeaways

today let's look at streaming and caching speech using superbase Edge functions and super base storage so let's have a look at the demo real quick so let's say we want to generate you know a bit of longer text uh into audio and we you know select a specific voice here Sarah and so you can now see that um we're making a request 11 Labs raises 180 Ma in series C funding to advance its AI audio technology and we can look at the timing here so you can see that and sound you know the request is not finished yet so it is streaming uh the response back so initially we're just waiting for the server so I'm running this locally here so there's a bit of overhead there but then once we get the speech streaming we kind of keep um you know streaming it in the background until it's finished also when we pause here you can see kind of the the buffer sort of coming in uh and streaming in and so this is great so basically the first time you know we're generating this um you know we want to generate it and then we want to save it to super base storage so say the next time that we're um generating the exact same text with the exact same voice 11 Labs raises we can now see that it's coming immediately from the CDN and we're not actually generating anything anymore and so that is exactly what we want to build today and we can leverage superbase Edge function and superbase storage which has a built-in smart CDN for this so let's have a look at the code so uh our front end here is just a nice little static side generated by lovable uh it's great love very much and so you can see here we're just using react and um initially we're just fetching all the voices here from the 11 laps API uh and putting them into our drop- down here so we can generate kind of the you know different voices that are available and then uh what you can see is we're you know just setting up kind of our react State status is here fetching the voices so once we have the voices and then you can see really the magic is happening here in the handle generate so all we're kind of doing is you know we're running locally here we're triggering our function which is called text to speech with a text parameter and a voice ID parameter and so let's have a look at the superbase function here that is making that happen this is our superbase function so we have the edge run time here uh our superbase client our 11 laps client and we're using uh an object hash uh library to generate a hash for um the parameters that we're taking in from from the API and so you can see here we have um a background task so uh one thing that we want to do here is that we want to immediately start streaming back the response you know as the speech is being generated immediately stream that back to the user and then in the background upload it to superbase storage so we just have an async method here um where we're passing in a readable stream and then you can see superbase storage and we're just uploading our audio here to superbase storage so we can see now if we open the local um superbase studio and we go to storage audio so we have two things that were generated here 11 Labs raises 180 MERS in series and so these are different voices 11 Labs raises 180 m and so there different hash so we're just naming our file with the hash of the request parameters and so once we have that uploaded that's great that's our Asing function here and so this is our request Handler so as we're setting the audio Source here we're setting the audio Source on our audio um element you can see that here so we have um once we have an audio Source we then um setting that into the source of our uh HTML audio element and so when that is happening we're making a request to our server and we're just getting out the parameters so specifically the text and the voice ID and so for the same text and the same voice ID we want to look up if we have um you know a cached an already generated audio file in superbase storage so what we're doing is just we're generating hash um over the request so the text and the voice ID parameter um and then what we're doing is we're looking up in superbase storage if we have a file for this hash and then create assigned URL for it and so if we already have a file what we can do is we can fetch that file and we can then simply return the response of that fetch request back to um you know our front end and so the great thing here is that superbas storage has built in smart CDN so we're basically just serving this from a global uh CDN which makes this really Snappy really fast um as you saw I think it was like you know obviously also it's St here locally on my machine so um it's on the edgiest edge it can be um okay great but so say we don't have it cashed already we want to generate it and so that's where we're making an API call to 11lbs and specifically here we're using the text to speech and we're using the stream method so we're passing on our voice ID our text um a specific MP3 format and the model ID and then what we get back as a response is uh a node stream so we can then uh chunk through the node stream create a new browser readable stream so that's what we're kind of using in Dino and so once we have that stream uh a really cool feature of this stream is that we can tea off the stream so you can think of it as you know like a like a branching off so we're basically cloning our stream which is a really cool feature here so now we can pass one part of the stream to our background function so we can start uploading you know in the background uh and the edge run time will just wait until that upload is finished but at the same time without you know blocking we want to immediately start returning the browser stream back to the user to the client so that immediately as we get um you know the initial part of the speech generated um get that streamed back to the user response and you know you can see really this is just a couple lines of code but by combining these Technologies you know 11 laps uh Speech generation with streaming and then super base Edge functions and superbase storage which has the smart CDN built in this you know becomes really powerful in building applications here and so now lastly we can say thanks for tuning in and we can maybe see what uh Roger sounds like all right let's generate that thanks for tuning in all right thanks for tuning in let us know what you want to learn next and I'll see you then bye-bye