Synthetic Data Generation for SQL Commands: Step-by-Step Tutorial

hello everyone and welcome back for another tutorial my name is mik masi your guide for today and in this video I'll be walking you through how to generate synthetic data for your own use case last week we talked about how to fine-tune large language models and this tutorial is a perfect followup for that if you've been working with AI for a while now you know that data can make or break a project but let's be real getting good data is tough it's expensive hard to find and sometimes just not available in the MS we need and with the world starting to run low on data right now fine tring models is becoming a real headache that's why synthetic data is becoming more relevant than ever creating artificial data that mimics the pattern of real data can help us bypass all those issues for this tutorial I'm focusing on creating an SQL commands instruction response data set but you can easily adapt this notebook to create any type of data set you need anyways enough with the talking let's open our notebooks and get started first thing first let's set up our environment for this tutorial I'm using the quen 2.5 72b instruct model which is great at generating SQL queries but you can use any model available on hagging face just make sure that the output it generates matches what you're going for then we can start the data generation process everything begins with subtopic generation subtopics help us break down the broader theme into smaller more manageable chunks in our example we're generating data related to SQL commands so instead of asking our model to just generate generic instructions related to SQL we break it down into subtopics this helps the model focus on more specific aspects of SQL ensuring that the generated instructions are precise relevant and detailed without subtopics we are getting broad less useful data that doesn't cover nuances now that we have a list of subtopics The Next Step will be to generate a number of instructions for each subtopic these instructions are essentially the kind of questions a user might ask or the tasks they want to perform we will prompt our model to do just that here's a quick tip I usually keep the temperature setting lower for a more focused output this helps the model stay on track and stick to the prompt when generating instructions after this cell finishes running we can take a look at our instructions if you're not satisfied with the output you can edit the prompt and rerun the cell but what we have here is perfect for me so I'll be using it in the next step which is generating the responses that go with each instruction all we have to do is feed each instruction into the model and tell it to Output the appropriate response as you can see in my prompt I asked for only the SQL command with no extra text and if you look here that's exactly what I got looks like we have a solid result if it looks like some pairs are not good enough don't worry because in the next step we'll be filtering and refining our data set to do that we need to use something called a reword model reword models are a type of machine learning models specifically designed to help evaluate and rank outputs think of them as judges they look at the data we've generated and assign a score based on how useful relevant and accurate it is in our case we'll be using the neotron 4 reword model specifically to evaluate relevance and helpfulness we set a quality threshold if it data pair doesn't meet the threshold we discard it this ensures that we have only the most relevant high quality examples so that when we use them for fine-tuning we get the best results possible and there you have it all we have to do now is save it and put it on hugging face go back to our fine tuning tutorial and try using it I hope you found this tutorial helpful feel free to leave a question or comment down below