How-to do Sentiment Analysis with Flair in Python

Key Takeaways

hi welcome to this video on sentiment analysis using the flare library so flare is a incredibly simple easy to use library which contains a load of pre-built models for nlp that we can simply import and use to make predictions so it actually allows us to use some of the most powerful models out there as well so in this tutorial we're going to be using the the stilbert model which is based on a bert but it's a lot smaller but almost as powerful as burst itself so we're going to go ahead and begin first if you haven't already you need to hit install flare and alongside flare you are also going to need pi torch if you haven't got pytorch installed already you'll need to head over to the pytorch website and they give you instructions on exactly what you need to install so we come down to here and we can see okay for me i have windows i want to install using conda using python and then cuda so this is if you have a cuda enabled gpu on your machine if you don't know what it means you probably don't so in that case just click none but for me i have 10.2 so all we need to do is copy the command underneath here and then we would run this in our anaconda prompt i already have these installed so i'm going to go ahead and actually begin coding so we're going to need to use pandas and also flare so now we have import flare we can actually import a sentiment model straight away so all we need to do is i'm going to pass our sentiment model to a variable which we will call center model and we just need to write flare.models.textplusfire and load and then in here we pass the model name that we would like to load and in our case it will be the english sentiment model which is en dash sentiment like so okay so now we are downloading the model and in a moment that will have downloaded and we can begin using it now obviously we need data i have downloaded some data here which is a sentiment data set based on the i think it's the imdb movie reviews so you can find the same data set over here okay so it's central analysis on movie reviews data set so it's from rotten tomatoes and you scroll down and we have the training data and test data here i'm going to use the test data but we can use either we're just going to be making predictions based on the phrase here so we need to read in our data so it's going to read it in as if it were a csv file and we'll just pass a tab as our separator because we are actually working with a tab separated file okay so here it's actually a tsp not csv okay so the first thing you'll notice is that we actually have duplicates of the same phrase that is actually just how this data set is it just contains the full phrase initially so this first entry here is the full phrase and then all of these following are actually parts of that phrase so what we can do so let's change it so we can actually see the full phrase first okay so we can't really see that much more anyway but that's fine so to remove this we just want to drop all of the duplicates whilst keeping the first instance of the sentence id so you see each one of these they all have the same sentence id it's actually only the first one that we need so we just drop duplicates on this column keeping the first entry okay so we're keeping the first entry dropping duplicate some sentence id and we're just doing this operation in place okay so now we can see each sample is now a unique entry okay so now our data is ready so we need to actually first convert our text into a token as list using flair so flare does this one sentence at a time so if we for example pass hello world into the flare tokenizer we will be able to see what it's actually doing okay so here we can see that it split each one of these into tokens so we've got hello as a token world is a token and then we have also split the estimation mark at the end there and you can see that flare is telling us that there are a total of three tokens there so each one of our samples here will need to be processed by this flare.data.sentence method before we pass it into the actual model once we do have this so let's um let's call this sample as well we will pass it to our model for prediction which is really easy all we need to do is call the predict method on the sample and now this doesn't output anything instead it actually just modifies the sentence object that we have produced so it modifies sample and we can see now that our sample we started a sentence and we saw the number of tokens but we also have these additional labels which are the predictions we have the label which is positive which means it's a happy or it's a positive sentiment and then what we have here is actually the probability or the confidence in that prediction that's great but realistically we want to be extracting these labels so we're actually able to extract these by accessing the labels method so we have labels here and this produces the positive and the confidence to access each one of these we access index zero followed by dot value okay so this will give us the positive and then we can also do the same to get the confidence called score like that so what we can do now is just create a simple for loop that will go through each sample in our test data and assign a probability for each one so we will initially create a sentiment and confidence list and then we will just as we are looping through the data we will append our sentiment values so the positive or negative and the confidence to each one is list so here we are first tokenizing our and sentence we are making a prediction using that tokenized sentence which we are calling sample and as we did before we have now got this labeled sentence and we just need to extract the two labels that we have here okay so we can see here that one of our sentences was just blank so we'll add in some logic to avoid any errors there okay so looking at this it's also whenever there's a space as well so we just need to trim this which we can do easily using the strip method okay so it took a little bit of time but we now have our predictions so what we want to do is actually add what we have here in the sentiment and confidence list to our data frame so to do that we just add df sentiment to create a new sentiment column and we made that equal to the sentiment list that we have created and then we also do the same for confidence as well then we can see our data frame okay so initially looking at this it looks pretty good so intermittently pleasing but mostly routine effort incredibly negative but basically saying it's occasionally okay but generally nothing special solves it's a negative sentiment which is matched up to negative sentiment here here we're saying okay kidman's the only thing that's worth watching him birthday girl and it says is another example the sad decline of british comedies in the post full monty world fair enough also negative so this one is our first positive once you get into it it's real the move becomes a heady experience yeah i mean sounds pretty positive to me so it's quite good even here where we're not saying anything particularly like a negative or positive word we're just saying that the movie is or the movie delivers on the performance of striking skill and depth which must be pretty hard for a machine to understand and actually get right but looking at all these it's doing really well and i think it's really cool that we can actually do this with so little effort and we've only actually written a few lines of code in reality and it's producing really good accurate results which is really impressive to me so that's it for this video i hope it's been useful and thank you for watching and i will see you again in the next one