Deep Learning for Developers

Key Takeaways

all right i think you can get started now please take a seat so uh good morning everybody my name is julian i'm a tech evangelist with aws i've been working with them for almost three years now and these days i focus on ai and machine learning and it's really nice to be back in krakow i was here for a code europe in december it was cold and dark very cold and very dark it's very nice to enjoy the city with sunlight and getting a chance to visit it's really beautiful so good to be back so my topic today is deep learning for developers and well the title is really important because it means if you can read and write let's say 50 lines of python you can do this okay so we're gonna start with an introduction to the underlying concept some of the theory that actually holds deep learning together but i'm gonna try to make it as pragmatic and and concrete as possible with minimal theory and then we'll look at some code run some examples and try to put everything together okay so you've heard of deep learning as a subset of machine learning that uses a technology called neural networks right so the first thing we should uh we should look at and i realized this is really hard to start with this slide but hey we got to understand what this thing is all about right so stay with me here we got to understand what a neuron is and what we're trying to do really is trying to build uh some kind of approximation of a biological neuron so what we know about biological neurons is they have inputs they have connections and if those inputs are stimulated enough then the neuron will actually fire it will emit an electrical current if it's not stimulated enough it won't fire okay so we're trying to build something that looks a bit like that so obviously for a deep learning neuron we will have inputs okay and these will be floating point values um we'll see where those come from and to each of these inputs is associated a weight which is another floating point value and this weight in a way makes this input more or less important than others okay and the first thing that we do when we actually uh run something on that neuron is to compute this multiply and accumulate operations so we take each input we multiply it by the corresponding weight and we add everything together okay it's called multiply and accumulate so there you go right deep learning is really about adding and multiplying stuff and doing this at very large scale so if you can add and multiply you already know 50 of what deep learning really is okay but the problem with this function is is that it's a linear function if we change the inputs right if the inputs vary linearly then the result that u value also varies linearly and that's not what we want because remember we want something that fires or not okay so there needs to be some kind of limit some kind of threshold that says when does the neuron fire or not so this is why we add the activation function at the output of each neuron and over time a number of functions have been used these days a relu is the popular one and as you can see uh here it's it's non-linear right if if the input value to relu okay so if that u value here is negative then the neuron will output zero and if the input to the activation function is positive then the neuron will output that same value and it could be a very very large value okay so this is really what we want right nothing happens until a certain point and then we output a value and this could be a really large value okay so if the neuron is stimulated enough it can actually output very large values and this is what we want and this is how we introduce that non-linear behavior in the neuron okay so obviously a neuron by itself doesn't do much they become interesting when we combine them into layers and networks and this is probably the simplest one you could build okay so we have an input layer where we will actually put our data data samples to predict we have an output layer where we will read results right we will read the predictions and in the middle we have a hidden layer at least one so in this case just one and as you can see it's fully connected right so all neurons are fully connected to the inputs fully connected to the outputs okay that's why they're called fully connected networks okay so let's say we're already trained listing okay we'll we'll talk about training in a few minutes but let's say it's already trained how would we use it okay so of course we need data samples and for this discussion let's say we're trying to classify pictures okay so we have a number of pictures let's say animals right dogs cats elephants tigers anything that you want and we know what they are okay so we're going to take each image we're going to flatten it into a vector and we're going to store the pixel values in a matrix like that okay so this is the first picture this is the second picture and so on and so on okay so each picture has has been transformed into a vector of pixel values okay and we know what those pictures are okay so uh um we know for example that the first picture here is a category two and maybe category two are dogs right and we know the second picture is category zero maybe that's cats and so on and so on okay and let's say we have ten categories in total so that's the data set okay a number of pictures that we flatten into vectors and that we label with category numbers from zero to nine let's say okay actually we're not we don't like using this if you already work with machine learning you know we don't like categorical values to be handled like this we use another technique called one hot encoding okay and that's a complicated word for a simple thing one hot encoding means instead of using category numbers you're going to build a vector of bits okay and again let's say we have 10 categories here so we will have 10 bits and we'll flip to one the bit that corresponds to the right category okay so if this one is category two then bit zero one two is flipped okay this one is category zero so we flip bit zero and so on okay and why is this important because well this form right is actually much more expressive than this because here we can see how many classes we have first and second we could look at those numbers as probabilities for each of the classes okay and what this really says here is uh this sample has a hundred percent chance of being category two and zero percent chance of being any other category right we know for sure because we know what the data set looks like okay same thing here we know that this last sample here is category four so it has 100 chance of being category four and zero percent chance of being anything else okay and this is this is actually what we will get on the output layer when we when we put those samples on the input layer here right and we run those multiply and accumulate and those activation functions etc what we will read here in those neurons are the probabilities for the samples okay so hopefully if we train our network right okay we would get to something like this we take a data sample and as you guessed we need add many we need as many neurons in the input layer as we have features okay so if we have a thousand pixels here we'll need a thousand neurons here okay then we run what is called forward propagation so running that sample through the model multiply accumulate activate etc and you read some results on the output layer and again if you have 10 classes you will need 10 neurons on the output layer okay and you will read the probabilities for each class so in a perfect world this is what we would see right we would really see zeros every everywhere except a one in the new round that corresponds to the right class to the right category but nothing is perfect and it's not going to happen like that anyway what we care about is accuracy so being able to successfully predict the maximum number of samples okay this is what we're going to measure and track okay so this is where we want to get but of course initially the network is not trained okay so all the weights corresponding to all those connections here right all those parameters they have random values so if you take a data sample and forward propagate you get something completely wrong on the output layer and it would it would not be something like this actually it would not be a one in the wrong place and zero zeros it would just be random probabilities for the ten classes okay because again the network has not been trained so if we see this neural network as a as a function and we take a sample and compute the output we're not going to get the right output we're not going to get the right label the right probability vector we're going to get something different okay let's call it y prime one so obviously you need to measure the difference between what you expected and what you really got okay and we do this using a math function called a loss function and remember that y one and y prime one are really vectors okay probability vectors so it's not as easy as uh subtracting one from the other but you don't have to worry about those because all the deep running libraries provide those loss functions already so you can just use one that matches you the problem you're trying to solve okay so this loss function will give give us the error okay just a numerical value measuring the error the distance so to speak between the two vectors so actually we're not going to do this sample by sample okay imagine you have 10 million samples in the data set it takes too much time to process every single one and and train on every single one so we actually train on batches of samples okay so we're going to take 32 samples for 64 samples at a time run them one by one through the model and compute the total error for the batch okay and then we're going to make decisions on how to reduce the error okay more on this in a minute okay so keep in keep in your mind we train on batches of samples not individual samples okay so in a nutshell the purpose of the training process is really only one thing it's to minimize loss okay minimize prediction error for the data set by training over and over right iteratively and by by adjusting the weights gradually during the training process okay so what we're really trying to do here is um in these examples we i don't know how many weights we have maybe let's say 25 or something like that okay we're trying to find the set of weights the set of parameters that give us the highest accuracy for that data set and that means the lowest possible error for that data set okay and we start from random weights and we have to try to go as close as possible to that optimal set of weights okay and that's a difficult problem because you cannot compute them right it's not something it's not an equation you can solve so you have to gradually discover what those weights should be okay more on this in a second so the training process really looks like this we have a data set like i said we slice it in batches and again we'll see in the code that deep learning libraries do this automatically so no no work needed and we take one batch okay let's say it's 32 samples and we run each one of those samples into the network okay and we get to prediction and we compare the prediction to the actual result we were expecting and we compute loss okay and we do this for about all the batch samples and we add up all the errors into the batch error okay so we now we have a batch arrow that tells us you know what's the what's the mistake how big is the mistake we made for this batch okay and now we can take decisions on how to reduce it and this is done with an algorithm called back propagation and it's a it's a scary one it usually that's usually where when people stop looking and uh and stop studying deep learning because they want to understand back propagation in detail and it's a bit scary okay so back propagation as the name implies will go back from the output layer to the front okay and layer by layer and neuron by neuron it's going to adjust the weights in a direction that we know reduces error okay so for example here okay this neuron the the error value that you get for this neuron depends on three parameters okay so for these three parameters you have to figure out if each of them should be increased or decreased to to lower error right or remember these are floating point values so these are the only two choices that you have right you can increase them a bit or decrease them a bit so how do you know okay i will answer that in a second so once you've done that here okay you've updated those you've updated those weights so you could compute the new error for the next layer and then you do it again so this the error that you get here for this individual neuron depends on one two three four five parameters okay so once again here you need to make five individual decisions on increasing or decreasing those weights and you do this for every single neuron and then you move back to the previous layer until you get to the input okay that's why it's called back propagation okay so now obviously uh you want to do this all over again for the next batch right compute the batch error back propagate and you do this again and again and again and again okay until you get to the end of the data set okay and this is called an epoch and typically you're going to train for 50 100 200 epochs okay so it's really an iterative process and you train and you predict again and again and again and again right computing the batch error and then adjusting the weights in a direction that you know will reduce error and and you do this until you hopefully get to the right level of accuracy that you were expecting okay so that's the big picture okay and at the end of that process you get to a trained neural network okay and as you can see um we have a number of parameters that are really important the batch size is important if you have very tiny batches you know you back propagate a lot you take takes a lot of time so yes you will probably get to the right spot but it takes too long if you have a very large batch size then you get less opportunities to run back propagation per epoch so maybe that's a problem too the learning rate will actually decide on the the the size of the updates that you make to the weight okay we'll we'll talk about that in a minute so it's important as well small learning rate small updates to the weight large learning rate large update to weights okay and again too small will be a problem too large would be a problem and the number of epochs is how many times you go through the data set these are called hyper parameters and if you don't get them right you will have lots of problems training okay so that's the big picture okay it's not really complicated the only weird thing is hey i get the batch size the batch thing and i get the back propagation but then you said we need to adjust the weight the weights each weight actually in the direction that we know reduces error how do we know right how do we know if a given weight should be increased or decreased okay let's look at a real example so imagine you have a function here let's call it f and it has two parameters okay and the output is let's call it z okay and let's let's say that x and y are parameters and z is the error okay and which we start with random values of x and y and we want to get we want to figure out what x and y should be to get the smallest value of z okay that's really what we're trying to do find the set of parameters that give us the smallest possible output okay and if we plot that function let's say it looks like it looks like this okay and remember x and y are initially random so we're going to start anywhere here okay so let's say we start here okay and let's say this is x and this is y all right so we want to get to the smallest possible value of z so probably that's here right so in a way we want to walk down that slope until the lowest point in the valley right we're trying to get down to the valley and remember we cannot compute x and y okay we have to discover them so how would you do that right if you were in the mountain how would you do that imagine you're in the fog you see nothing um and you have to decide if you uh you know if you should go forward or backward and if you should go left or right okay so what you would do is maybe you know you would with your foot you would try in this direction and say ah yeah okay this is down all right fine and then you would go right and say oh no this is up oh this is down okay so this is down this is down so i'm going to take a small step here and i'm a little lower than i was before okay and you do this again and again and again and again right and if there's not a big crack if it's a smooth surface like this if it's friendly you will actually get to the lowest point iteratively by taking small steps in the right direction okay that's the intuition now obviously deep learning doesn't have intuition it's it's math so how do you know for sure that you need to maybe increase x a bit okay or actually here it would be if the origin is here okay x and y actually we have to decrease x and y a little bit to go down right initially so how would we know well remember high school that's the part you're gonna hate by the way so get ready to hate it uh remember high school right i remember slopes and remember derivatives right no all right are you too kind right well that's exactly what we do here okay this is a function okay yes it has two parameters but we can compute the partial derivative for x and we can compute the partial derivative for y at this specific point okay and so that gives me the slope in the x direction and that gives me the slope in the y direction and then if i have the slope i know which way is up and which way is down okay simple so once i know that well i should decrease x a bit to go down in the x direction and i should decrease y a bit in the y direction then fine i'm just going to do that i'm just going to going to modify x and y just a bit in the right direction and i get let me say here okay and i do it again and again and again and again okay and this algorithm is called stochastic gradient descent or sgd it's the granddaddy of optimizing functions uh it it dates back to 1951 okay so it's nothing new it has actually nothing to do with machine learning it's a it's a math optimization function and that's the one that is still heavily used in deep learning okay so coming back to my example here this is what actually [Music] will happen during the training process when you run back propagation okay the library that you're using will will compute the derivative for each of those three weights okay with respect to the error and then decided they should be increased or decreased do that and then move on to the previous layer and do it again and again and again okay and this is how you know in which direction all those weights should be updated and since we're taking small steps we need to do it again and again and again right and this is why we do batch training and this is why we need many epochs because we take very tiny steps but we if we take tiny steps always in the right direction right eventually we get to the right place okay that's all there is to it right high school math nothing to worry about um so obviously surf the surface is not going to look like this right would be too easy it could look like this right or you could be seeing things like this where yes you seem to have a nice global or you know a lower minimum here and you have you know not as good ones here so maybe you could start you know on the mountain here and and walk down and maybe you could fall in this hole here or this one right and let's say this one gives you a higher error than this one does so it's probably more desirable to be here than to be here right and they're called local minima and another problem could be saddle points and this one is actually even worse so it's look it's like a horse saddle and uh in this direction right we can see yes this point here the green point here is actually the minimum so fine but in the other direction right like this this point here is actually a maximum right and if you remember high school again you know that the derivative is is equal to zero when a point is a minimum or a maximum okay so if we actually started here and went down exactly to that green point uh we'd be in trouble because derivatives would be zero we could not update the weights okay so saddle points are actually a problem too so it's a it's a long-lasting debate in the deep learning community on whether these things are really exist whether they really are a problem do they do we really meet them in our daily experiments etc and this reference article from 2015 says pretty much yes okay these things exist okay and keep in mind here we have only two parameters two dimensions but state-of-the-art models they can have millions tens of millions of parameters okay so you have to try to visualize this in 10 million dimensions okay if you can actually do that welcome to earth right please solve our problems okay so chances are you cannot okay and i can't either so that's why intuition is really important so yes it's quite likely that we have weird things like local minima and saddle points but what uh ian goodfellow says is yes they're here but they don't really impede the training process right we by some magic we don't quite understand we actually managed to go around them or escape them so you know don't lose too much sleep about those anyway right that's the number one question i guess ah yeah but deep learning is just uh it's all crap because of local minima it's like yeah okay in theory yes there is a problem in practice um it it doesn't really bother us okay if you want to read that one you'll get more information okay so that's for the training process right so we do that again and again and again um and how do we know that we're actually making progress well we keep part of the data set we call this the validation data set okay and it's very typical to do this in machine learning you split your data set between training validation and and periodically so at the end of each epoch we run the validation data set through the model uh in training right which this should really say in training neural network not trained because it's not finished we're still training okay and we measure the prediction accuracy for this data set and this gives us a sense of how well or how bad the network does on sample on samples that it hasn't seen so far okay because remember these samples are different from the training set okay so we're going to do that we're going to see this at the end of each epoch and hopefully it goes up showing that not only is the network learning it's also learning to predict samples it has never seen and at the very end once we're done right once we're completely done with training we're done tweaking and we want to compare this model to the previous models that we trained last month last year etc we want some kind of benchmark we're going to use a test data set okay so a third data set that gives us just um again a benchmark accuracy for that model okay so it's a very typical way of doing things three data sets okay so if we plot all those things it should look like this so we should see training accuracy going up really quick and then plateauing if that's a word and if we train long enough we get to 100 accuracy okay guaranteed it's if the network is deep enough large enough if you train long enough if you have enough data you always get 200 okay so the loss function the the error right the prediction error uh obviously goes down almost to zero if you plot the validation accuracy it's possible you see something like this okay so again it goes up it tends to follow the training accuracy and plateaus and and you might get you know a high value here like a small bump here and then plateaus again and then it drops and it never never recovers okay it's quite possible this is really more you know more jittery than this it could be like this right very very jittery but then at some point it drops and never recovers and that's extremely bad okay this is called overfitting it's a the number one problem in machine learning and what overfitting really says is you train so hard on the training set that it's the only thing you can predict okay so the model is specialized in predicting the training data set it cannot do anything else okay so that's really bad because obviously in real life you don't want to predict the training set you want to predict real life samples right and these will uh come from you know the validation accuracy anyway so this means that when you're training you should really uh plot this you should keep an eye on the training accuracy of course but you should also keep an eye on validation accuracy and and you should plot this thing at the end of training and decide which one is the best epoch right which which version of the network should you actually use for prediction okay and you cannot know in advance so in order to do this what you really do is save at the end of each epoch you save the weights right you save the version of the model at the end of this epoch and then you plot and you say okay i trained for maybe 200 epochs but the best one is actually 100 129 and that's the one i'm using okay so you have to save the weights um all the weight at the end of each epoch to know which one you're going to use in the end okay so to sum things up deep learning is really about finding a minimum right you you don't care about finding the minimum you'll never know if you found it it's a it's a np hard problem so there you go so all you need to find is a minimum that's good enough right to give you the accuracy that you need for your business problem okay if you if your business problem requires 95 accuracy then fine right do that um no point in in chasing uh 97 for weeks and weeks and weeks you maybe you'll never get there so you need to decide what accuracy you need to solve your business problem and once you get there good okay and of course you need to find it fast because if you were able to find it fast if you can actually learn quicker then you train for shorter periods of time you can iterate more you can save money on training et cetera et cetera and if you can't find it reliably if it's not just a lucky accident if you can find it again and again and again it's very good because again you will train many models you will try many combinations you will try different data sets you will try different hyper parameters you will retrain periodically etc so you need to be able to hit this accuracy level again and again and again if you just hit it once and then have trouble hitting it again then you know it's not it's not a viable solution so let's look at a first example yes that's the one okay i guess that's large enough so i'm going to use here i'm going to use a deep learning library called apache mxnet which which is the favorite library at aws and we're going to try to classify a data set called mnist which i'm sure you've seen before uh it's a it's a nice toy data set it's not good for anything except experimenting okay so this is mnist right over there so 70 000 digits handwritten digits so zero to nine and obviously the game is to learn to classify those images in the right category okay and they're black and white images 28 by 28 pixels so the first thing i'm going to do is download the data set that is already split in training and validation i'm going to import mxnet i want to train for 50 epochs i'm gonna use an iterator to load the data set from the files okay and the iterator will slice that data set into batches automatically for me okay so i don't have to slice it into pieces and pass that to the model the iterator does it automatically for me and then we build we build the network and this is almost yeah this is pretty much the one we saw on the slide except it has two hidden layers so i've got a first layer which is the input layer i call it data i don't need to give it size mxnet will figure it out automatically from the from the iterator and if you remember that slide with the fully connected network um my input layer is flat okay it needs that it needs a vector right it needs uh the data samples should be vectors so i have to flatten my images okay so i flatten the image the images into vectors and then i have a fully connected layer with 10 24 neurons active activated by relu and then i have a second fully connected layer with 512 neurons activated by relu and then i have the output layer with 10 neurons because i've got 10 categories right zero to nine okay so those six or seven lines are all i need to actually define my neural network okay and it doesn't matter which library you use it you'll find something similar to this you just stack the layers like this you connect them and obviously you never worry about connecting individual neurons that would be that would be painful okay so now i've got a model okay and i've got a data set in an iterator and i have a model so i need to put the two together this is called binding here okay so binding the model i just created to the training iterator and the validation iterator and then i need to set an optimizer okay so i could actually use a sgd right here with a learning rate a fixed learning rate like this why not um but there is a whole bunch i don't know i said sgd is a host that was invented in 1951 so you know more you know more interesting options have been invented since specifically algos like ada delta and so on that can actually modify the learning rate during the training process right so if this if the slope is very steep they can speed up and if the slope is very flat they will actually slow down and and allow you to explore okay and that's what adam means with adaptative okay so they will change the learning rate um during the training process okay and then i can train so let's actually run this one okay and we can see the uh we can see the training process going on okay whoops so we see the we see the epochs going by and we see the batches going by okay and uh and we see uh you know we we actually have a training accuracy that's uh that gets to one very quickly because it's a large network and it overfits pretty quickly so let you know i've done this before in the interest of time and i saved my model here and i get to a validation accuracy of 98.13 it trends for a few minutes but i don't want to waste a few minutes um okay so this is what i get right i train for 50 epochs that fully connected network on my data set right and i get to the validation accuracy of 98.13 so is this good is this bad how do you know right sounds good okay 98 but you will not know until you try until you try real-life samples so i did what i had to do i took my paintbrush application and i i drew some digits right and you can see them here and we're going to try and predict them using the model that i just trained so i can load that model again okay which is pretty much as easy as this right load checkpoint with the name of the file storing all the weights okay and then i can predict my digits okay and predicting one of those images is very easy you load the png file uh from disk you turn it into a vector right you flatten the image into a vector and you push it through the train network okay there's nothing to be worried about that's really one line of code okay load the image put it in a vector push it through the trained model and then read the outputs okay so very simple to predict and when we do that and let me run those cells again i just want to make sure i have the right model here oops there we go run all the cells okay and so for each of those pictures we see a vector of 10 probabilities right this is exactly what we read on the output layer for the for the neural neural network okay so we see that zero worked out okay because actually uh probability zero is the highest 99 okay and for the one probability one is the highest etc etc so two is very good three is very good and uh you never really see zeros and ones it's because i stopped at four decimals so this really means it's at least 0.9999 something okay you will always see uh non-null values here four is fine five is fine six good seven eight and nine this should be the highest right it's not this one is the highest so zero one two three four okay so my model is confused it thinks this is a four okay so that's what i told you you have to try different samples you know the validation data set is one thing but the test data set which should really be composed of real life samples stuff that people will really predict not just sample data sets this this is really what tells you how well your network performs so this is a disappointing right because i try 10 and one of them is wrong so why is that so you could say well this is really a very ugly nine so yeah you know you can't expect the model to predict ugly stuff like that okay fair enough but some people write really really bad uh and uh digits so you should still be able to figure it out but remember what we did to these images right we flattened them okay so intuitively it doesn't sound like a really great id because images are really 2d objects and we flatten them so we lose the relationship right the proximity between some of the pixels and probably that doesn't help understanding what's in those pictures so to solve this a different type of network was uh was invented 20 years ago already and they're called convolutional neural networks and these networks are the kings of image processing image classification okay um and the the basic idea here is we'll will work with the 2d image or maybe 3d if it's a color image right it's going to be 3d because it's red green blue okay but anyway we don't flatten anything we stick to uh to multi-dimensional objects and we use two operations we use convolution and we use pooling also called sub sampling okay so convolution is actually extracting features okay here's an example if you take this filter three by three and you slide it across the image right so nine pixels by nine pixels going all the way applying the convolution operation which is pretty much taking the underlying pixel here multiplying it by the corresponding value in the kernel okay and you do this nine times for the nine pixels here and you add everything together and that becomes the new top left pixel and then you slide and you do it again and again and again okay if you use this filter the specific values you go from this image to this image okay so this is really an edge detector filter and this what did we achieve here well we we detected the edges in the original picture and we threw away pretty much everything else everything else is black right we only kept the edges so we extracted the edge feature in this image okay and this is what convolution is about so by running many different filters okay it could be you know 20 30 50 filters you you extract different features from the image so it could be edges it could be contrast it could be vertical lines you know horizontal lines whatever you know and you get a combination a new set of uh convoluted images which you then shrink with pooling i'll show you how in a second and then you do that again okay and you get your collection of very tiny images that look nothing like the original image but they kept the good stuff they kept the features and the important information and then you can flatten this and use a fully connected network okay that's the the intuition between behind convolution networks you automatically extract features using a collection of filters and then you throw away you shrink the images you throw away the useless information in these pictures and you do that again and again and again and if you heard that deep learning extract features automatically this is it okay this is what people mean you use kernels or filters to extract important information and throw away the rest and obviously we have no idea what those values should be right here it's just a it's an example but initially the values for all the features generating those pictures are random so they are learned during the training process so the training process of a convolutional neural network is really about finding which filters extract the right features to help you understand those images right and it's the exact same process back propagation sgd et cetera et cetera okay so it gets a little more complicated but the overall picture is the same okay you learn the filters that lets you extract the right features okay and pooling is much simpler pooling is just about if we do a two by two pulling here we just uh take two by two pixel blocks and we keep the brightest right the brightest pixel the the highest value okay and if we do this all over the picture then you know we get a smaller picture and the intuition here is that in this image you don't care about the black stuff okay you want to throw that away it tells you nothing it was filtered out okay what's important is really the white pixels the edge of that weird beast okay so you can shrink the image by keeping the white pixels and it will gradually distort the image obviously but you uh you keep enough information to learn to classify them that's the id so let's really quickly try this so we're going to try to classify the same data set but this time with a convolution network and it's pretty much the one you saw on the slide right input layer convolution with 32 filters pooling 2 by 2 pulling convolution again with 64 filters pulling again 2x2 right so here from the initial image we built 32 and then from the 32 we built 64. okay and we shrink them so we end up having a large collection of tiny images and we flatten everything at the end so all those tiny images become one big vector and we use a fully connected layer to classify them into ten categories because once again zero to nine okay so you see this api is quite simple you can literally look at the at the network and encode it right just stack the layers and uh and there you go and the rest is the same right we bind we train so i train again in interest of time takes again two three minutes and we get to 99.15 accuracy this time okay so it's a bit higher than the previous one but now we're you know we're suspicious so we're gonna we're going to try that new model the one that we saved and we're going to predict again so 0 1 2 3 you know they all look very good six seven eight what about nine so is this the highest now yes it's the highest it's still not great right which is a tribute to my ugly nine here it's still not great but it's the highest okay so if i had to decide which thing is this then i would say okay it's a nine not super super sure about it but okay if i have to make a guess i'll tell you it's a nine okay and obviously you could keep tweaking this to get better results you could add have more filters more layers etc etc but again the intuition is really important okay that's why cnns work really well on on images it's because they keep the 2d or the 3d image and and they don't flatten stuff okay they just extract features from the actual image so it makes sense that they would be able to classify them better so you can do all kinds of crazy stuff with this and there's a an extra api in mxnet called gluon and specifically gluon cv which means computer vision that provides you with a collection of pre-trained models okay and these are state-of-the-art models with hundreds of layers they've been trained on really really big data sets and uh and you can do classification detection segmentation right these are just a few examples and any of those is literally five lines of code so i'm just going to show you one but you'll find everything on github let's look maybe at uh yeah let's look at uh okay classification okay so how complicated is it to do this so uh we're going to call this api to load a pre-trained model okay so it's called model zoo so we fetch a pre-trained model from the web already trained no training needed we load a local image we're going to normalize it okay normalization is important so make sure the you know the red green and blue channels etc are normalized and then we predict it so we take that image push it through the train model and then we can read the results and that's it okay so training your own model uh is fine and we could we could run one of those yeah let's try uh okay let's try this one okay let's try to segment this here we go okay okay so that's the original picture and i'm running it on my mac so no gpu so it takes a takes a few seconds but here you take an existing model pre-trained and and you just get the job done right so of course you could train that model on your own data set right there you go five six seconds no gpu so on a gpu it would be just like this um and this really takes five lines of code okay so before you go and try to build everything from scratch just consider existing models pre-trained models that that might be good enough to do the job right and you would save an insane amount of time by using those pre-trained models okay very last thing i want to talk about is we looked at fully connected networks and we looked at convolution networks okay and if we predicted with those if we took 100 samples and predicted them in any order we would still get the same results okay sequence doesn't matter which is a problem because when you want to work with time series when you want to work with sequences of data the order of the samples does matter right when we're translating for example we don't translate each word independently we look at the previous words to uh to actually uh get some context so we need some kind of memory short-term memory on the past few predictions to make the right prediction and this is what lstm networks are all about so they have a different type of neuron which i will not explain i don't have time and it's a it's a little more involved but you just need to remember that a prediction done by an lstm neuron depends on the input and it depends also on the past few predictions so they have some kind of memory right that's why they're called short-term memory networks and this is very good if you want to predict sequences right so time series machine translation bitcoin prices anything you want and uh talking about machine translation uh we have an open source project called sockeye that you'll find on github that uses an lstm architecture to to let you train machine learning models for machine translation okay and uh it's super simple to use and maybe we can do a really quick demo if i still have a come on i still have an ssh connection here so i took a data set for uh german to english and no oh yeah slow connection but we'll find okay so i have a few million sentences in english and in german and english and i trained a model from german to english it took a few hours on a couple of gpus so i could take some german sentences here and i could run them through that and i have just one for you here hopefully that works yeah okay see so if you want to build your own translation model just go grab a sockeye train for a few hours if you have the data set it's not complicated it's a pre-existing architecture and you can just do it like that okay so you can build your own translation service just like that okay all right i'm out of time so okay i'll take 30 seconds for the crazy stuff because i hate not talking about it and then i'm done so the last type of architecture i want to talk about is called gans generative adversarial networks and they they are weird right so who knows who these people are i know you're working hard but you gotta watch movies right come on tv or something no no one no one knows i i had nothing to win but okay if i had something to win i would give it away no no that's your last word okay now i know you're just very suspicious now right of this frenchman showing you weird stuff okay you're right to be suspicious because these people do not exist right they are generated faces and they're not copy paste right so it's not the nose of brad pitt and the ears of johnny depp or something like that it's really generated pixel by pixel by looking at a large number of images in in the celebrity data set that's why they end up being quite good looking the network learns to generate similar samples okay but these are completely fake right here's another example you remember that right when you were five-year-old or if you have kids you have stuff like this on your fridge right i'm sure i do so these are cars right this is the road these are trees etc and this is called a semantic map okay and again you can by training a neural network on images and the corresponding semantic map you can then predict this semantic map okay so you give this to the model and the model will generate the actual picture okay and this is hd right now these are hd pictures so you can zoom in and you can see a lot of detail actually right these are really really precise okay so this and this gun area of research is moving all the time it's it's really crazy and and if you're interested you know i would suggest you read about that you'll see some even crazier applications of guns right generating a new reality right so um how do you get started with all of this um well i would recommend obviously taking a look at uh the the adobe's website where you will find information on all the tools and all the libraries that our customers use to build a pretty cool stuff okay the blog has some technical articles etc if you're curious about mxnet and gluon it's all on github right so just go and grab the sources grab the tutorials there's lots of documentation for it and you can replay a lot of those examples if you uh if you're already doing machine learning in production and you have scaling problems right we have a service called sagemaker that lets you train pretty much anything tensorflow mxnet pi torch scikit-learn we have some built-in algos as well to save you more time so if you just want to focus on machine learning and not on infrastructure i would recommend taking a look at sagemaker we have sdks and we have a very very cool integration with spark if you use spark sagemaker makes a lot of sense as well and finally uh here's my blog on medium where you'll find lots of articles on machine learning deep learning sage maker and more crazy stuff i have a by now i've got quite a collection of talks on youtube as well so if you want to dive deeper on on those topics or or explore what aws has to offer you'll find something here and the code that i use today and much more is available on gitlab so just go and grab that and you can run these examples and and more all right okay i'm done thank you very much if you want to stay in touch thank you [Applause]