Neural Network Representations (C1W3L02)

Key Takeaways

see me draw a few pictures of your neural network in this video we'll talk about exactly what those pictures means in other words actly what those little neural networks have been drawing on represent and we'll starts were focusing on the case of neural networks with what's called a single hidden layer she is a picture of a neural net let's give different parts of these pictures some names we have the input features x1 x2 x3 Stata vertically and this is called the input layer of the neural network so maybe not surprisingly this contains the inputs to the neural network then there's another layer of circles and this is called a hidden layer of the neural network I'll come back in a second to say what the word hidden means but the final layer here is formed by in this case just one note and this single note layer is called the output layer and it's responsible for generating the predicted value Y hat in a neural network the you train with supervised learning the training set contains values of the inputs X as well as the target outputs Y so the term hidden layer refers to the fact that in the training set the true values for these nodes in the middle are not observed that is you don't see what they should be in the training set you see what the inputs are you see what the output should be for the things in the hidden layer are not seen in the training set so that kind of explains the name hidden layer just means you don't see it in the training set let's introduce a bit more notation whereas previously we were using the vector X to denote the input features an alternative notation for the values of the info features will be a superscript square bracket 0 and the term a also stands for activations and it refers to the values that different layers of the neural network are passing on to the subsequent layers so the input layer passes on the value X to the hidden layer so we're going to call that code the activations of the info layer in superscript 0 the next layer the hidden layer will in turn generate some set of activations which I'm going to write as a superscript square bracket 1 so particular this first units or this first node will generate a value a superscript square bracket 1 subscript 1 this second node we generate the value now with a subscript 2 and so on and so a superscript square bracket 1 this is a four dimensional vector or if you want in Python occurs this period or by one matrix or four column vector which looks like this and it's four dimensional because in this case we have four nodes or four units or four hidden units in this hidden layer then finally the output layer will generate some value a two which is just a real number and so Y hat is going to take on the value of a 2 so this is analogous to how in logistic regression we have Y hat equals a and prolific regression which we only had that one output layer so we didn't use the superscript square brackets but within your network we're now going to use this superscript square bracket to explicitly indicate which layer it came from one funny thing about notational conventions in your networks is that this network that you're seeing here is called a two layer on your network and the reason is that when we come layers in your networks we don't count the input layer so the hidden layer is layer 1 and your output layer is layer to another notational convention we're calling the input layer layer 0 so technically maybe there are three layers in this view Network because the input layer in the hidden layer and the open layer but in conventional users if you research papers m and L so in the course you see people refer to this particular neural network as a two layer neural network because we don't count the input layer as a as an official layer finally something that we'll get to later is that the hidden layer and the output layers will have parameters associated with it so the hidden layer will have associated with their parameters W and B and going to write superscript square bracket 1 to indicate that these are parameters associated with layer 1 with a hidden layer we'll see later that W will be a or by 3 matrix and B will be a 4 by 1 vector in this example where the first coordinate 4 comes in the fact we have four nodes or four hidden units and lair and three comes from the fact that we have three input features we'll talk later about the dimensions of these matrices and it might make more sense at that time but in similarly the open layer as associated with it also parameters W superscript square bracket 2 and B superscript square bracket 2 and it turns out the dimensions of these are one by four and one by one and this one by fours because the hidden there has four hidden units the output layer has just one unit on bergame we'll go over the dimensions of these matrices and vectors in a data video so you've just seen what a two layer neural network looks like that is a neural network with one hidden layer in the next video let's go deeper into exactly what this new network is computing that is how this neural network inputs X and goes all the way to computing this output Y hat