Applied Data Science With Python Full Course 2026 | Applied Data Science With Python | Simplilearn

Key Takeaways

What if one programming language could help you handle data, build structures for analysis, create visualizations and make sense of real world information all in one workflow? That is exactly what today's session is all about. Hello everyone and welcome back to this applied data science with Python session. In this course, we're not just learning Python as a programming language. We will be learning how to use it as a practical tool for data science, data analysis, real-time decision- making. And in the earlier sessions, we explored the core building blocks of this journey. We started with Python where we understood homogeneous arrays, n dimensional structures, vectorization, broadcasting, array attributes, and why numpy is faster than lists for numerical computation. Then we moved to pandas where we learned about series data frame labeled indexing, positional indexing, slicing and different ways to create and access tabular data. We also discussed why data science matters in the first place. Data is everywhere in business, healthcare, finance, smart devices, online platforms and everyday activity. The real value comes from transforming raw data into useful insights, identifying patterns and using those insights to make better decisions. In today's session, we will continue that journey with practical focus on data structures, indexing, data frame operations and data visualization concepts. We will also see how Python libraries help us work with structured and unstructured data more efficiently. By the end of this video, you will have a clearer understanding of how Python supports data science workflows and why these concepts are essential for analysis, visualization, and model preparation. Also, if you're interested in boosting your career in business analysis, do not forget to check out our AI powered business analyst course. So, this course is perfect for professionals looking to enhance their skills with the latest tools like PowerBI, Excel, SQL, all while gaining hands-on experience with real world projects. So, you will also learn how to leverage generative AI for smarter and faster decision- making. Our program is IIBA, Babok, V3 align and helps you prepare for certifications like CBAP and CCBA. You will also engage with 10 plus industry projects, 40 plus practical activities and benefit from live online sessions led by experts. Plus, with simply learn job assist, you will get the support that you need to land your next big role. Before we move ahead, here is a quick quiz question. Which pandas feature is used for label based indexing? Is it a ilocc describe? Comment your answers below. >> All right. So today we will start with what data science is, the basics of data science, what all we need to cover practically. We will be starting working with the advanced libraries of Python that is numpy. Uh you know where we concept on numerical Python then we move on to the pandas library. Then we will move on to data visualization library where we would be covering math plot lib seon as well as the plotly library. Another important aspect of data science is that no data science is complete without the concepts of statistics and maths. Not very deep concepts of maths but definitely some concepts of statistics as part of data science. Then we will move ahead and try to understand what probability distribution is. How they really help in analysis of the data. Move on to the advanced statistics part of it which refers to inferial statistics, data wrangling and feature engineering. These two are the last points of this particular course where we would be practic working a lot of data sense how we clean it, refine it and prepare it finally to be built to be used into the model. So we would be practically covering and dealing with lot of realtime data like data amputation, scaling, bin and grouping oper operations. Yes learner are you all getting this point or not? Right. So these are the course components. So that is why again I am requesting you and suggesting you to go through your basic Python because that would be required and as already discussed by the LSM that you already have your LMS the learning management system where you get your hands-on exercises to practice and gain knowledge course and projects as well as ebooks uh for reference. So now again the question comes that what is data? Yeah. Can you tell me what is data? What have you understood? Data science consists of the term data and science. So let's try to understand these two terms that what does data separately mean and what does science separately mean? Collection of information is data. Is this correct? Another important question is that do you think that data only exists in today in this today today's era or data has has always existed for the human mankind. Next question is do you think that data exists only for today or it always existed? How did it exist Ishan? How did it exist? How did it exist? So you just think of ancient man how did the ancient man u you know store data when it start communicating accounts and banking system we still have accounts and banking system I'm talking about you know uh you know u data existing since the evolution of human mankind you know so when it was ancient man you know where where we never even had houses. We used to kill animals, you know, cook on fire and everything. And then we started storing information. Some kind of language of de developed in terms of sign language symbols and everything was written on maybe on a bark or leaves or on stones. Right. Right. We have always been storing the data or whatever is preferable to us. Whatever we want to store, we have always been storing those in terms of walls or caves on the carvings of cave. Do we see that? And as human uh you know civilization developed right the civilized houses more uh you know like we understand harapa civilization, indis valley civilizations right we started living in houses with other things. So the the data also started getting evolved our education system getting evolved and now we started writing on the paper you know keeping all the accounts your name records started keeping we started keeping into files right we've always kept that data in terms of files yes or no right and after the evolution of computers that's around you know the 1990s and 90s you that the data started getting stored in terms of our computer system and of course with the evolution of internet the whole world now converges. So since it has been 60 uh you know se 7 8 decades to this whole uh revolution. So we only understand digital data right today the data that we talk about about analysis of the data or the science related to it. That data we understand it is in digital in form of digits stored in terms of memory. That is why you see a big you know noise now about battery systems, semiconductor devices you know because these are the lifelines. If this hardware is not there then the hard then the you know data also does not make sense. You know you you you might have heard about this in the AI summit that people are talking about data centers and the energy and the power to need uh to store that data right that is why there is this big buzz because now data is the new oil of the whole world yes do you all agree whoever has power to the data whether it's China America you know everybody wants to be a big player that is where the you know that uh country or that will be the ruler of the whole world that will have the whole power right or wrong. Are you all getting this point? Right? Since you have been born in this right so you only understand digital data that is why it was important for me to make you understand you know it has always been part of our civilization right but today we are living in the world of digital data whether it's in mobile phones whether it's internet I mean without internet there is no lifeline we cannot even breathe without our mobile phone you know any payment that we want to do anything that we want to study anything that we want to So you know without mobile phone we are helpless. Agreed? Do you all agree? So when we talk about data data is nothing but facts and figures, observations, measurements that we take and now we want to do analysis on this data. Why do we want to do analysis of this data now? Right? The numbers, the factments. Tell me why. Why do we want to do the analysis of this data quickly? Why is there so much of buzz around this data to fetch meaningful information from raw data? So why do we want this meaningful information? Webhuff, what do we want to do with this meaningful information? Come on. Come on. 100 plus. Come on. NMIT. Come on. Give me some answers. Quick answers. M A E. Okay. Make decisions. Okay. So it helps us to take decisions, right? The company take good decisions, right? Whether we should invest in that particular product or not. We should invest for that advertisement or not. And that's it. So the whole world is running on the pro profit, right? Suchin is basically to recognize patterns. So what do we do by recognizing patterns? Agreed? We recognize patterns. So what do we do by recognizing patterns in the data? Tell me like to you know maybe report about your medical now do I see AI and now let me ask you a very basic question I'm not getting response from you all let me be very very clear with you do you see AI and data science around you some practical applications yes to build the model and to automate task that's the new thing that AI has been doing dash absolutely correct but the question is why do we want to do it dan so patterns helps to visualize easily instead of okay they help to but why do we want to why do we want to analyze it why do we want it to automate task what is the usage that we see around us do we see some AI data science application running around us do we see that yes we see so judge come on tell me what have you observed to ease our lives in terms of what give me an application come on nobody is able to give me an application where this data science AI is getting used come on it's part and parcel of everybody's life now no practical application creating PP in minutes it's absolutely correct so we have tools which Create PPS in split of a second. Absolutely correct, Darian. Good. Very good. What about others? Nobody else is using AI data science. Pattern analysis helps us to find out where exactly we are lacking in where which lacking such a it's a very very general statement. I want a particular application related to that. It would be better to if you can answer from that level. Build apps with AI agents. Okay. Maybe in terms of sales. So give me an example that how it will help me uh in my sales to understand the pattern. Yes, it it does that. It does that. Absolutely correct. What about others? We don't see so many applications around us. Don't we see recommener systems around us? Don't we see in health care systems, in pharmacy, in agriculture when we move on to uh you know booking up of our airline tickets or even if uh you know we are traveling uh with a Google map do we see some intelligence some data science happening automated mails traveling um apps how is the traveling apps helping you mana ready good gooders so what is science What does the term science mean? Recommendations is online platform. Yes. So what? AI and data science is a part of it. Right? So what? So what? Now I want everybody to tell me what does the term science mean? Science means that something that we are observing and based on certain facts, observations, facts, we are going to trying to conclude uh you know some conclusions right study about something. It's not something but something like you know a plant requires uh you know sunlight or photosynthesis requires sunlight, air and water. Then we perform certain experiments on it. And then we conclude that if I don't give if I keep a plant in dark for five seven days, it will you know start uh drying up. So that concludes that plants definitely require sunlight for photosynthesis and if I don't water plants that is how it is going to be done. So we prove the observations through the experiments. Right. Sahana Ishan Nagendra Pranav. Right. Right that point is getting clear. So in this whole journey of data science where we will try to analyze data and conclude uh you know the analysis based on certain facts. It's not that you know I am concluding something and it would be accepted. No the conclusion about will be based on certain facts and proofs. Clear? Right. And if I technically talk about data, please learners concentrate here. If I technically talk about data, data is divided into categorical and numerical data. When I say categorical data, what is your marital status? Are you married? Are you single? Are you divorced? Right? Are you widowed? So that's the marital status. What is the political party you are working on? Are you for the Democratic, Republican, Congress, you know, different political parties? What is the eye color? Do you have green eyes, blue eyes, black, black, brown? What kind of eye? So, different categorical data. And the other discrete values. How many number of children are playing in the garden? Of course, there cannot be 2.5 or 3.76 children playing. So, it has to be discrete value. 0 1 2 3 4 defects uh per hour. You know basically the items which can be counted and continuous can be any ranging from minus infinity to plus infinity. The weight can be 67.763 to 88.8123. Getting my point? Is this point getting clear to everybody? Learners, are you there? Right now I'm going to give you one practical example where you will understand the use of data science completely. So data science for business is also known as the DI KW model where D stands for data, I stands for information, K stands for knowledge and W stands for wisdom. Now what does each term mean? Please try to understand. So data is nothing but raw facts and numbers. For example, if I'm talking about sales of a car, right? So that is the data for a particular company. When I process that data, that becomes my information. For example, what has been the maximum and the minimum sales in the last one year. So maybe the dark blue points are referring to the maximum sales and the green one is referring to the minimum sales no or the months which has sales has happened. Getting my point? So again I'm repeating data is facts and raw facts and figures information is processed data. Now knowledge means something which has been occurred over the year years you know not based on one particular data but over so now I have data for the past five years you know and now I'm looking at the data for the past five years and what do I observe that maximum sales happen during in the month of my festivities maybe during my navatras or during the financial year of end this is what my observation is and minimum sales happens in the month of June and July clear. So this is the insight or the pattern that I have analyzed and based on these insights I will have a wisdom that definitely you know now what we can do is that we can you know now we can give observation to the you know sales team or the marketing team that you know maybe we can have more offers during the navatras or the festival times Christmas times right so that our sales can have can have more increase or bumper offers so that it can actually impact our business. Getting my point? So are you now understanding how data will actually have the impact or the you know what you were saying were all all of you were correct but you were not able to relate the terms. So how actual data really has impact on the business by understanding analyzing the patterns right and this is known as the DI KW model clear one more example one more practical example from the New York Times story that you know Hurricane Francis you know we we see these things happening around us hurricane Francis was on its way on the Florida Atlantic coast. On Florida, there was a hurricane which was about to hit you know the Florida and it was Linda M. Dilman, Walmart's CIO, the chief information officer, she thought that, you know, one of the hurricane had struck, hurricane Charlie had struck several weeks earlier, right? So what did she think? She thought that, you know, let's try to analyze the data where hurricane Charlie struck several weeks before and try to understand the history of the shoppers, right? because we want a good impact of on our sales, right? So, what do you think? You know, now these trends are quite popular that a particular hurricane is about to hit. What would be the items that you would like to buy and keep it for yourself? Like when you move to any Walmart or any kind of you know smart bazaar or tea, what would you like to buy from there? And you know if I say you know uh some natural disaster is about to occur. How do you want to go about it? What do you want all you want to buy? Tell me some kind of you know remember the covid times what all everybody was buying. Everybody wants to buy basic groceries, fruits, vegetables so that they have sufficient to eat, right? The water bottles, the medicines. Yes or no? Right. Right. So you know there that could have been and maybe you know this is a little older times uh you know you know story 2004. So maybe maybe people would like to watch some kind of a movie CD which was more hit and they would like to buy it that buy more of such things right maybe a flashlight or even a power bank would work. Surprise everybody. The expert the data and found that the stores would indeed need certain products and not just the usual flat flashlights. And from the past it the the top selling item was prop strawberry pop-tarts increase in sales like seven times their normal sales rate ahead of the hurricane. So, Miss Dilman confessed in an interview that the prehurricane top selling item was a beer. Getting my point? Are you now understanding the power of data and its analysis is impacting today's business? Yes, learners. Are you understanding this point or not? So now moving ahead with the lesson number two in the LMS. So you all have your LMS your LSM guided uh to you? Yes or no? Everybody has the material to access learners. Yes or no? Right. So lesson number two is all about basics of data science. So what is data science? What have you understood? Data science is a multi-disciplinary field that uses scientific methods, right? So we would be definitely use using certain statistics certain stat uh you know scientific research methods to conclude about our results processes algorithm systems to defi uh you know to derive meaningful insights and from structured as well as unstructured data. Do we understand the term structured and unstructured data? Structured data means something more related to in the form of table, Excel file maybe or you know SQL that you have done. Unstructured data is in terms of audios, videos, log files. Getting my point? So that is where this is the power of data science that will help us to give conclusion. So using a search engine or making purchase on Amazon provides valuable data to data science-driven software system operating in the background. So data science emerges from a combination of expertise scientific methodologies and technology. It's not only about data but it is also about mathematical and statistics model that we would be covering up in this course. how we would be using it to analyze it and different scientific tools and methods. So there are lot of tools available different libraries that are available you know but in this particular course we are trying to be using the Python language to do the analysis or uh you know to find the conclusions from the data. Clear? Is this point getting clear to everybody? So one of the applications of data science in healthcare which nobody was able to tell me is that we all are using smartwatches. Is the smartwatch capable of analyzing my BP, the heart rate, right? What kind of stress I have? The number of counts I have done. Then it also advises you advises you to go and take an activity, drink some water. It's been a long time. Do we see that you know things happening through the smartwatch? Do we see that? Can I get a response from everybody? Yes or no? Something which is practically that we see around which we are using. Kishan, Kushi, are we using it or not? Pragna, Gesh, Puja, Lokesh, Sachin, Sahana, Sahil, Shrian, Sedhar, Susti, Shri, Shria, Tanisha, are you all there? Gaga, Nishan? No. Learners are you there or not in the session? Amita, Diksha, Daran, Dia. Yeah. So if we are wearing that smart watch right and we transfer the data the data gets transferred to the different servers and it is the enterprise server which analyzes the data and you know tells me you know what kind of activity I have been doing along my month you know or maybe I have to focus more on my cardio drinking more water so that I can take more informed decision if I am looking for a weight loss if I want to have a better health care system right? So we can analyze the data our own data how we are living what kind of water how much water are we drinking how much steps we are walking based on that certain decisions can be taken Google also employs data science to offer relevant search recommendations as users as they type in their query so it gives the real time the moment you're typing it you know predicts the next word you know that what do you want to search that data science in health or in healthcare course or a healthcare research paper etc. Similar finance domain using data science can help a loan manager can easily access you know whether you are you know applicable for a loan or not. So once you have applied for the loan in the port portal like what all is important right what is your salary how many dependence you have do you have a medical insurance or not what are the different properties based on those kinds of analysis what kind of you know your civil uh report is credit report approved amount risk based on those criteria decisions can be taken whether you will be approved for load alone or not. Clear users? Are we understanding the applications of data science? Have we understood what data science is? So the data science process what what all is required. First thing as I told you is problem definition for different projects for different domains for different data. What are we looking at? What are we looking at? the loan application part of the uh person or what are we looking at the health care part of the person. So the problem needs to be defined based on the pro data authentic data needs to be collected from the correct resources. Right? If you want to know how many people have got you know negative side effects after the COVID vaccine we have to collect that reliable authentic data. Once the data is collected data cleaning and exploration is required. Now what are the issues in the data? Can anybody tell me since you've done SQL what are the problems uh you know that a data can face? Can anybody tell me? Come on tell me technically missing data. Absolutely correct Sahana. Inconsistencies in data very good mira. The null values in data right errors in data duplicate val data. Absolutely correct. So all that needs to be cleaned and explored before we send it for further analysis. What will happen if we are not cleaning this data? Dup Sahana if you're not removing the duplicates or null values. What will happen? Mira Sahana what will happen? Normalization is a little different. If you're not cleaning the data what will happen dan? Inaccurate results. Right? So that is a very very important step in data science. And once done we then move on to the feature engineering part of it. What does what does feature engineering do? That uh if any kind of you know especially categorical data needs to be converted into numerical data. Scaling needs to be done. Bin needs to be done. Removing of the outliers. So this is till here we are going to complete our data science journey. Right? This course you know deals till this particular part of the whole data science process right in the next step as you will move forward in machine learning model building and training will be covered or the deep learning models training will be covered. Finally we can move ahead with model evaluation and the model deployment. Getting my point? Is this point getting clear to everybody? Right. So the first step in the data science process is clearly define the goal or the question to be addressed through the data analysis. Gather the relevant data set information from different sources. Pre-process handling missing values outliers. Then we need to uh transform the new features to enhance the data sets uh information model building and training. And finally model evaluation and deployment also needs to be done. Clear? And here in this particular course we are going to do understand data science from the Python perspective. And can anybody tell me why is Python a hit language? Anybody who can tell me why Python is a hit language in today's world. Come on, we have already done that. Why is Python a hit language in today's world? Is it easy to understand? It's a high level language. It's interpreted language. Huge collection and libraries for data science and machine learning. Very good. Very good Abija. Very good Sahana. What about others? And as we have seen it has multiple open-source packages that we are going to study that is the numpy, the pandas for data cleaning, exploration as well as visualization. Clear? And one more feature which makes Python a big hit in the market. I discussed it in the beginning. You all forget that feature. It supports functional programming. Please always remember this point. And what is functional programming? Ishan. Good. Good. What is functional programming? What is functional programming? Functions can be treated as objects, right? They can be passed as parameters to the functions also. Yeah. So we all are clear. None of you are answering. Most of you rather I would say are not answering. So try to understand the advantages of Python for data science that it is an opensource. It's an interpreted high-level language that supports objectoriented programming. Ease of use and simple syntax. Scalability when compared to R. Availability of the wide variety of data science libraries and packages. Compatibility with all major operating systems. Creation of new data science libraries daily by vast number of online user communities and powerful visualization libraries are there. Right. And the different Python packages for data science that we would be covering in this particular courses. First we will concentrate on the NumPy library which is used for scientific computing that supports large multi-dimensional arrays matrices and includes comprehensive mathematical library. Second is we are going to cover pandas which is efficient storage and manipulation of structured data such as the time series and tables. Sci sci is an open-source library built on top of numpy and is used for scientific formula that also we would be covering up in this particular course. Stats model also we are going to cover is a Python module that provides classes and functions for estimating many different statistical models conducting statistical exploration and scikit learn. Scikit learn we will cover a little bit part of it not much because it's part mostly used for machine learning. So, scikit is widely used for open-source machine learning library for Python. Known for its simplicity, ease of use, versatility in handling various machine learning task. It identifies objects and images for autonomous vehicles and facial recognition systems. It detects fraudulent transactions in banking and ecom e-commerce platform. It analyzes customer reviews for sentiment classification in marketing and social media analysis. Clear learners are you there with me? And if I talk about Mattplot Lib, Mattplot Lib is a library. It is a comprehensive tool for building statistic statist static animated and interactive visualization. We would be covering up mattplot lib in detail as we move along this journey to build different scatter plots, bar charts, histograms, pie charts. Advanced library of a mattplot lip. Seaborn is another data visualization library in Python that is built on top of mattplot. It provides a highlevel interface for creative attractive and informative statistical graphics like histograms, box plot, violent plots, etc. Then we would be covering up plotly which is used for creating interactive publication quality graphs and visualization. It is suitable for web applications also clear. So now let's do a quick recap of the different plots that you might know before. For example, let's first understand the line plot. Yes, learners. What is the use of line plots? I'm sure you must have all done it in your school days, in your mathematics, in your graphs, you know, uh maybe science, physics or math, statistics. So, can anybody tell me what is the use of line plot? Yes, learners, where do you use it? Do we see around these graphs? Now, come on. Sahana, Ishan, Nagendra. Answer. Yeah. Do we see line plots around us? Of course they are straight lines which show relationship even over two quantities comparing growth. So this is like x-axis. This is my y-axis. What kind of growth? good f what kind of growth for X and Y axis especially if you are dealing with stock markets or anything which has been over a period of time the sales of car over a period of time or the sales for any monthly sales of fruits or vegetables in terms of economy very good fell anything else do we see now graphs on the temperature also something like temperature also what will be the temperature in the next 10 days monthly basis do we see that first in weather applications that's also very common do we do we do we do that also we are always worried what's going to be the temperature tomorrow it's going to be really hot cold rainy do we see that around us or not practically Come on learners, do we see that or not? Are you not using weather apps uh for uh things? Where come on, tell me. Tell me learners. No, nobody wants to answer. Nobody's using it. Sahana. So, a line plot displays data points connected by straight lines often used to visualize trends or relationship between two variables over time and other continuous intervals. Agreed learners? Then a little uh you know appearance making the line plots better using the marker plots. What is this marker plot? It displays the data points with markers useful for scatter plots for visualizing individual data. If I really want to find out the relationship at this particular point, do we do we also try to draw the graphs like of your marks? Do we do we do that also? So Faz has changed his name to Trump. That's interesting. That's interesting. faculties your students are changing changing their names. So fairs has hires has changed his uh uh you know name to Trump. Okay. Okay. That's interesting. Come on. So a marker plot is something which can you know use to highlight certain points you know maybe even it could be the marks or or a particular specific uh you know condition in in the graph. Then scatter plots. Where do you see the scatter plots which gives the relationship between two numerical values? Come on learners in healthcare data maybe the BP and the weight people who are more with the weightage have more BP issues or you know two numerical values can be used to compare using the scatter plots right height and weight in a particular population then have you seen some graphs like this such as area plots which help us to give the cumulative data. If this is the sales of the company, maybe the blue one is giving the sales for the first quarter, this is for the second quarter, this is for the third quarter. So an area plot visualizes the cumulative data changes over time such as tracking the total sales revenue over successive quarters. So where we tried or maybe you know if I want to analyze or give you a report card so what was your result semester wise what was your result in your first semester second semester third semester cumulative results can be easily represented using the area plots bar plots do we see bar plots I think so that's very very common that we see around us do we see bar plots learners nothing but rectangular graph that show vertical and horizontal data based on another axis. It which gives the comparisons of the sales of different products over the month. It could be different uh you know we see it as uh you know students as display student grades in different subjects plots that you might have been using in your science or statistics classes to draw to see the downfall or rise and other things to mark certain particular uh points. Yeah. So grid plots basically help in comparison of multiple plots by enhancing the visualization points. Another very very important graph is a histogram. Anybody who's aware about histogram? Yes learners, can you tell me what is the use of histograms? This is the graph which we are going to do a lot. Very very important graph in terms of data science. Let me tell you histograms in this in which we divide the data into different class intervals and then draw the uh you know different values coming in that particular in interval. So we say that histogram displays the distribution of data by dividing the values into bars and representing the frequency of each bin with the bars. Right? So histogram visualize the distribution of numerical data like income levels or exam scores. They help make inferences about data characteristic, underlying patterns, guiding decision making process etc. Yeah. Last but not the least, one of the favorite graphs which we all have deal with is a pie chart where the whole data is considered as 100% and a part of it of fraction is you know shown in different parts. clear right it could be the you know the share of languages in a part uh in the whole 100% area or the uh you know the different uh expenditure of the house in different areas that can be taken. So pie plots show the proportions of the whole like the market shares or the survey responses right nothing to worry we will be dealing up with all these charts in detail when we take up data visualization clear right so a quick recap of what we have done data science involves analysis and interpretation of data to generate actionable insights numpy is an open opensource library predominantly used when working with arrays. Seaborn is a data visualization library in Python that is built on top of mattplot and Python is a preferred programming language for data science projects across the industry. List one and what is this concept known as? Can anybody tell me known as? Yes. web hub. This is known as list comprehension. Is any other way we can achieve this without list comprehension? A quick code without can I achieve multiplying it by three without u without using this list comprehension quickly? Can I get an answer? Yes, learners without list comprehension. Can I get a quick answer? Without using this concept, I have to create an empty set and explicitly run a for loop. Anybody who can give me that solution quickly? Yes or no? Quickly. Explicitly we can. Who's who's going to give me I want an explicit solution quickly. I'm waiting for the result. I'm explicitly waiting for the result. How can we do it explicitly? Come on others. As Ian, can you give me the result? Good. Abijita. Good. So let me quickly do this. Abijita I don't need to give this list. I'm going to keep this uh thing. I don't even want to give m is equal to three. Do I need to give the range as length? Do I need to give it as range as length? Abijita? No. Let me copy your code first. Let me copy your code. No, not really. Your code is little little wrong in that sense. It's giving me 30 60 90 and 120. But if we look at this, I don't require these two things, right? You kept it as variable. But why range? Why range you want to do it? Okay, you're keeping it as list of I whatever is the value at zero 1 2 3. Okay, clear. Okay, fine. Let me put it as three. And this is my answer. So all right. So now when we did this example, what was the flaw that you saw in the list and tpple or dictionaries especially the list which is a mutable data type. What was the flaw that you found out that when I want to do perform a mathematical operation that when I want to multiply the elements of the list with a numerical value I have to you know imply the for loop right without that it is not capable of multiplying each of the elements. Do you see that flaw? Yes or no in the list? Yes or no? Asha, Anusha, Avira, Shanmok, Yadav, Podar. So that is why the Numpy library advanced library was being introduced. Numpy is a numeric Python library for computation of on homogeneous and this is the power that this library has which helps us to create arrays arrays not in one dimensional twodimensional but in n dimensional to store the data. So let me quickly give you what this particular package looks like or library rather looks like. Since NumPy is an open-source material, let me share it with you. Yeah. Do you see this? Everybody please open this package which is available online. Right. This is the you know the numpy 2.4.0 release. It's a it helps us to create n dimensional array computing tools opensource library which helps us this is the you know uh the material where you can you know practically use it. It helps us in you know working in scientific domains array libraries in data science. It helps to extract uh transform and load the data, exploratory analysis, model and evaluate, report and dashboard. All these things can be done with the help of machine learning, visualization, scientific domains, array libraries, etc. Clear? Lot of lot of practical use. All the tensorflow, all the deep learning activities, unstructured data is all stored in terms of nd dimensional array. Clear? Are we all getting this? Yes, learners. Yes or no? So, what is the use of numpy? Numpy is nothing but a numeric Python. It is a package used for computation on homogeneous and n dimensional arrays. Right? What are the properties of array? They are mutable. What does the term mutable mean? We can and which is the other data structure which are mutable. Other data structures in Python which are mutable list dictionary and which is immutable? Which are the data structures which are immutable? Touples are immutable. Strings are immutable. Int, float, all are immutable. Right? Very good. Good. What does the term homogeneous mean? What does the term homogeneous mean when I say the term homogeneous? Yeah, they all are of same data type. Secondly, it can be accessed using integer position, right? Integer position. Strings are accessed using integer position. Lists are accessed. That is in other words, indexing is allowed. What are the two types of indexing available in Python? What are the two types of indexing available in Python? Very good. Positive and negative. Arrays deal with numeric data and high performance in calculation. Right? So what is the advantage of arrays over the list? Right? We just now saw that in list basically we store pointer to the data location. So every data is stored in a different location. Therefore accessing the elements of the list are different. And when I talk about arrays they are stored in contiguous memory location. First of all, they are of homogeneous data type. Int will store some bytes of information. Float will store some bytes of information. So they all are stored in contiguous memory location. That is why arrays are faster than list. That's why this data structure was created because it is much more faster than the list. Clear? Is this point getting clear to everybody? And now let's understand the code and then you will start writing the code. I want everybody to open their Jupiter labs and start writing this particular code. I will ask anybody to show the file. I can randomly ask anybody to share your screen. I want everybody to start typing and writing this code and let's understand this code side by side. Learners please start typing this code on the lab Jupiter lab that you have. The first and the foremost thing that is required for this particular course is import the numpy as np that we import the library. The first step whenever we need to create uh you know arises because it's not part of the basic python we will always need to import numpy as np. I want everybody to type in the code. I want faculties please look at the uh learners students are they doing it in the lab or not this particular assignment or not and this is how we create the array. np do array is the name of the function and these are the elements to be created in the array that is 0 1 2 3. Yeah. So when I print a this is the output I have get and when I do a type A it seems like a list but it is not a list because it's an object of class numpy dot nd array. So by doing type a I get the output as type numpy dot nd array. Now a dot end these are nothing but the attributes of the array. A dot endame gives me the dimension of the array. This is a onedimensional array. The shape the number of elements or the number of you know uh rows basically it's a column vector. It is written like this. It gets stored like this 0 1 2 3. So the number of rows over here is four and length gives me the number of elements in the array. Everybody got this? Again I'll repeat first we need to import the library. Whenever we have to create arrays once once you have loaded the file then you will load it not every time you write the code. Then by using the np dot array function you are able to create arrays. Print the array. Give the type a that is class numpy dot nd array. a dot endm gives me the dimension of the array. a do.shape gives me the number of elements and length do a gives me the length of the array. Clear? shape always gives me the rows and column. Clear? Everybody has done it. Anybody who's facing any doubt, any questions on that? Come on tell me. Dhanosh Manoha ready Chetan Kumar Kana Bhika Fairs. How many times fairs has joined? Oh, so Malika Arjun is here. I was looking for you. Yes, Malika Arjun are you able to run the code? Risha, Karan, Kishan, Kohli, Kushi, Palak, Pandit, Mokshhata, Raksham, Magna, Shifan, Jum, Saswat, Sahil, Shrea, Shashank Pay, so many Sags are there. Webhab, Versa, GB, Balvik, Hush Pranit, Chetra, Bhavisha Chandra. Are you all there? Have you been able to do it? Good. Now do the next next activity. Now do the next activity. Well, you have if you have imported this one, you don't need to write this code again. Now we had the range function in Python. Similarly in arrays we have a range function and what exactly the range function? It will create or have data from 0 to 39. Please type in this like we had the range function in basic Python. Similarly we have the a range function over here to create the data. Clear learners? Are you there? Now by using the shape can I change the 40 elements into a twodimensional array that is a matrix that is it now this array consists of five rows and eight columns. Do you see the output? Are you able to run the code? Give me a quick thumbs up. Who's able to get the output? Good. Good. Good. Others only two two out of 100 plus batch. Now you're 120 plus and not even 20 of you respond in this batch. Is it possible to change this to 8a 5? Is it possible to change it to 8a 5? Is it working? Give me a yes or a no. Is this 8a 5 working? Very good. Can I change it to 4a 10 also? Can I change it to 4a 10? That means the multiplication factor is again 40. But can I change it to 4a 4? Run it and then tell me. Run it and then tell me when ready. Akash run it and then tell me. No. Can I change it to 2a 4 comma 5? Can I change it to 2a 4a 5? Let me run the rub all this. Can I change it to 2a 4 comma? Why? Why is why nana? Why Abijita? Tell me why. Why can I change it to 2a 4a 5? Overall the factors or the multiplication is 40. 2 into 4 is 8. 8 into 5 is 40. So can I change the dimensions of the array? Good. Akash. Can I change the dimensions of the array? Can I change? So what do I mean by dimension? Everybody is done with the activity? Yes. So now please look here what do we mean by dimensions of array. A onedimensional array right will only have one dimension that is axis equal to zero and the shape is only having four elements or you can say it in terms of four columns right so the shape the axis and the dimensions are very strongly related with each other a one-dimensional array will only have one axis and the shape will also O give about one dimension and a onedimensional array is known as a vector quantity. Please be clear with this concept. A onedimensional array is known as a vector quantity. Now when I talk about twodimensional array, it consists of rows and column. And a two-dimensional array is always known as a matrix, right? Agreed learners? And of course it's a matrix. It consists of two axis. Axis 0 refer to the rows. Axis one refers to the column. So the shape is 2, 3. I hope I'm going very very slow with you all giving you all the uh you know examples making uh you to do the practice on the lab. Still anybody having any difficulty please let me know. Yeah. Are we clear with one and two dimensional? Now when we look at threedimensional array, we it will obviously get consist of three axis. Axis one, axis 0, axis one and axis 2. Axis 0 refers to you can consider this as a slice of bread you know. So each slice so this particular array consists of four slices. In each slice we have three rows and two columns. Clear? And the three-dimensional and above are all known as nd dimensional array. Clear? Any questions? Any doubt? Tia learners. Yes. Kani Balv Havesha Chandra Vive Vinaya. Any questions? Any doubt? Versa, Vehav, Yaganpati, Samir, Sahil, Suhana, Sri Khan, Shusti, Sedh. Yeah. So, let's do a lot more practice on this. Yeah. So I want have you been able has everybody downloaded the material? Has everybody downloaded the material. I want everybody to open 3.01. I'm I'm sending it through the chat also. I want everybody to open this 3.01 in your Jupiter lab or whatever to tool you are using. Anybody who's facing difficulty to work on the lab, I was I was there with the lab and I've already opened 3.01 01. I'll be opening it in collab uh sorry in Anaconda because it's more clear. So anybody who's facing difficulty please let me know. You can share your screen. I can help you out with that. Excuse me. Excuse me. Are you ready with the file? Can I get a quick thumbs up? Thumbs down. Shrian, Shrian, Trusti, Siddhart, Harsha, Nagendra, Bhumika, Harsha, can I get a thumbs up, thumbs down? Good. Nigam, you're ready. Ashish, you are ready. Great. Na is also ready. Omega is also ready. What about others? Good Ashwaria. Good web of anybody who's getting an error when uh when uh one is running the uh importing the numpy file. Anybody who's getting an error? Good. Ganga, good KM. What about you? Tell me quickly anybody who's getting an error. So what is numpy? Now if I ask you see over here I've already given you the link of the numpy.org right? I've already shared this uh you know open-source link. Now, now if I ask you the question, what is numpy? Who's going to answer this? Come on, learners respond. Yes. What have you understood? What is numpy? Everybody answer. Come on. It's we've just done it. What about shashank? Shashank pi. Why are you not answering? Shifa sahil, come on. Prajhi answer Prakiad Pranav Pesh Sahana says open source library for mathematical operations okay but it's not a complete answer used for computational and what multi-dimensional what multi-dimensional abija it's again incomplete give me a good complete answer not break it give me a complete Answer who's going to give me a complete answer what is the use or what is numpy library give me a complete answer others can also respond mumika monika miraulu akash nagendrai nana raomesh come on answer no you don't want to respond Kushi Kiran Kishan Lakshmi Shria ready no you don't want nobody wants to answer nobody understands what is what is the use of numpy no versa so numpy is a library which helps us to create. Yes, it helps us to create arrays which help in faster mathematical numerical operations. Very good. Right. So, what are the properties of these arrays? Tell me three properties of these arrays. What are the three properties? I gave you a lot of things. So anybody who remembers the three properties of arrays they are mutable very good Hamsika first is mutable second homogeneous excellent and the third one can be accessed via indexing good excellent only Hamika was there fixed size not fixed size it's it's mutable Sana it's not fixed size it's mutable easy access in what terms We can say faster mathematical operation. Su Hana. No, both answers are wrong. Sana got it. Homogeneous mutable can be accessed via indexing faster mathematical operations. These are the properties. Clear? Jant Yadav, are you there in the session? Why are you not responding? Parnika, what about you? Okay. So what is the object called when arrays are created when uh in um numpy library? What are the object? What is the name of the object? Numpy creates array objects known as what are they known as? Arrays create in numpy which are known as nd dimensional objects. No they are not heap objects. Wrong wrong akash. They are known as nd arrays. Clear right. So what are the advantages? They are faster than the traditional Python list. We will just now see how it is faster. It provides supporting functions. Arrays are frequently used in data science and they are stored in continuous memory in memory unlike the list. Did we understand this point? list are stored in different memory location. Which is the Python function which is used to create arrays. Is the Python function which is used to create the numpy ND array object. It's the np array function. Good manoh. Good. Yeah. Learners, are you there with me with this code? Everybody has this code. I'm just making a few changes over here. So now do you see the first step in creating this thing is importing the numpy library. Then I have created this array using np array. Here my array gets printed. The type is that it belongs to the nd dimensional array object type. Please remember this point. Even a one-dimensional array is an object of nd array. Dimension over here is one. Tape the number of element is three. The length is three. And the data type is integer data type. Now what do we mean by homogeneous data type? Moment I add floating value. Please look here learners. Moment I add a floating value all my values become floating automatically and it gives me the data type as float. Clear? Are you all getting this point? Why these arrays are set to be homogeneous? So when I create this np array simply learn it's it's a string it's a array of string data type yes or no learners are you there with me let me put this over Learners, are you there with me? I'm making the changes over here. You can play around with it now to understand the concepts. So it gives me an error over here. So this is an array simply learn which gives me that it's a dimension. Dimension is zero. So the shape is also zero. Length it's unable to give me the length of it. It's an unsized object. So we can remove that. And the data type is uni code 11. So uni code 11 refers to the string data type. Is this point getting clear to everybody? I want everybody to practice this code. I'm sending on the chat also. Are you able to do it or not? pairs. Hamika, Harsha, Parika, are you able to do it or not? Jayati Yadav, Jita, Shiri, Maha Ready, Manoha ready, Dhanosh, Daran, Diksha, Rag, Anusha, Anvita, Azba, Aayush, Adita, Yadav, Ashwary Nha. Jayan are you facing difficulty? What about you Malika Arjun Sushmita? So here we have array one again it's a one-dimensional array and the data type is of float. it will make all the other data type homogeneous that even if you know I make one of them as um uh string data type okay you see all of the data type become string data type have you understood this homogeneous property of arrays now let's create arrays of zerodimensional arrays. So zerodimensional arrays are known as scalar values one value that means any integer float value they are nothing but zero dimensional arrays they are scalar values. Do you see this array is a vector twodimensional array is a matrix and a threedimensional array is n dimensional array. This is the code for zero dimensional array one value. You can give any values over here. That doesn't make only one particular value over here is allowed. Even if you put it as minus, maybe as negative, it will still run it. When I create onedimensional array, the number of elements become my shape. So four elements are there in this vector. Matrix. Now look at matrix. One small hint or tip. The number of square brackets starting gives me the dimension of the array. So two two square brackets. So it's a two-dimensional array. And look at the beauty of this particular library. Please look here. So this is my zerodimensional arrayed one-dimensional. And see it gives me my twodimensional array as a matrix. And what is the shape? The number of rows is two and the column is three. And what about threedimensional array? Please look here. These are the three square brackets. So this is my threedimensional brackets. So how many slices do I have? This is my first slice. This is my second slice. So the first dimension goes as two. Each slice has two rows in it and three columns. So therefore the shape of a three-dimensional array is 2a 2a 3. Again I'm repeating two is the number of matrices different matrices in the three-dimensional array. Each matrix has two rows and three columns. Now let's understand arrays versus list. So which one is a better better data structures? Arrays or list? Yes learners tell me which one is a better data structure arrays or list? Come on. Nobody's there with me. No, nobody is able to answer this question. Again, I'm repeating it. Which one is a better data structures? Arrays or list? Only Vehav says arrays. What about others? Why arrays? Tell me. Tell me, Nana. Why arrays? I'm say va tell me why arrays it's mutable. Even list are mutable. N that's not the correct answer. Even list are mutable. Even list are mutable. Why web? Tell me list is better. Versa why why list is better version why list is better tell me web huff tell me now let's look at practically let's look at practically look here learners what are we doing over here what is an underscore array Okay. What is this? And how many elements am I creating? What is this? An underscore array. Tell me by using the a range function. It will have elements from 0 to how many? If I look at over here 99 lakhs 99,999 agreed that means how many elements am I creating? 1 k 1 2 3 4 5 6 7 Yeah, nobody talks less than cors now. So this is one ko uh elements I've created in this array. Agreed? Similarly I've created in the list one k elements agreed nana vsha webhava hersa hamsika again gendra are you there or not? Okay, when I print an array, it gives me a beautiful answer, right? But when I try to print the list, it gives me that data rate exceeded. It doesn't give me the output. So why arrays? Because arrays are stored in homogeneous memory location. Homogeneous not homo we will not call it as homogeneous sorry we will call it as contin cont contin cont contin cont contin cont contin cont contin contcontin cont contin cont contin contcontin cont continuous not continuous the means the same but contiguous memory location now clear everybody is getting this error but somebody is not getting this error or this warning rather Now what am I trying to do in this particular code? Look here learners. What are we trying to do in this particular code? We're trying to calculate the time the CPU time taken by you know multiplying each element of the array with two but 10 times. So what answer am I getting? It takes 438 milliseconds and the same thing I want to do it with a list. Now look at over here the beauty I'm only using an asterisk symbol but here I have to use explicitly for loop for multiplication. So which one is faster arrays or list? Which one is faster? Now tell me Versa Web Abhijita tell me arrays are faster or lists are faster which one is taking more time or which one is taking lesser time? Which one is faster? Quickly, right? Arrays are faster to do mathematical operations. So if I want to do mathematical operations on the data that's why it's known as numpy. Do you understand? Generally we talk about numerical data. And to process or to do mathematical operations on this numerical data arrays are faster. Now clear to everybody the answer was not mutable. Got it Nana? That's not the reason but definitely arrays. Numpy arrays are faster than the list. All right. So we have this numpy. What is NDR? NDR is the name of the object which array gets created. The different attributes of arrays are shape. What does the shape give you? The number of rows and column of the array. A one dimen what is a onedimensional array known as? Yes, tell me. One dimensional array is known as vector. A two-dimensional array. A two-dimensional array is known as a matrix. Threedimensional array is known as n dimensional. Z dimensional array known as a scalar. Good. What does end give you? What does the end name give you? What does the nm give you? The dimensions. So, and what does the d type give you? What does the d type give you? Yeah. The data type and what does the item size give you? Good. Chetra, what does the item size give you? What does the item size give you? It gives you the number of bytes occupied by each element and size gives you the number of elements. Clear? Please try to understand. If this is my array, right? This is the matrix with 3, 5, right? The shape is number of rows are three, the columns are five, the dimension is two, right? The size, the number of elements in this matrix is 15. Data type, each data type is of int 32. What does int 32 mean? 32 mean, right? So n32 means that the 32 bits of memory are required to store the integer data type and item size all you in terms of bite. So 32 divided by 8 one by is equal to 8 bit. So the answer is equal to four four bytes. Clear? Any questions? any doubt till here right so now let's start working on 3.02 02 to open the next file. I want everybody to open the next file that is attributes and functions. Anybody who's facing difficulty in opening attributes and functions Are you all ready? Can I get a quick thumbs up? I will not be able to share this file web. I want everybody Okay, let me share it with you. I want everybody to now work and be ready with the files tomorrow. Will you be able to do that? Today I'm sharing it, but tomorrow you have to be ready. Will you be able to do that? I'm sharing the whole folder with you all. Can I do that? I'm sharing the whole folder. Web of Does that help? You should be ready with the file tomorrow. I want everybody to be ready with the file tomorrow. We've only started working on 3.01. 01 3.02 tomorrow we will be covering up 3.02 we'll be doing a part of it that's just the attribute we will cover 3.02 tomorrow again with functions then we will move on to 3.03 03 with numpy arithmetic statistical and string functions indexing and slicing of numpy. Clear? After completing numpy tomorrow then we will move on to the pandas data structure. Getting my point learners? Is this point getting clear to everybody? Any questions? Any doubt? Quick learners tell me let's begin with 3.02 are you ready now? So what are the different attributes of numpy array? data, item size, the shape, the n dimensions, the dimensions, right? Array size and the data type. So now if we look at the first example, please look here learners. If you look at the very first example, right? This is the library that we import. This is the array which I create. If you look at over here, there are two square brackets. So therefore, it gives me the dimension, right? It is a twodimensional array. This is how beautifully my array gets uh printed in terms of matrix. And now I can see it has the shape 2, 3 that is two rows and three columns. The number of dimensions is two. The size of the array, please look here. Array dot size gives me the number of elements. It is six. The array stores do d type that each data type is of integer 32. size of one array element is four and array's data what is the memory location at which the data is located and if I make certain changes with a string or with a float value we know that all the values will be considered as float so the number of dimensions is two the shape is again same size is also same but the size of the bite changes so the size of one array element is 128. Clear? And if I talk about the explanation of attributes, what is endm? Endm gives me the number of axis of the array. So arrays. So the shape as I told you the dimension is related with each other. If a particular array has two axis, it has two dimension. Shape provides the size of the array for each dimension. The output data type is generally in the form of tpple. If it consists of two rows, three column, the output will be 2, 3. If it has three rows, 2 column, output will be 3, 2. The size gives me the total number of elements in the array. So 2a 3 will give me six elements. And if this array has uh how many rows? 1 2 three rows and four columns. 3 into 4 12 elements data types gives me the data type of each of the arrays and item size shows the length of one array elements in terms of bytes. So y divided by 8 one byte is equal to 8 bit. So therefore we get the answer in bytes and then it is an attribute offering access to the raw memory of the numpy array. Clear? So a quick revision of numpy that we had done yesterday. So numpy or the numeric python is a package for computation on homogeneous uh arrays. What do we mean by the term homogeneous? What do we mean by the term homogeneous learners that basically they contain data of the same type n dimensional array. So what is the biggest advantage of arrays? You understood yesterday. We discussed that. What is the biggest advantage of arrays? They're faster than list. Right? They're definitely much more faster than list. And why are they faster if I ask you? Because the data is stored in contiguous memory location. Did I explain that point right Weber? So what are the properties of the numpy array? The properties of numpy arrays are that they are mutable. That means they can be changed. Homogeneous they are all of the same data type. Can be accessed using integer position. arrays deal with numeric data and it gives high performance in calculation. Clear? Are we clear with all these points learners? And this is what we were talking about. List stores the pointer to the data location. That is why list are slow as compared to arrays. arrays. The data are stored in continuous memory location. Clear, web, Ganga, Bumika, Nagendra, Rajual, Mira, right? And yesterday we had written codes to create array. So the first thing that's required is importing of the library, right? And then it is np.ray array function which is used to create the array. What are the different attributes of an array? Can anybody tell me what are the different attributes? End gives me the dimension of the array. Shape gives me the shape of the array. The number of rows and column of the arrays. Right? And we also have a a range function. Do do we understand this a range function which is exactly the same as the range function in Python? But the values lie between 0 to 39. And this is nothing but it refers to numbers of rows and column. Clear? Right? And as I told you the dimension, shape and axis are very closely related. Onedimensional array has access zero. The shape will have only one parameter. That's the number of elements. What is onedimensional array known as? What is onedimensional array known as? Very good. That's known as a vector. Only web understands in this session. Only web understands. He he is only going to respond. Nobody else. Then we had uh two-dimensional array right which consists of rows and column that is axis 0 refers to the number of rows axis one refers to the column so what is the other name of arrays two-dimensional array very good that's also known as the matrix very good mirad dimensional or threedimensional array three-dimensional array will consist of access 0 1 and two. Access 0 you can consider it as the slices of the bread. Access one the number of rows and column in that particular bread or array. Clear? So what are the different attributes of an array? Dot shape gives me the axis or the number of rows and column. End dot endm gives me the dimension. dot d type gives me the data type. 32 refers the number of bits required to store number of bits. Item size gives me the number of item size required to store one element. So we do divided by 8 because one bite is equal to 8 bits and dot size gives me again the number of elements. Clear? Is this point getting clear to everybody? So now today we will till now we were only revising what uh we had done yesterday and now we begin with today's session. We have to understand functions. We'll continue. I request everybody to open the 3.02 file. Please go to your LMS download the material and be uh ready with the material in your Jupy Tabs 3.023 045. We are supposed to complete it today. I request everybody to go to your LMS, download the material and be ready with the files in your Jupiter labs. Anybody who's facing any difficulty regarding the material or the lab, please let me know. Anybody who's facing difficulty regarding the material or lab, please let me know. Yes or no? Can I get a quick thumbs up? Are we ready to begin with today's learning? A quick thumbs up from everybody. Quick, quick. Can I get a quick thumbs up? Are you all there in the uh you know session or not? Or are you sleeping? Parika, Suas, Malika Arjun, Rakkesha, Rishita are you there or not? Raitta, Parv, Parvikshika, Pranav, Lokesh, Ranjan, are you all there or not? Okay. Now let's understand with few function first is transpose function. So transpose function as we understand from the perspective of matrices that it interchanges the rows and column. Flatten will convert any array any shape of array into onedimensional array. It will convert it into one dimension and reshape will change it into rows and column. Clear? So the numpy transpose is nothing but if this particular array is of the shape 2a 3 y 2a 3 it consists of two rows and three column and when I run this np.transpose transpose function it becomes into 3 comma 2. Clear? Is this point getting clear to everybody? when I talk about flatten now the arrays of any shape it's not that two-dimensional uh arrays it's applicable to two-dimensional transpose is also not applicable to two uh only to two dimensional but any n dimensional array similarly when I use the word flatten any n dimensional array can be change is always changed into onedimensional array please be clear so What what can be the two ways in which flattening can happen? One is row major that I take this as my first row right so the elements of the first row are placed first and then the elements of the next row and so on. But if it is column major I have to change the order equal to f. So first the elements of my column get flattened and then the elements of the second column get flattened. Clear? Is this point getting clear to everybody? Is this point getting clear to everybody? Yes or no? Ishan, Mumika, Nha, Nagendra. And when I talk about reshape, we can change it to any shape. A one-dimensional array can be reshaped to 2a 3 2 and even in more dimensions. 6A 2 2a 6. But can can I change this 3A 4 to 4a 4? Can I reshape this 3a 4 to 4a 4? No. Why? Because 4 into 4 is nothing but your elements are 16 which it will give me an error. Clear? Is this point getting clear to everybody? Now there is this activity. I want everybody to open their Jupiter labs and start writing this first code. I want everybody to open their Jupiter lab. Faculties please take a note of this that learners are practicing this particular code. I want everybody to first do this code copy code quickly. Let me see who does copy paste the code and gives it on to me on the chart. Let me see who is the smarter one in this particular session and batch. Yeah. So the first code says import the numpy. We are creating an array A of one dimension X. We are creating a copy. And here we are trying to change the value of one of the positions of the second array like 0 1 and 2. So I want to change this value of 3 to 99. So when I do it for a it does get changed. But when I do a print x, the value does not get changed. Can anybody tell me the reason why value in x not getting changed? Because x over here we are creating a new variable in memory. That means a new memory location is getting allocated. The the memory location is not same as that of the A. It is different. Getting my point. Has anybody done the code and check? Check the output. Writing these four lines. Yeah. Paste it. Paste the code on the chat web buff. Good. Good. Nana. Good. Very good. So, did you see the change? Did you see the change? Yeah. Similarly, but when I create a view of the array. Now, run this code. Make quickly make the changes in this. Now, instead of copy, create a view. And now you will see that both values are getting changed. It will give me an output change in B also as well as X also. Getting my point that means both B X and B are pointing to the same memory location. Everybody is able to view this. Everybody's able to do this particular activity or not. Only Vehav has given me a thumbs up. What about others? Is it that difficult? So now getting back to the file I want everybody to be ready with 3.02 are very clear with the functions of the numpy. Good. So are we clear with the attributes of the arrays? Let's start with the functions. Can I get a quick thumbs up? Thumbs up. Everybody is ready. Are you ready with this? Great. All right. Yes, learners. Let's start with the first code that import numpy as np array np dot array and the dimension is onedimensional and it consists of 12 elements. So, can I change these 12 elements into 12 comma 1? Can I reshape this? Are you there with me with the code? Yes, learners. Uh do we see that? So now we see this onedimensional array now changes to twodimensional with the shape 12A 1. We can also reshape to 3a 4 because the total number of elements 3 into 4 is 12 2 into 6 is 12. So we can reshape it. Can I reshape this particular array into three dimension? 3A 2a 2 is it allowed? Yes learners is this allowed? So reshape we can change any dimension to any dimension from higher to lower and lower. Yes. Can we change it from can we change arrays from lower dimension to higher dimension and from higher dimension lower dimension using reshape? Yes or no? Tell me reshape function can be used to change any dimensions to any yes or no. Of course when the number of elements are the same. Can we can I change the dimension from one to uh three and then three to one? Is it possible? Webhov, Bumika, Mira, Prajal. Yes, we can too. This is the power of reshape function. Only VBove understands in this particular batch. Nobody else. Mana, what about to you? Mamata, Monica, Nagendra, why are you all not responding? Ashwaria everybody is getting this code or are you all facing difficulty? Please let me know. Can I change it to further dimensions such as 1 2 3 4 5. Can I change a threedimensional array or a one-dimensional array to five dimension array also? Is it possible? Yes. Because 2 into 3 is 6. 6 into 2 is 12. 12 into 1 is 12 and 12 into 1 is 1. Again 12. So the total number of elements remain 12. That is why we get these whole of arrays. Clear? Is this point getting clear to everybody? All right. And we see that a reshape function minus1 will automatically bring it back to onedimensional array. So if you pass the parameter in reshape as minus1, you will flatten it out to get it as onedimensional array. But there is one more function which exists in numpy array that is flatten. It consists of these attributes. C means flatten array elements into row major C style. F means in colar column major style as I told you. So it's either you know row major uh style or a column major forron style. A and k are not used nowadays so much because a means to flatten array in a column major order. If a run continuous in memory or row major wise. So now we have uh you know evolved from these uh issues. So you know the two main uh orders or the uh you know types of uh arrays that exist are C and F. Then we have K which means to flatten array elements in order of the elements laid out in the memory. Clear? Is this point getting clear to everybody? All right. So now what is A? Can you tell me the dimensions of A? Quickly tell me who's going to tell me what is the dimension of this array A? Highlighted array A. Yes, learners. What is the dimension of it's a two-dimensional array and when I do a flatten right normally by default it will do it row-wise so it will take order is equal to C okay and C is equal to A dot flatten order is equal to F so what is the difference if this is my two-dimensional matrix so if I Do row major it becomes 1 2 3 4 and column major 1 3 and 2 4. Do you see the difference? Do you see the difference? Another important point that you have to understand over here is that B is the copy not the view of array A. What is the difference between copy and view? What is the difference between copy and view? Yes, learners. None of you understand this. Again, I'm putting copy does not change the original array. It creates a new one. Whereas view does. So if I make changes in my B, will it be affected on A? If I make elements in B, will it be affected in the original array? Yeah, it will not have that effect. Good. Dan, Ian, are you there? So, nd array do.flatten it returns a copy of the array flatten into one-dimensional array. So that is the difference between reshape and flatten function. Are you getting this point? Flatten will always create a new array. It will return a copy of the array flattened into one-dimensional. So if we have this threedimensional array and if we flatten it using order f what is order f column wise that is I will take this first element 1 and 7 then four and then 10 then 2 and 8. Do you see the difference in three dimension? Then 5 11 3 9 6 12. Clear? And if I want to change the order, what will be the output? Let me put this Tell me for order uh C normal normal one. Does are you doing it along with me? It is like the normal order. And if I don't pass this parameter, will I get the same answer? Yes, I will get the same answer because by default it takes row major wise. Do you all see this the problem statement where it says as a data scientist your task is to create a Python project that explores the numpy arrays attributes and function. So this is the list given to you. Explore the key attributes find out the endm shape size and demonstrate the functions. Will you all be able to do this assignment? Will you all be able to do it? All right. All right. Okay. Now let's move ahead to the very important topic that is how uh you know arrays or the numpy library help in mathematical operations. So please try to understand if you know numpy. These are the two concepts that is you are expected to know that the arithmetic calculations in numpy are completely based on two main concepts. First is broadcasting. Second is vectorization. I want everybody to understand these two points because if you don't know these two points that means you don't know what is numpy if you're not able to answer this for any arithmetic operations to be performed on arrays they both the both the arrays have to be of the same shape and size. Okay, of the same shape rather not size but the same shape making the arrays of the same shape. Then we also saw that that vectorzation is that using for loops for element byelement operation that when we when we wanted to multiply a particular term with a in the list we explicitly used the for loop yesterday you remember we had explicitly used the for loop but in arrays you don't need to define that explicit looping it by default does the operation thus making it again much more faster. Right? Clear. Is this point getting clear to everybody? So what is the idea that if suppose you have a list and I wanted to multiply the elements so I had to explicitly use the for loop to multiply two with each of the elements right to get the answer. But when I am talking about arrays first of all the two arrays have to be of the same shape and size. That means broadcasting will happen. What is broadcasting? That it will make the two arrays of the same shape. Both will become vectors with two and two. And by default, the implicit for loop runs to give you the output. Is the concept of broadcasting and vectorization getting clear to everybody? Nha, Vehav, Bumika, Mira, Ashwaria to understand the concept better. Let's understand this with an example. Learners, please look here to understand this with an example that suppose these are my arrays. Now, can you tell me the shape of this array? this two-dimensional array. Quickly learn us what is the shape of this two-dimensional array? Yeah, this is 4a 3. Good. And what is the shape of the other one? That is also 4a 3. Are the two shapes same? Yes, they are. So, can arithmetic operation happens? Yes. element by element. This element will get added with this. This element will get added with this. This element will get added with this to give you the output. Clear learners, right? So that means in these two arrays no broadcasting is required only vectorzation happens and the arithmetic calculation is performed. The first case is clear. Now look at the second case. What is the shape of this array? This is 4, 3. What is the shape of this array? Ganga mira web of it has consist of one row and how many columns? Three columns. Very good web. Now are my two columns matching? Are the two columns same? Yes, they are. And broadcast this one to the same number of rows. Now, so broadcasting is possible in this case. Therefore, it expands this array into four rows to perform the operation. Getting my point? Is this point getting clear to everybody? Yes or no? And of course vectorzation will happen. Good swam. And of course vectorzation will happen for element byelement calculation. Now look at the third example. What is the shape of the first array? has consist of four rows and one column and this is again one row and three column. Good Chetra good bomega. Now how is broadcasting possible? You have to check that if one of the uh you know rows is four and other one is one right so can we expand it to four rows yes that's possible and if this column is three and this is one can I expand it to three columns yes so broadcasting can happen it cannot h it it is not necessary that the broadcasting happens every time and it is not necessary that it will happen only for one array it can also happen for both the arrays also. So here the both arrays again become of the same shape. Then the vectorzation happens and then you get the very very important concept. One more example. Please look here. I'm sorry for the marker which is being uh run over here. Yeah. Now np range. What is the shape of this array? It is one row and three columns. Agreed? And a range function is giving the value 0 1 2. And when I do a + five, what is + five? It is a scalar value or a zerodimensional array. Broadcasting will happen. Yes. It will broadcast it into five 55 and then the vectorzation operation happens. Clear? I'll again repeat it here in my first example. This is my onedimensional array with the shape of 1 comma three values. This is my scalar value. So broadcasting will make both the shapes same as 1 comma 3 and then vectorization will happen output clear. Is this point getting clear to everybody? Now ahead to np1's 3 comma 3. That's one of the other ways to create uh an array with all values one. So the shape of this array is 3a 3. And this is np a range 3 1 comma 3. Now there should not be any confusion since the columns match and one of the dimension is one. Can it be expanded? Yes. And therefore broadcasting happens and then the vectorization operation. Clear learners. Chetra mumika mira nagendra. Are you all getting this point? And again in the third case can both the arrays be broadcasted? Yes. When I have this shape of 3, 1 and this is 1, 3. Now how come broadcasting is happening? Since one of the dimension is one, we can broadcast the rows. The column is also one, we can broadcast it into three. So both the arrays become of the same shape and therefore the vectorzation logic happens. Swam are you getting this? Is this point getting clear to everybody? Yes or no? But where broadcasting doesn't happen? Now suppose if this is the array of the shape 4a 3 and this is 1a 4. So now the columns are not matching then it into four but since the columns are not matching it will give me an error. Clear? Let's do it practically. Learners are you ready with the file 3.02 sorry 3.03 learners are you ready with 3.03 03 web muma nha praal miraam good ganga good mumika so do we understand now this particular code yesterday we had worked on this that this is my list one and if I do an asterisk two that means this particular value values are repeated they don't sorry they don't get multiplied right but if I want to multiply it one of the way is by using I create an empty list I run the for loop and then each of the elements does it but a beautiful way was solved by way above that we can also use list comprehension to get the output. Is this point getting clear to everybody? Are we clear with this these points? Yes, learners. Okay. So now do you understand the beauty of arrays? Another beauty of arrays that by simply creating this array 2 3 4 and just by doing a multiplication two I get the values multiplied by two. So what is the concept that is working behind this? Yes learners tell me what is the concept working behind arithmetic operations. What are the two main concepts working behind arithmetic operations? First is broadcasting. Yes, that is 2 into 2 into two. And then vectorzation that is element by element. Clear to everybody? Vectorization is now clear. All right. So different arithmetic operations are possible. One way is by using this plus operator or you can also use the np do.add function for addition. Minus is for subtraction. Subtract. NP.gative is a unitary method. You get all the values as negative. Then we have multiply that is multiplication. NP.ide that is single division. Can anybody tell me what is the difference between divi divide and flow divide? Who's going to tell me this? Tell me what is the difference between divide and flow divide? Yes, learners tell me. Come on, tell me. None of you can tell me the difference between divide and flow divide. That's basic Python. I'm asking gives large gives integer value. Are you sure? Flow division gives only the integer value. SW divide gives the division value. Flow divide gives the remainder. No, SUM is correct. It is the Merson operator, the modular operator which gives me the remainder. gives me the quotient along with the defic uh you know the decimal values. This gives me the integer part of the quotient. Getting my point? Is this point getting clear to everyone? mod gives me the remainder. Clear above? Please be clear. And double aster is used for finding a to the power b. Is this point getting clear to everybody? So now when I do a is my array, one-dimensional array or a vector, d is my scala value. So what will happen? np do.add Add is vectorization and broadcasting happening over here. Is broadcasting happening in case of this particular code? Tell me in A and B vector is broadcasting happening. No mira says yes. What about others? One yes. One no. What about others? Why no mira needs to be broadcasted to the same shape, right? Then only the vectorzation will happen. Got my point NRA? Yeah. So broadcasting is happening over here. So either you can use an np.add function or even a plus sign can work. Clear? Now tell me what is A and what is B? What is the shape of A and B? Tell me quickly what is the shape of A and B 2a 3 right? Why web of 2a 2? There is it's not 2a 2 it's 2a 3 right web of so is broadcasting happening over here we are using subtraction is broadcast broadcasting happening over here is broadcasting happening in this example no why ma no why nha Yeah, because they are of the same shape. But vectorization is happening. Very good, Nia. Very good, Mina. So, but vectorzation is happening. Is vectorization happening? Is vectorization happening? Yes. Yes. Vectorization will always happen. What is vectorization? Implicit running of the for loop. Getting my point? Yeah. Similarly we have a and b both are of one-dimensional array. So either we can use np dot multiply a into b. So is broadcasting happening over here in multiplication problem. Is broadcasting happening over here? Yes learners is broadcasting happening in in multiplication problem or not? Come on nha mira answer mumika. No, it is not because again they both are of the same shape. Yes or no? No. For what? Nha. Are you understanding it or not? And vectorization will definitely happen. That element byelement multiplication will happen. Clear? Is this point getting clear to everybody? Nha. Yes or no? Vehav Rajal Nagendra. If we talk about division, A is twodimensional, B is one-dimensional. is broadcasting happening over here in division is broadcasting happening yes because they both will be made of the same shape and that's how we get the output and if I talk about power of a to the power b so 2 to the power 2 is 4 2 to the power 3 is 8 2 the power 4 is 16 2 the^ 5 is 34. 2 ^ 6 is this much. Clear? Getting my point ma'am? But how can we multiply 1a 3 matrix with 1a 3? See it's not matrix multiplic element operation. So if this is A, B, C, D, this also has to be E, F, G and H. It's not matrix multiplication. A will get multiplied by C, B will be getting multiplied by F, C will be getting multiplied by G and D will get multiplied by H. It's not that matrix multiplication like this. It's element byelement operation. That's why it is vectorization. Good Nana that's that that was a very good question. Everybody is getting this point. Now, as I've been telling you, statistics is going to be a very very important part of our whole data science. So, do we understand different statistical function? Do we understand the difference between mean and median? Can anybody tell me the difference between mean, median and mode? First tell me between mean and median. Yes, learners come on tell me what is the difference between mean and median. Anurag come on answer. Anvita come on are you there in the session? So respond. Hamsika what about you? Mah ready. Gas shrieka jvita kiran kishan kohi shria ready. Lea come on answer. Palak pesh. What is the difference between mean and median? Tell me. Chetra, vani, venal, vinaya, vaveshu chandra. None of you understand the difference between mean and median. Come on. Come on. Tell me. Tell me. Tell me. Tell me. Yes. Mean is sum of all the values upon total number of value. Median median is the middle value of the array. Do we understand std? STD function in numpy or python is used to calculate the standard deviation of an array. That is how far are we away from the mean. Percentile returns the nth percentile of the elements in an array. And then min is equal to returns the minimum element of an array. Max is returns the maximum element of the array. Clear? Okay. So now over here we are are you all there with me with the code. So if I have created an array a with a range uh 11 that is value ranging from 0 to 10. Then when I want to calculate the median I will use np domedian function. mean is calculated through np mean and np std gives me the standard deviation and npvar gives me the variance of the function. Now np dot again if it is even if it is not a one-dimensional array it's a two-dimensional array we can easily find out the mean the mean of all the values the median of all the values the standard deviation of all the values as well as the variance. Clear? Are we understanding how numpy library helps to calculate the mean, the median, the standard deviation and variance of the data? The file that you are accessing is little different miracing is is like this. I'll show you the file. Is it like this? Is it like this? Something like this. since I have made so here it is. So this this function is there. Some of them are missing. Some would be the same, right? Okay. Now let me work on this only. Yeah. So this one is there now. This is what you are talking about learners. Now fine. Let me use the same file. I'll not use mine. Let me use this. Yeah. Now are we clear? better. This point is getting clear to everybody. Good. It means that you are working along with me. The ones who are able to answer and they are responding that means they are working along with me. The rest are not. The rest are not. I don't know what are they doing in the lab. Yeah. Now when I talk about percentile, np.person function is used to compute nth percentile of the array elements. The nth percentile value should be in between 0 to 100. So when I want to calculate the 50th percentile or the middle value, it comes out to be four here. The moment you feel there is any discrepancy please let me know. So you know do you have the assisted practice with you assisted practice or not? No I don't see this assisted practice. Do you have the assisted practice or not? I don't see in my okay okay fine okay okay fine then perfectly fine now moving on to the string functions in numpy some of the files have been so now the string functions what what are the different operations that can be performed on the strings. When I do addition on string, what does addition on string mean? What does addition on the strings mean? Concatenation, right? Do you see this? And if I increase one of the values, you can do that. So what m wherever you feel the code is missing, you can tell me. I can place we can wait and we can go slow right so it's something like this so if we have two elements and if I increase one so this is n mit session you see this how it can concatenates the two elements at the same position And if we uh you know uh want to replace something, I want to replace this hello with a hi. I can use np.car replace function which replaces this this hello with an hi. Or if I just want to replace this capital edge with also be there. So the hello ed changed to yellow h of y also changes to whatever you want to call it. Wow. Clear. So I think so now you should get that confidence to play around with things, right? So try playing around with these you know functions. Similarly we can also work on converting the strings from lower case to upper case and from uppercase to the lower case. Getting my point? Is this point getting clear to everybody? If you look at further thing down, you see this assisted practice arrays for arithmetic operations, for statistical operations and string of functions. Will you be able to do all these functions? This is the assisted practice that you have. Mira bumika nha web just confirm. Webuff says no. Why no? Please go to your LMS and download the material. Anyone of you please download the material and sh send it on the chat. My material does not have that web you took it from my material. My material does not have it. That is why you're saying a no. Na bumika can you do that for me? Mira download just just send that uh folder compressed folder of notebooks and other material on the chat so that everybody has access to it. I'm not sending mine because I have all the solutions. So I don't want to give you the solutions. Aha mira will you do that? Please do that for me. Chetra bhavvesa ve are you there? Oh good nana that's good. Very good. Web not yours. You have it from the LMS. Yeah, from the LMS. Yeah, good, good, web. Good. Yeah, please check it out. Everybody work on assisted practice. Tomorrow we don't have a session. We will be meeting next week and I will first discuss and ask any one of you to share the screen. Okay. So the uh you know faculties who are there please make sure that the learners have completed this assisted practice. Now move with the next concept that is indexing and slicing. So what are the two types of indexing available in Python? What are the two types of indexing available in Python? Nobody understands this. Positive and negative. Very good web of positive indexing always starts from positive indexing always starts from zero and negative indexing always starts from minus one. Very good. Now does array support that? Yes, array support both positive as well as negative indexing even at the same time. That was not possible in list or uh tpple, right? But in arrays, it is possible to use positive and negative indexing at the same time. So if this is my array with 1 2 3 4 5 6 values, positive indexing always starts from zero on the left hand side. We move to the end. So when I say a to the power two what will be the value? What is the value at index number two? What is the value at index number two? Come on learners tell me for this index for this array it is three. Only they just got it. And if I change the value to minus2, what is the value at minus2? It is fine. Right? Both indexing are possible. Similarly, if I have a matrix or a two-dimensional array, this is nothing but my row. This is nothing but my column. So this is zero. This is one. This is two. And this is again zero. This is one. And this is two. Right? So how do I take the position? This is my first row and second column. This is my zero row and zeroth column. Can I do a mixture of positive and negative indexing? Yes, that's also possible that this is my second row. Row will always come before we cannot interchange that. The row will always uh any uh term written before the comma will always represent the row and any term written after the comma will represent the column. So and this is how we get this element as nine. Clear? Is this point getting clear to everybody? Same simple right. So twodimensional array we have two axis. The first axis always represents the row which is also known as axis equal to zero. Right? And this represents as axis equal to 1 that is the columns. Okay. Now let's see who's able to answer this question. Right. Let's see who's attentive in the session now. So if this is my indexing and slicing before the comma I'm talking about the zero row agreed learner so this is zero row like this and which columns am I talking about? What is 3 col 5 mean? Slicing is also same as the basic. What does 35 mean? What does the slicing mean? We are talking about the very good mirror, third and the fourth column. Right? So in this row we get the elements as 3 4. Very good. So are you are you clear with this answer? Now tell me which row are we talking about? Tell me the row that we are talking about in the next question. We are talking we start from the fourth and we are talking about fourth and fifth row. Agreed learners? Fourth and fifth row. Similarly we start with fourth. We are talking about the fourth and the fifth column. So therefore we get these points. Clear? God web na bumika praal mira. Yes or no? Only Nha got it. Only got it. Now tell me the answer for this question. Tell me the answer. Which rows are we talking about? Which rows are we talking about? All the rows. Very good. So we are talking about and which column second. Very good. Very good. So we are talking about these elements. Got it? Now tell me the last one. What is the concept that is getting used? What are these two colons known as? What is the concept of two colons known as and end position? We are talking about slicing. If we have a step parameter also, what is that known as? Yeah, what is that concept known as? Good meter. It's start, stop and step. But what is it known as? What is that concept known as? striding. No, that concept is known as Please remember that that concept is known as striding. Okay, please remember. So in this case we start from the second row jump to parameters. So we are talking about the second and the fourth row. What about columns? We start from the zero column uh column. Yeah. So and jump to parameters 0 1 2 3 4 and 5. So this is 12 14 16 24 26 28 getting this point. Have you been able to check the output? Understood the output mira. Yes. Or do I need to repeat it where again? This is my starting position. I start from the second row. There is no parameter over here. It will go till end. And the jump parameter is two. Here there is no starting position. It will start from zero, go till the end and the jump parameter is two. Now clear everybody to be concentrated here and then we will go in for a break because now we need to understand how does the threedimensional array uh slicing indexing works right we know that in threedimensional array they are three parameters or three axis which define that axis zero selects the matrix which is the slice of the bread that we want to work on that selects the matrix. Second index is J which selects the row and the third index K which selects the column. First point is getting clear. The first parameter over here is not the row but it selects the matrix. J or the axis one represents the selects the row and case selects the column right so if I have this example how is it represented it's like numpy array threedimensional array that's how it is represented and that's how a even a colored image is represented with the channels R G and B right so three-dimensional array is an example of a colored image image. Okay. So now let's take this example. If I want to access 2 comma 0 comma 1, what does 2 comma 0a 1 mean? That my value of I is 2. The value of J is 0 and the value of K is equal to here. Learners are you there with me? So now when I look at I, J and K, right? So if I look at I, I start from here 0 1 and two. So which matrix am I start talking about? I'm talking about this matrix. Clear? Then it talks about rows. So if I talk about the rows, I'll again start from positive indexing 0 1 and 2. I am talking for this particular matrix, this particular row. Clear? Is this point getting clear to everybody? Then I talk about K. K is equal to 1 that is 0 1 and 2. So first column so the element 31 gets picked up a mira how the three parameters are doing indexing and slicing over here and taking it forward if there is slicing and striding please look here Now this first comma before the comma is representing my I. This is representing my J. And this is representing my K. This is I, J and K. All right. So this is slicing happening over here. Since there is no starting point, it talks about the first two planes that is 0 1. The two is not included. clear when I say about J it talks about the rows 0 1 and two. So it starts from one goes till end for this matrix as well as for this matrix. So whatever matrix uh has been selected that many rows will get selected. And if I talk about columns it is 0 1 and two. So we start from the zero row first column two again is not uh included for both the matrices 0 and 1. Therefore we get the output as 13 14 16 17 23 24 26 27. Clear? Is this point getting clear to everybody? or do I need to repeat it? Tell me where miraa bumika quickly understood indexing in three dimension. Ajas Bara. Mana Monica Nagendra Malika Arjun Chaitan Kumar Ashwaria Abhijita Ashish. Are you all getting this point or not? Let me quickly repeat the threedimensional array for all of you. Let me quickly uh repeat it. So whenever we are talking about threedimensional arrays, threedimensional arrays are you know represented by three axis. Axis 0 represents the matrix. J is axis one which selects the row and K represents the axis 2 which selects the column. So if I have these three values okay to I is 2, J is 0 and K is 1. So I is selecting the matrix. So we start from zero 1 and two. So over here since two is given we are selecting this particular matrix. Got it Aijita mana they just got it. Now J represents the rows. So again I do it as 0 1 and two. So I talk about I'm talking about this zero row. Now clear and if I say k is equal to 1, k represents the column 0 1 and 2. So k is one I get it as 30. Now better they just and if you look at over here there is this slicing happening for I J and K. So if I talk about slicing it starts from the zero matrix two is not included. So I'm talking about the first two matrix. When I talk about the rows it is 0 1 and two. So I'm talking about the first two rows and column again I am talking about the first two columns. Rows I'm talking about the last two and columns I'm talking about the first two. Therefore I get the output as this. So, Numpai practically in today's world of data science has lot of practical applications in terms of data analysis, finance, machine learning, NLP, signal processing, astronomy, physics, climate, uh, science, robotics, chemistry, biology, etc. Right? Okay. So, can we add the elements directly? Yes. If I add the element at array position at zero at one position one position is two and at zero position it is 1. So 1 + 2 will give me the output three. Are we clear with this particular output also? And if we talk about twodimensional arrays then we have to pass two axises that is before the comma we are always talking about the row and after that we are talking about the column. So add the zero at row and second position we get this output right learners. You can also paste it over here. Play around with the code. Learners as I've been telling you start playing around with the code. Yeah. So we get this element over here at 1 one. So at the first row, this is my first row and this is my first column. So the element is five clear indexing is clear to everybody. Similarly for threedimensional array we can access the elements using 1 0 0. Who's going to tell me the answer if I change it to 1 0 1? Tell me quickly what is the answer for the values 1 0 0 1. Tell me code along with me. If this is my array, what is the output with 1 0 0 1? Is it eight web of only has done it? is the issue. What is the error? I missed out the square brackets. Yes, the answer is negative also exist over here. So when I do negative indexing for the uh onedimensional array I get the answer as four even is a mixture of positive and uh negative indexing allowed in arrays is mixture of positive and negative indexing allowed? Yes, it does not give me an error. All right, clear learners and again you have this assisted practice. Everybody has got this assisted practice will you? So there is lot of assignments to be done in the file 3.02. 2 3.03 3.04 3.05. So you have four assisted practices to be done. So now let's move on to 3.05. Slicing. What is the concept of slicing learners? What is the concept of slicing? That we have a start and a end position. If the start position is not specified by default it will start from zero. If the end position is not specified it will go till the end. And if there are three parameters start, end and step what are they known as? Start, end and step. What are the three parameters known as? Very good. Only web understands nobody else. That is known as striding. Clear? Clear to everybody. So let's start with slicing over here. Yes learners are you there with me? So this is my array. This is 1 colon 7. So it will start from which position? This is zero. This is 1. This is 2. This is three. This is 4. This is 5. This is 6. This is 7. And this is 8. So we start from one and the seventh one is not included. So the answer is 4 3 5 6 8 9. Agreed. Tas are you getting it? Abijita. Anybody having any questions any doubt please let me know. Is the code matching now? Mira is the code matching now. Aha is the file matching now. Okay listing of five colon. This is again slicing. Start from 0 1 2 3 4 and five. Since um the end is not specified, it will go till the end. So the answer is oops Java and cloud. Clear? What is this striding? What does striding mean? That it will start from the first position. 0 1 2 3 4 5 6 and 7. It will not include the sixth. It will start from first and jump. So 1 2 3. So the answer will be 7 and 4. Output is getting clear everybody. There with me? Is the output getting clear to everybody? And similarly when I talk about twodimensional array if I talk about twodimensional array. So here we are talking about the zero row and how which columns 0 1 2. So the third is not included. So only the second column. So the answer is 33. Yeah. And similarly when I talk about this threedimensional array you can print that we get this answer as five and six. Why are we getting it as five and six? Please look here. This is my zeroth matrix. This is my first. So, we are talking about this matrix. Okay. I is clear. J is 1 colon. That means we are talking about this particular row 1 colon, right? And if you're talking about columns, so 0 1 and two. So, it will start from one and two. So, the output is five and six. Clear? Got it? Ahija, Vehav, Nira, Prajal, Nha, Omika or do you want me to repeat? Tell me, tell me quickly. Are you all able to run it practically? And if you talk about negative slicing and like 0 to -1 this part is not getting printed. So if you look at the our last thing do you have the assisted practice for this? Do you all have the assisted practice for this? This is the negative slicing. Will you all be able to do it? Yes learners will you all be able to do it? Four assignments 3.02 345 four assignments in numpy that will be discussed in the next session faculties please make a note of that that these assignments needs to be done by them. Got it? All right. So now before we move ahead let's do a quick knowledge check how well you have understood about numpy and then we will start with Right. What is numpy and what is it used for? Yes, Lana's quick response. Yeah. So answer everybody is answering B. Great. So the answer is B. Yes, it is used for mathematical operations in science and engineering applications. Second question, what are the key attributes of nd dimensional array? What are the key attributes of nd dimensional array? Abijita that was quick. What about others? A B C D quick? Yes, Chetra. Good. A it is a it is n dimension which gives the dimension shape size data type and item size of the array. Okay, this is a good one. Which of the following is used to change the dimension of the array? Any dimension not specifically to a particular any dimension of the array. Which is the correct answer? It is absolutely correct. A the reshape function transpose will only change interchange the rows and column flatten as I told you will convert always into one dimension. Good. So are we clear with numpy learners? So now we'll move from the we did the numpy. So what was the basic advantage of numpy? What was the basic advantage of numpy that you saw that why this library has been built? Great Isan. Great. Yeah. Give me the biggest advantage of the numpy library. Speed for what? What speed? Mutable is not the correct answer. Even list is mutable. Dictionaries are mutable. That's not the correct answer. Speed and what web hub. Be very clear and specific. Please be very clear and specific. Faster in accessing doing arithmetic operations. You can say faster in doing arithmetic calculations. Can we say that? That's why it's it's a it's a numerical Python. And what are the two concepts for doing arithmetic calculations? What are the two concepts for doing arithmetic calculation? The concepts in numpy library. I told you not to forget those concepts. Come on, quickly answer nobody. Broadcasting and vectorization. Broadcasting means having having arrays of the same shape and size and vectorization is element by element uh operation. Very good. Good. Now another most important library of Python that is the pandas library without which no uh you know data analysis can be completed. So if you know if you say I know data analysis it is assumed that you understand this pandas library. Why is it important? Because it is helps to represent the data in the form of tables. When we say tabular data that means data in the form rows and column. This is how we generally see the data in terms of SQL Excel files that these are my rows right and these are my columns agreed learners this is what we generally see as data you know uh structured data to put it more precise that's how so a column is generally of the same data type right right and this can be represented as a onedimensional array Okay. Right. And the whole of the uh table can be as twodimensional array or in the form of matrix. Can we say that? So here we give them specific name. One-dimensional array in case of the pandas library is known as the series function and twodimensional array is known as a data frame. Please try to understand. So the two data structures available in the pandas library are pandas series and pandas data frame. Panda series is nothing but onedimensional labeled homogeneous array where the size is immutable and the data frame is twodimensional labeled array which is a mutable structure. Now the question arises that when we already had onedimensional and twodimensional array right then what was the need of these these data structures? So the need of these data structure is in their label indexing. Please try to understand here there are two types of indexing available. We can define our own label. That means here we don't have to give 0 1 2 3 but every column will have its own name or index. So the column name is known as the label index. And similarly I can give my own names to the rows also. It is not necessary that it will be 0 1 2 3 etc. Clear? Is this point getting clear to everybody? Right. So let's start with the first uh data structure of this pandas library that is a pandas series. Series is nothing like a column in a table and it is a onedimensional array holding data of any type over here but over here we see it as uh you know the default indexing but you can define your own labels also. getting my point? You can define your own labels also. All right. So, a series has a series name, a values associated with it, and of course, you can define your own index value. Right? So, how do we go about it? I want everybody to open their Jupyter notebook and start writing this code. The first thing that you will write, I want everybody to do this activity. Import pandas and pd. Everybody open your jupyer lab and start working on this activity. The first line is going to be importing of the library that is import pandas as pd. And here I am creating my two lists. One with the name temperature other with the uh days. Quickly quickly I want everybody let me see paste the co uh code initially and the function which is used to create the series function is PD dot series with the value temperature and index equal to days. Got it. And now you see the output. The output is vertically aligned. First difference and rather than having my index as default 0 1 2 3 I get this as Monday the temperature is this Tuesday. Anybody who's got this output quickly who's pasting the code first. I want everybody to work on this activity. So Nana was quick. So have you been able to do it? Series from the list. Now I want you to change this code right over here as days comma and write index is equal to temperature. Do that for me nana and then give me the output. Then check the output. Interchange the attributes of the series function parameters of just interchange them. Very good. Excellent. Right. Right. So do you see that the rows have interchanged? This becomes my index and Monday comes over here right Nana this is the output you get interchanged output good Aita very good very good so you've done it very good others have done it mira web nha mika the panda's library is built on the numpai library we have this two series s_sub_1 and sub_2 and when I do subs add we see that the add elements are getting added element by element so it uses the concept of broadcasting and vectorization well not broadcasting because both of them are of the same shape so it is using the concept of vectorization now quickly run this activity quickly run this activity Okay. Is it first? Sahana do the code quickly. addition of these two series. Very good. Very good, ma. And similarly you can do s_ub_1 to subtract s_ub_2. Good mira you're getting this output. And if I do not pass any index value I get default index value as 0 1 2 3 4. And by default it gives me the data type of these values. By default it gives me the data type of these values. Getting my point? So, pandas is nothing but an open-source library built on top of numpy and is used for data manipulation. It introduces data structures like data frame series that make working with structured data much more efficient. And the advantages are that alignment. It gives the data uh type along with it. Also, it uh u me data standardization. It's a structure intrinsic data alignment along with your label index. That's the advantages of the pandas library. The two data structures available in the pandas libraries are series and data frame. Series is one-dimensional labeled array. supports multiple data types and data frame is two-dimensional array right which can be a combination of series also right so now let's start with the very first code so first thing what is the first step that we need to do we need to import the pandas library what is data can you tell me the data type of this data what is this data it's a list index is also a list. So now PD capital series data is I pass it as my data and index I pass it as ABC D and that's how I can check my output. What we can do is we can I'll just do a copy paste so that it's easy for us to uh just understand the outputs. So just you can paste it and then do series with index. Do you see this index is a b c d and output is 1 2 3. Yes learners I've just done a copy paste over here. Nothing more than that. If you want I can just show it to you. Paste it on the chat also. Yeah. So this point is getting clear. We can also create series using dictionary also. The beauty with dictionary is that you know the keys become my index. The values become my data. Right? Do you see this? Both are giving me the same output. Learners, are you there with me? Me give it with index. Yes. Are you able to do it? above Nana, Muma, Nha, Mira, Chetra. So, Panda series can be created through list as well as through dictionary. Do you see this? This can be created with a specified index or list as well as through dictionary. Getting my point learners? And how do we do indexing? If I want to access any of the elements, default indexing is also there that is 0 1 2 3 4 and five. So it says to get the value two. So it will give me the value three or I can access only the element for B. Clear is with index and series with data. Okay. So this was print series. Yeah. You see the difference? If I do not pass the index, please try to understand over here. Learner, just add a print series over here. If I do not pass the date uh index, it will by default give me 0 1 2 3. And if I pass it with index, then it will use the index value. Clear? point getting clear to everybody. Now let's understand some basic functions in the series right in the panda series. These functions collectively help analysts summarize and understand the characteristics of the data facilitating effective data exploration and data analysis. So please try to understand these function everybody to be concentrated over here very very important functions which will be used in data analysis. The first is this head function which gives the first n rows. If I have passed the parameter three, it gives me the first three. If I pass it as 13, it gives me the first 13 rows. And if I do not pass any parameter default value is equal to five. Okay. Then the default value is equal to five. Tail is the last three rows. Shape gives me the dimension rows of column. Of course when we are talking about series will always be onedimensional array. Agreed learners? series will always be onedimensional array. When I move ahead to the describe function, it gives me the statistical analysis. Uniques, it gives me the unique value of the function. Please use these functions and n unique gives me the number of unique values. Got it learners? So it doesn't print anything. So we can print one of them. Uh see what I can do is learners you can also make the changes in the file. That's why I keep making changes in the file. So it becomes first. So do you see it gives me the first n rows and if I change it to 13 since there are only five elements it will give me five. And the moment I change it to one, it gives me only one error. Clear. Are you all getting this point? Learners, are you there with me? And then we go in with last n the last three rows. So I'm just cutting one of them. You can also do it along with me. Mira are you able to do it where? I'll share this file with you. It gives me five elements. Describe gives me the stats part of it. Just just copy paste it. That's it. Just print the outputs. It gives me the average values of this unique values. So 1 2 3 5 are unique values. And this n unique gives me the number of unique values which is five clear web are you able to do it nha mika now I've just broken those information and seen the output just simple functions got Now let's understand the operations and transformations in pandas series. Operations and transformation and panda series are crucial for modifying and enhancing. All these operations help in cleaning of the data. So if I want to see the element byelement addition series plus series with index right so basically over here it will if the um indexes are common then only I will get the output. Yeah. So 0 1 2 3 and then the other one is a b c d none of them are common. So I get the output as n a n. What does nan stand for? Not a number. Then I am doing the squared of the values. Again I'm just doing just copy paste cutting it and just checking out the output. So it gives me the squared of the values the map function. Right? Since I have given that one will become my one, right? So, wherever the value one was there in this uh series, let me get back to that series. Yeah. So, this one gets the number one, two gets number two and three gets number three. Clear? Similarly by using sort underscore values we can sort the values also. Well it is already sorted. So nothing uh to be missed over here. Just to check on the null values you can use the null. So this is these are the functions which give power to the pandas library for the analysis. So there aren't any missing values. So the answer is going to be false false right and finally it is the if there are any um missing values null value it will replace it with zero. So we will do a lot of um you know practical uh you know data with this. So nothing to worry so nothing changes. So are you there with me? Got it. So just to uh recap you know basically you can do element wise addition in series apply to apply each of the function mapping using uh dictionary values sort values to sort the data and to check the null value is the is null function. Yeah. Are we good to go? Can we move on to the next file? Not the next file, next topic that is quering a series. So if this is my series right over here if we check out the values greater than 30 it will give me only those values greater than 30 lesser than 30. Right? So these are the output. So this is my series the original series and I want to see only the values greater than 30 I just have to apply the conditional operator it has print below right so series indexing only those values greater than 30 get printed values only equal to 20 get printed values not equal to 40 get printed values greater than 20 and and operator is also clear lesser than 50 get printed right and is in one of them is series and the other one is data frame series is one-dimensional array and we also understood data frame as twodimensional array right and how is panda's library different from numpy can you tell me quickly what are the distinguishing factors niha sah that we can define our own label access right do you remember that we can define labels right so if we talk about pandas's library they help in manipulation of real time data in which is represented in terms of rows and column right right and the two main data structures in the pandas libraries are series and data frame series are one-dimensional homogeneous arrays whereas data frame are twodimensional size mutable tabular structure with potentially heterogeneously typed column. Getting my point? Have we all done series? Do you remember that particular file 4.01 that we had completed? The panda series is nothing like a column in a table and it requires the index that we can define our own along with the value. So you remember this practice activity that we had done that these are to my list and PD do series is the function this is referring to my data and these refer to my index value yes or no learners am I audible this this is the index so these are nothing but of my integer type data and this is nothing but your label index another important Important property of pandas is that they all the data is vertically aligned. A series will always be onedimensional and this was missing in the numpy files. And since the pandas library is built on numpy, we can easily add s1. S2. Here the important point is that the index numbers should be similar for addition of the series function. In this whole uh journey of data science, it is important to understand the concepts of statistics. Can you tell me the Python function which gives the statistical details? Anybody who remembers Python function that we have done detail of these values? Anybody who is the which is the Python function which gives the detail of statistical values? Yes learners function of head. What does the head function do? Yes. Learners, do we remember head? Gives the top five values by default. Do you remember tail? Yes or no? Nobody remembers. I think so. We need to do it today. And then did the describe function right which gives the you know the mean the standard deviation minimum maximum and the quartile values of the getting my point? You all there with me? Mean is nothing but summing up the value and dividing by the total standard deviation tells me how away from the mean. Then we have the quartile 2 three. So this is the function that I'm talking learners please look here. If all the values are numerical, describe function gives me count, mean, standard deviation, minimum Q1, Q2, Q3 and maximum value of the data, right? And if we talk about if all the values are uh characters as P pqr and if you do the describe function and if you do the count value it gives the four letters. Unique gives me how many unique values are there. Top gives me the top letter sprem has been repeated. Now moving ahead to the next concept today that is data frame. Data frame is nothing but a twodimensional data structure that is data is aligned in a tabular fashion in terms of rows and column. So data frame is nothing but a two-dimensional data structure that is the data is aligned in tabular fashion of rows and column. Data frames are you know are of different types because every column is different. It could be one of them could be int, float or string type. Definitely data frames are mutable. They have the labeled axis in terms of rows and column and it can perform arithmetic operations on rows and columns also. Getting my point right? So if we talk about the concepts of data frame we understand access zero refers to the rows. Please be clear access zero refers to the rows and the values can be accessed via df dot index. Okay. And columns refer to access one that is represented as df dot columns. And all the values in the table are can be accessed with the help of df dot value and each column has its own data type. Is this PPT getting cleared? So data frames can also be created with the concatenation of two series function. Getting my point? Let me see the new file. Yellow. Okay. So, are we clear with the key features of the data frame that it is organized as tables and rows, labeled axis, it has heterogeneous uh type of data. It is versatile also. Right now, do you see this particular data set? This is the file learners. Do you see this file? So the first way to create uh you know data frames is can you tell me what is the data structure? The first way to create data frame is using dictionary. Learners are you there with me? The first way to create dictionary is you uh creating data frame is using dictionary. First thing import the pandas as pd right. Secondly the key values auto automatically become my column heading and these are my list or the data points. Getting my point this is nothing but my data frame. PD dot data frame. Everybody is able to run the code. I have just split the code so that it is easier for us to understand the different ways to create data frames. Create uh you know data frame is using the list of list Yeah. Do you getting this? So this is PD dataf frame. This is the data list and this is the column list. Are you understanding the different ways to create data frames? Learners, are you able to run the code along with me? The way to create is array. What is what is the dimension of array that we are using over here? What are the dimensions of array that we are using over here? What are the what is which is what is the dimension of this particular array? Quickly tell me np array creates an array. What is the dimension of this array? Two two square brackets. What do they represent? Yes, it's a twodimensional array which helps us to create the data frame. That's a tabular structure. And another way is by reading a CSV. Should I share this CSV file with you all? I'm sharing the CSV and Excel file. Please download it and copy it into the same folder. So when I load this, this will run. It will not run because it's not there. So let me see. This is house prices. Please download this particular file. You'll find all these files in the data set folder. Then we have the iris folder. Once you have downloaded it, it should be present in the same folder. So how do we go about it? We do just a copy paste over here. right learners are you there with me? Do you see I've already copied the CSV file in here and now if I run my code it's able to give me the output. Do you see this learners? Learners are you able to run this load this house prices CSV? Only Sahana is able to do that. What about others? Anybody who's not able to do it? So data frames can be created from dictionary list of list numpy array reading the CSV file reading the excel file right and when I read this excel file now the iris.xls XLS that's how I get the output got it learners anybody with a thumbs up thumbs down who's not able to run this code please let me know data frames is getting clear to everybody Ready right now getting back to the PPT that I'll be sharing today on the LMS for the pandas. So this is what it is that data frames can also be created by simply using the pandas's readers CSV function and passing the path of that CSV file. So this columns get converted into a data frame along with rows and column. Clear? And another way that we have understood is using a dictionary. The keys become my column heading and it is the head function which gives me the top end values. Got it right? Now who remembers that that what are the of indexing available in basic Python or NumPy? Anybody who remembers that what are the two types of indexing available in basic py uh you know python or numpy. Nobody understands basic indexing. It refers to positive indexing as well as negative indexing. Please look here. What what does it mean? We we move ahead with positive indexing as well as negative. But Panda's library is different. Right? Here we can define our own labels. So how do we give the indexing part of it? How do we give the indexing part of it? Tell me quickly. That's known as label indexing represented as LOC and integer based or positionbased indexing known as IOC. Do you remember that point in series or not? Yes learners NMIT students are you there or not? So label indexing refers to the label or the values that we have given column value index they refer to the LOC values row index values refer to loc right and by default even positionbased integer indexing are present that can be accessed via LOC got my point. Getting my point? Explaining this with further example. Let me move ahead. That indexing in pandas can be label indexing or position based. Right? Label is what label we have explicitly given to the rows or column. Definitely columns have name. But if there are no names, no names given to the rows, it will take 0 1 2. Okay. Do you remember this type of indexing? This type of indexing is used to extract multiple values. This type of indexing is known as list indexing. If I want to extract more than another important point, slicing. Slicing is extracting the part of the number of columns right. So here we have 0 to2 right over here. But another major difference between label and position based indexing is that in label based indexing zero and two both are included whereas in position base this two is not included as in normal and basic Python. Got my point to everybody? Can I get a response? Are you understanding or do I need to repeat this? What do I mean over here? Is that slicing over here means that 0 to2 the labels that we have given to the column. But in this case 0 and two both are included. But in integer based or position based it's like the normal slicing only and zero and one are included two is not included. Now clear to make the concept even more clear let me let's understand with an example. So if this is my data frame the first bracket please try to understand this. So the first one is known as for the indexing but the first square uh bracket is used for indexing and the second square bracket is used for the list. Getting my point? This is known as the list and this is used for indexing. So what does it do that it in extracts the two columns automatically that is employee ID if I want to extract age that I can also add getting my point now right and if I say integer based indexing with zero at value it refers to the first row because here I am you know can you tell me data frame data frame is like two dimens row comma column. So if there is no comma given so it becomes row comma c. Getting my point right. But again if I want to extract multiple rows again I can use the concept of list indexing also. Now getting clear. Sahana, Ashwaryia, Nha, Nana, right? And do you remember this concept now learners? Do you remember this concept? This is label base indexing. If there are two colons used, what is this known as? If there are two colons used, what is this known as? Anybody who remembers the concept I explained. This concept is known as striding. It's not string, it's striding. Okay. So the last is the step parameter, right? We start from the zero row, right? And we will jump to values. So what are the rows that we can get? We get zero at second and the fourth row. Got my point learners? Since this is before the comma, this refers to the rows and this is after the comma, it refers to the column. So over here, what is the idea that the employee ID is also included? The scale is also included as well as the age is included. This is another difference. I hope this example in the PPT is making the concept clear. Yes learners getting my point. Is this point getting clear to everybody? Got it learners? So now let's get back to the practical file. So the accessing of the data frames is possible elements in pandas's library is based on two indexing. One is known as label indexing and another way is known as integer based indexing. Can you tell me how are we creating the data frame over here? How are we creating the data frame over here? learn which data structure are we using? What is data over here? Quickly tell me we are using dictionary to do that. So I'm just separating the code. If you want you can do that but so that that gives me you know a little more better easier thing. Or if I remove this, this is how I get the data from. Clear? Yes. Learners, are you there? Everybody's able to run the code. How can I access a single column? Just by giving the column name, it prints me the column data. If I want to access multiple columns, I can use the concept of list indexing. Clear? Difference between accessing single column and multiple columns is clear. Everybody is able to get this point just by single column name and if they are multiple then I can add the list onto it. What is ILC? Integer based indexing where I access the specific rows. The first row is given as the output. Everybody is able to run the code along with me. Yes or no? If I want to access the rows based on condition, I'm I I just do it because I think so it's easier to run the code and get the output rather than first see the code and we would keep on doing the scrolling. That's why I'm just cutting the code and checking giving you the output. So only those uh columns get you know get printed with column name greater than 15. Got it learners? Another way to access single cell by label is using is add. This is the zeroth column and the column name. Or we can also give the index value. Both will give the same thing or we can also use the label index. So all the I think so three will give me the same output. Yes learners do you see the output? This is five, this is 10. This is by label and this is loc. So why the position is not giving me the output? Position is not giving me the output because if I give zeros then I will get all the same values. Clear? So uh you would say that ma'am what is the use of label or integer base? They are exactly same methods but different ways to access the data. It's completely up to you. You want to use the index position or the label position. Clear? Getting my point? Is this point getting clear to everybody? Aha, Mira Goautamana Hamsika Walates are you there or not? If I want to do certain kind of conditional excess that can also be done. Clear? Are we good to go learners? Still waiting for the response. Everybody is has been able to run this code or not. Waiting for the response. Aijita, you were able to do it. That's great. What about others? Are you all there in the lab? 100 plus. Oh my god. You've got 120 now. Come on learners, respond. At least 50% of you respond. Manit, Mana, Marati, Mira, Monica. Quick, quick, quick. Rajual, Prakyad, Pragna, Pune. Come on, respond everybody. Shrea, Shivarat, Shashang, Saswat. Okay, great. Now understanding the data frame basics, the commands remain the same. Whether we are talking about u the series or data frame, the head and tail methods enable users to efficiently preview the initial and the final rows of the data frame offering a quick snapshot of its structure and content. These functions are invaluable for preliminary assessment. So now this is the power of pandas library. When I say that Python is a good language supports simple function so this is where the power of pandas library head function please be clear with these functions everybody it returns the first few rows tail gives me the last few rows info provides me the summary of the data I asked about describe it gives me the statistical summary in terms of count mean standard deviation minimum maximum minimum and quartile value. Then we move on to the shape which gives me the number of rows and column in the data frame. And without these functions, no analysis is complete. Column gives me the returns the column labels. LOC and ILOC are the way in which we can achieve indexing and slicing of rows and column. Sort value will sort the values. Group Y will group them. Apply applies each function to each element. Merge and concatenate will merge the data set. Plot will plot and drop removes the specified row or column. So we will as we move along we will explore these functions. Right? So if this is my data frame, please look here learners. Again I want everybody to look here. So if this is my data frame, right? What will the head function do? Come on, tell me what will the head function do? Since I've passed two parameters, it will give me the first two rows. If I have passed only one, it will give me the last row of uh of that particular data frame. Then if I pass give info, it provides the comprehensive summary of the data frame. And here it returns a tpple representing the dimensions of the data frame. Learners, are you able to run these first two rows? The info have we understood this basic operations head tail info shape. Now coming on to the most important part again statistical operations which give the insight into this data. So if we talk about descriptive statistics which is the function which gives me the details of descriptive statistics. Come on learners tell me which is the python function which gives me the detail of the descriptive statistics. Please remember this function. Okay. Sahana nha gautam hamika malates please remember which is the function python function which helps us to calculate the mean median and standard deviation. This is exactly the same function that were present in numpy. df dot mean gives me the means of the value. Median gives me the median and std gives me the standard deviation of the value. getting my point? Is this point getting clear to everybody? Are you all getting this point? Now the most important thing is correlation analysis. Let's try to understand correlation analysis. Okay. Now there is another important parameter which is part of the data frame that is known as the in place function. How will you override the existing data frame. So do you want to override the existing data frame? If yes then we will always make the in place parameter equal to true. And if you do not want to make the changes permanent then we will make in place is equal to false. So that we will see once we do it practically. But before that we need to understand correlation. Another very important aspect of statistics. Please try to understand this is the formula for R that is Pearson's correlation coefficient. What does it help us to do? It helps us to find out the relationship between two to three variables. So Carl Pearson's coefficient of correlation where value ranges between minus1 to uh + one which gives tries to give the relationship between x and y. So what does min -1 represent? -1 if the r value of the r is minus1 it it shows a strong negative correlation between x and y. What does negative relationship mean? that if x increases y decreases. All right? On if y increases x decreases. That's uh you know that's the uh negative correlation. And when r is equal to +1 what is the positive correlation mean? x increases y also increases or x decreases y also decreases. Getting my point that if x decreases then y also decreases. Getting my point is this point getting clear to everybody. And when r is equal to zero that means there is no relationship between the data points. Got it learners? very very important to understand the in-depth of the column for example for example if this is the data that has been loaded for your maths maths physics and history marks simple example I'm giving simply we have loaded the CSV file or the excel file containing the marks of maths physics and history and when I run the function dfc core It tries to create an n byn matrix. Please try to understand. Since there are three columns, so a 3x3 matrix is created. What is the use? It gives me relationship between maths and maths. Of course, diagonal elements is going to be one because we share positive relationship with ourselves, right? We are perfect with ourselves. So there is this perfect relationship between 1 to 1, right? But what is the relationship between maths and physics? It's very strong. It's very close to 1.9. But the relationship between maths and history is negative with each other. Similarly, if we talk about history and physics, they also share a negative relationship with each other. Getting my point? Is the correlation getting clear to everybody? Have you understood what do we mean by correlation? Sahana, mira then there is this another function value counts which helps us to calculate the unique value. So based on the categories they are north, east, west and south. How many times east has been repeated? Only two times. How many times north? One. How many times south? One. And west was two times. Got it learners? Is this point getting clear to everybody? So getting back to the practical file. So which is the Python function which gives the correlation between the values? Yes learners, which is the Python function which gives me the correlation between the values. It is dfcore. And what is the range of the values? The range of the value is always between minus1 to + one. Here they all share a positive relationship. But if I want to make it to negative, yes, aa, there is some issue. Yeah, it's df. And if I really want it uh to be uh negative, maybe I can put a negative sign over here. Then run and see the output. So between column one and column 2, there's a negative relationship, but one and three uh you know share a positive relationship. two and again two and three share a perfect negative relationship and that is true also. Clear? Then we understood value underscore counts which gives me the categorical data along with the frequency value. Are we clear with this particular point? Are the concepts clear? Are we able to run the code? And this is the assisted practice for you all that you have to load the housing data set and then perform such actions on it. Will you be able to do that? This is the data set. You can save it for your homework assignment. Got it? So what is the use of datetime function in any coding language? Can anybody tell me what is the use of date time function in any coding language? No. For example, if if I now look at over here, I am creating this. Do we have any date type over here? What is the date today? Let me put that as that 04 03 and 20 26. All right. Now suppose if I don't give the code. Okay. Right? Do you see it doesn't allow me to uh even print this? Suppose it's it's saying that no literals are allowed. It doesn't allow me to give three of so let me remove this. So it works. Let's see. It's - 2025. What does this minus 2025 mean? We are not even getting the correct answer. learners. Are you getting it? So that is why a library has been built in Python to deal with date time objects. It is nothing but an integer data type. Right? Got it learners? And if I do a date + one, do you think I'm getting the correct answer? Do you think I'm getting the correct answer? No. Right? So that is why to deal with date time. Most of the time the real time projects will contain columns which are you know of date and time. It could be arrival, departure of the flight, when that particular recording was done, how long the session went on, right? And who uh bought at what particular time, when the items were sold, whenever they were shipped. Every real time data uh requires this datetime analysis. Got my point? So in pandas the datetime module provides robust functionality for handling date and time. So pandas library because pandas is a library which helps in data manipulation realtime data. So in pandas the date time module provides robust functionality for handling date and time data while the time delta class allows us for convenient manipulation of the time interval. This combination is particular part particularly useful for timebased analysis and working with temporal data in the data frame. Right? So if I want to access what is the date time now then what is important over here is that we need to import the date time module. The first thing over here please try to code along with me learners. Import date time. Okay. And if I say I'll just send the code today is equal to date time dot null Oh, I put it as data. So it doesn't have now or if let's see then it's not working. Let me see. So dt sorry this is the accessor and then we have this as date time dot today. Now do you see it gives me today is uh you know um what is it 4th of March 2026 and if I want the time to be now how will I go about it dt dot date time dot got it learners are you getting this point. Are you able to understand this or not? This is the time for today and now clear learners na everybody's able to run it. Should I uh copy paste the code? Yeah, I'm pasting this code on the chat. Please run this code. Yes, learners, are you able to run it or not? And if I want to do any arithmetic calculations on that, then I can use this time delta function. For example, if today's date was 26th June, it would have added three six days to that to get me this output. Clear rather 5 days. It is actually 5 days. So there is this error. So it is five days which can be added. We'll just see getting my point learners are you there with me? So that's why data handling is very very important because it involves handling of the realtime data set. So first thing how do we create a date range in panda. So the date range function is used to generate a sequence of dates with a specified range. So we can generate sequence of dates. It's a powerful tool for creating time indexes or date column. For example, if I want to create dates starting from 1st of January 2023 to 10th of January daily, I can pass this as frequency equal to D. And if you want to start it from today, let's start. Or you might be thinking that ma'am, do we need to pass always the year? Let's see if I can pass the today's date 04 203 2026 maybe I want to go till 14th 03 to 2026. So let's understand is the format important or not. we don't get we are getting a blank index over here. It's not able to interpret the output. So if I give this as 20 26 so there is a format in which it understands right. So 03 and maybe I can give it till 20th 206 03 till 26. Now it gives me the output from 20th to 26th March. Got it learner? So this is the format in which the date works. It is year four times year then the month mm and the days. Now look at the beauty of this that if I enter a wrong date for example in the month of let's let's put it as February and put it as 29th was this a leap year no it wasn't right and then till 26 does it allow us to do that let's see it gives me an error automatically day is out of range automatically it will give you the day is out of range So that is the beauty of using this date time library. Are you all getting this point point? What I'm trying to say over here is that if I give 28, it will not give me error. But the moment I give 29th February, which is obviously not a valid date, it will give me an error. Clear? Is this point getting clear to everybody? Are you there with me? learners. Yes or no? Yes. Lers are you there? Maruti, Pranav, Nha, Nagendra, Nana. Okay. So now if I want to extract the part of the date. Okay. So what is this? Here I have created a data over here. Again, I'm just doing a cut paste to make. It's of object type, right? Or it's like this. Can we do it like this? No, it doesn't work like that. Or does it work like this? Let's see that it is a series. But I want to know the data type. What is the type of the data? Are you getting this point? I want to know the type of its values. Or if I say values that's numpy ndi no what we are looking as it is as yeah please try to understand learners are you able to run Okay. Or the one we forgot. It's not as type is D type. So it belongs to object data type. Got it? It's like object data type. Date is object data type. Can we work on this object data type? What I'm trying to say is can I do DF dot date + one? Can I do that? No. It will give me an error. So which is the function which will convert it into datetime object. This function please try to understand learners. It is this function that will converted into date time. So now if I check the df date well the output will look just the same. Do you see this? Has the output changed? Let's let's see this. Do you see this? This is year than this. So this is exactly the way we are going to get the output. Got it? And now if I check it type, this is of date time checked. Got it learners? Or another way to do it is other way to do it is please look here learners what I'm doing is let me just cut this rather than uh d type let me do an info it will give me that this is a date time object consist of one column. Now I'm adding one more col column into the data frame. Right? My data frame looks with two columns. And now look at the difference. They both look the same but they are not. So now if I do a df dot info the first date is of date time and the second one is also of the date time. Did I convert that? Oh earlier we have done that. That is the reason again I'll load it. So this is the df. Again please look here learners. I'm going to give you the code. This is the data frame that we created. And if I check the info of this then this is of object data type. Now I convert this. This is the output. So the first date is now of the object type and this is new one is of the date type. Now clear. Now if I want to extract each individual's day I can use df dot date dt access uh and day access specifier then month and the year. So by using these functions we are able to extract that what is the error. Okay. So you have to change this to date one because now we had changed it to date one. Now do you see this? Got it. Las getting my point? Are we good to go? Please run the code and give quick confirmation on the chat. Faculties please check them that are they running the code or not? If they are facing any difficulty, please let me know. Come on, quickly learners. Please run this code. Okay. Now moving ahead and now we have again created a data frame ranging from 1st of January to periods 5. So we will put it as today's date so that we understand the significance better. Okay. So if we see this this is 4th 5th 6th 7th and 8th of March weekend. Now automatically what happens is in at back end can you tell me 6th which which is the day on the 6th 4th is today Thursday Friday is 6th then we have 7th as Saturday and Sunday is that true 9th is Monday right right so how do we go about it automatically the weekday number this is add five, right? And uh let me put it as nine. Now check it over here. Today is 4th of Oh, it was March. It it is 2023. That's what I'm confused with. Okay. Now let's go ahead do this right now look at over here today is Tuesday or Wednesday so by default Monday is given as value zero please try to understand how is this daytime module helping us this is Monday this is Tuesday this is Wednesday and this is Thursday so Thursday tomorrow is Thursday tomorrow is this then Friday and Saturday Sunday is 5 6. So the moment we will it knows that it is 5 and 6. How do we know that? If the value is equal equal to zero. What does equal equal to 1? For example, if 5 is equal equal to 1, it is true. That means it's a weekend. And 5 / 6 is also 1 something. Then also it is true. Rest all are not weekends. Are you able to understand this particular logic? Here we are trying to find out weekday and weekend information. So simply by using data accessor week day and using the logic of dividing it by five equal equal to 1 able to get it in true and false whether it's a weekend or not. Clear? Is this point getting clear to everybody? I've not made any changes except this date and period also. So now if you put it as 7. Now you can see 0 1 2 3 4 5 6 7. Got it learners? So again what is the use of this datetime module? We can convert any date time module any object data type string data type to date type module. And if I want to add and subtract any previous or next date that can be easily be done by the time delta module. Getting my point? So if that was the previous date on the 1st of January the previous the the 1st January 2023 the previous date automatically it gives me 31st December 2022 getting my point so we don't have to decide the logic it is automatically decides the logic and does the arithmetic operations on that got it las delta function is used for arithmetic calculation. Can we pass this as once days, hours, minutes are allowed, weeks, days, hours are allowed, months are not allowed. makes. So it talks about the previous week. Do you see? And this is the next day. Do you see the difference? Now when I give this as weeks equal to 1, this gives me the previous week. This is how it gives me clear learners. So time delta in pandas is using to you know represent the duration or the differences between the two dates or time. It can be created by specifying the desired duration such as the days, hours or minutes. So again what are we trying to do over here? Here I am my data frame with date ranging from 1st of January 2023 to 10 periods in terms of hours. The value one is range this much and value two is the range this much. So this is my data frame. Let's cut it over here so that we understand it better. Says that future warning it says do not use capital H use small H. So please change that. These changes keep happening in the community. Yeah, I'm making the changes in the code. Please make the changes along with me. And now I am adding days three creating a time delta of 3 days. So what are we doing over here? Can we do this date plus delta? No. Date of what? PF is equal to no. PF changed it equal to this. Okay. Okay. Now you see this three days are getting added. Right? I'm just giving you the code. Whatever changes I'm making, I'm giving you the code. Yes, learners, are you there with me or not? Please run this code everybody. So, arithmetic operations can be performed using this time delta function. Got my point? So now if we talk about have we understood the importance of the datetime module? Yes. So basically any uh whenever any uh you know CSV file or an Excel file is loaded the date column is by default of the object type. Please remember this sorry that the datetime object is by default of the object type. Getting my point and it is once we use the pandas to dataf frame function it gets converted into datetime object. And why do we want to convert it into date time uh data type? We want to convert it so that arithmetic operations can be performed on it. Right? Otherwise it is not possible to do it on the object. And how do we perform arithmetic operations on the datetime uh module using the time delta function. Getting my point? So, pandas offers a rich set of functionalities for working with date and time data primarily through its uh NS data type that's the dot data type and by using the accessor DT one can extract the week day the day and the year in the date. By using the accessessor DD one can extract that. Got it? Is this point getting clear to everybody? Yes. So now we understand that basic class is date time. when I do class date time. Okay, for specifically date we can use the date function. For time we can use the time function. Then we can combine the two also for the date time function. And for time delta again for adding or subtracting the values. And then we have the TZ info which helps us to give the uh for the time zones right for my time zone and your time zone can be different time zone for Asia, Tokyo, USA etc. Clear? So first date time is nothing but the module that needs to be imported. Then we have the Python class and today and now are the Python function which help us to give the date. So how can we create datetime object using date or date time class? by using now or today by conversion of object data type to datetime object or as we have understood using the pandas date time range function which helps us to create a date range object right and if we want to specify the format of date time you can use now do strf time or you can also use strp time Right? To convert a string to a particular format, you can change the format of the date. For example, if you want the month before, then the day and the year, that can also be done. Clear learners? And these are the different percentage. Capital Y for fourdigit year, small Y for twodigit year, a small M for two month and capital percentage M is for the minute of the time. So I've added all the description into this PPT whenever you need to uh you know if you have any issues once you go through it again you will understand it better when if you have any issues you can use it access the data if you have any doubt and of course it is the pandas date range function which helps us to create the range of the data got it and extracting the date time component is you is due uh is through the DT or date access access specifier clear. So if I have this date as now and I give time delta days is equal to 3 days 3 days get added and if I want to add the number of hours or week or minutes also that's also possible to add that clear and if I want to add years then we have now is equal to date util relative delta years is equal to two clearance any down till here right and you can add even subtract the days to perform any arithmetic calculations clear are we good to go learners right now Another important concept in when we deal with real data is categorical data. What do we mean by categorical data? What are the color of your eyes? Green, blue, black. What is the gender? Male or female? How what are the grades that you have got? A, B, C, D. So, categorical data is not numerical data, but it can be further divided into nominal. That means there is no importance of the order that is it could be the postal codes or the gender male and female or the order for example the ranking that you give or the grades that A has better uh ranking than the Fgrade or socioeconomic stata right so categorical data needs to be understood you remember the first session which I had t taken that basically data is divided are understood into two types. The two types of data are numerical and the second one is categorical. Please try to understand this particular point. So data can broadly be divided into numerical or categorical data and the categorical data is further divided into nominal or ordinal data. Got it? So categorical features are can like if the data is nominal then the concept that we use to deal to convert categorical data to numerical is one hoting encoding. You might be thinking that ma'am what is the need or why why what is the need of conversion? See practically uh categorical data such as eyes are red uh blue, green or uh black computer doesn't understand. We have to convert that blue, green, brown, brown or black into numerical data to make the computer understand in the algorithm. Right? So the categorical data is further divided into nominal that is one hot encoding and the other one is ordinal due to label encoding right so the two types of encoding available are one not encoding for the nominal data in pandas we use the get dummies function which is also used in uh machine learning but for machine learning I thought I should give you an idea scikitlearn library we also use the one hot encoder and for ordinal uh data in pandas's library we use the cat.codes codes function and for scikit learn we use the label encoder function. Getting my point? Is this point getting clear to everybody? So how does it happen? For example, if we have a column of fruit and these are the categories, right? I want to convert it into categorical data. Presently, it is of object data type. What do we mean by object data type? string data type and the moment I change is data type that was the mistake I was doing I was doing a type I was changing one type to another to know the type of the column we do d type and for type conversion for changing one data type to another we use the as type category and that's how I get the output is exactly the same there's no difference in the output Yes, but the data type has now changed and it has also understood that fruit has got two categories. One is apple and the other one is orange. Clear? Is this point getting clear to everybody? Right. And then print type fruit categorical value. We get these values. Got it learners? Categorical data is getting clear to everybody. Getting back to the practical file of this we have performed arithmetic operations. Datetime module is now very clear and we can also do reampling of the time series data that is the time time series data can often come with irregular intervals right it's not always a day after a day so we can also do reampling in time data which is the process of changing the frequency of the time series data either by upsamp upsampling in frequency or downsampling decreasing frequency. So how do we go about it? We set the index to date in place true and then select the D types number based on the days and do the sum. So are doing on the DF. Where is the DF? This is the DF that we are talking about. So this is the DF the change and the future date. And here we get the DF reample where we are trying to sum up all these values, right? 10 + 11 + 12 + 13 14 15 16 17. We have reampled the values and therefore we get the our values as 45 and 145. Got it? This is all for the 1st of January 2023. So I can take the summary of the data. Resampling allows us to take the summary of the data. And now coming on to categorical data. There are two types of category, ordinal and nominal. So here I have created a list of categories and values. And how do I create an categorical data? By using PD dot categorical values, I get this output. And how can I count the occurrences of each category? By using value counts function. Right? That point is also clear. Value underscore counts function is clear to everybody. Come on learners respond. Is this point getting clear to everybody? And then we have creating dummy variables. Again I have this categorical data and dummy variable will create the number values true and false right and label encoding is achieved with the help of dfc category as type. So there are two types of encoding happening over here. This is for nominal data and this is for ordinal data. Clear? So again the dummy uh assisted practice is given for you for on the same data set the one which I had shared it before that is housing data. Will you all be able to do this? faculties are the learners doing the assisted practice I want to know the response any questions any doubt in that please let me know so basically text is nothing but your string variables what are the different operations that we can perform on string variables changing uppercase to lower case lower case to upper case finding out or replacing any substring so what are we doing in this case yes learner Just tell me we are creating a data frame with these columns and what are we trying to find out? Can anybody tell me what are we trying to find out using strleen function? What is this function doing? Tell me learners. Now we are trying to calculate the length of each string. Got it? And and we are adding have you understood how to create how to add a column in a data frame. Just put the name of the new updated column equal to sign and the new column with the length gets uh you know with the output. Clear. getting my point. Any questions? Any doubt? What is happening with STR lower function? Tell me what is this STR lower function doing? Abijita, Ashwaryia, Akshhat, Pragna converting the uh elements into lower case. Yes. Are we understanding the code? Simple. How to deal with string function which is very common in normal pandas also in normal numpy also we had done that even in data frames. Yeah. Yeah. What are we trying to do in this case? Here we are trying to find out the substring that which of the data has data in it. Hello world python. So data science is the only uh word or uh element in the row which contains the string. So that is why all else are false. Only data science has true. Getting my point learner. So it's returning me a boolean value in terms of true and false. Okay. Yes. Learners, are you there? Gordana, Abija, Bragna. Now tell me what is iteration? Now tell me what is iteration? Tell me quickly what does iteration mean in any programming language? I'm waiting for you learners. traversing through every element, right? We want to why do we want to traverse to through every element? So that we are able to check some condition criteria, right? That's iteration looping basically, right? So in normal uh you know Python, what are the different looping structures available to us? How do we iterate in normal uh Python? Tell me what are the two looping structures available in Python. Tell me what are the two looping structures. Come on, nobody understands that. What are the two looping structures available in Python? Tell me quickly. For and while. Very good. And what is the difference between for and while? Come on, I'm not asking very tough questions. Please respond. What is the difference between for and while? Nobody understands nmmit. Come on. What is the difference between for and by loop? Nobody understands. Come on. This is very strange. I'm asking a very basic question. Ashwaria aka Amaruna. Amratona. Come on. Dan Dhanosh. Answer four iterates through initial value to end of the value incrementing the iterator. And can we say it's a count control loop? That's the correct way to answer. Four loops are count control loops where we know the number of iterations in advance. Whereas while checks the condition and then iterates, right? Then if we want to do this iteration in pandas, what is the challenge in pandas? How is it different? What is the thing that we are looking in in in pandas library? Now in pandas we are looking things in the as real world data set in terms of rows and column. Generally we would want to iterate through the rows of the data frame in one go. Not a particular item, not one single item, but a complete row together like in SQL. Yes or no? Here we would like to iterate a row that is all the elements in the row together, right? Or all the elements in one column together. Agreed? Right. The whole idea is that we want this processing to be fast. We don't want the system to slow down because it has to iterate. Suppose there are 10 columns in one row. Then we don't want it uh to be you know slow down. We want it to be efficient. That is where the pandas provide efficient methods for iteration and applying functions to all the elements of the data frame. So the different types of iteration available are iter rows iter tpples right and iterms. Please look here. These are the three methods that can be used for iteration that is iter rows right. Then iter tuples and then we can also have iter items right. So iterating over the rows. Please look here how are we going about this. Here we are creating this data frame. Let me first cut this and then explain you right here we have this data frame df.pd dot dataf frame over here with column one and column two. So what is the challenge over here that I want to iterate all the elements of the first row and then second elements of the second row. Got my point learners right? So how do we go about if I do a df do iter rows automatically it will give me index and rows of the column that at f at the first index this is my data at the second index this is my data in the form of tpple got my point now suppose if I add and existing column. Suppose if I add one more column to it, right? Suppose I've added one more column then also it is able to give me an iterate. So even if there are two columns, three columns or 100 columns, iter rows is capable of giving me the answers in uh completely. Got it? Now why is it not giving me the output? Can anybody tell me why is it not giving me the output? Right? Because I'm not added this. So for the row I want to give got it now. So has this made iteration easier and faster learners? Then we also have an apply function to the existing data frame. All right. So let me again cut it. So this is my existing column and the new column I want to apply is multiplying each element with two. Got it? By using the apply function. So arithmetic operation is done. But which one is the best operation? Vectorized operation is best. Why? Can anybody tell me? What does vectorize operation do? It does element byelement calculation. Clear learners? Are you able to see the difference? And this is how it adds it here. We don't even need to run the for loop. What is the beauty of vector operation? What does vectorz operation mean? Do you remember that? Do we need to run the for loop explicitly? Do we need to run the for loop explicitly? It automatically iterates and implicitly runs the code for me. And similarly, can I do iterating over series? So if I have created the series, I can run on series dot item. These PDFs and PPDs are useful in understanding the concepts. So iteration in pandas can be achieved with the help of iter rows apply or vectorize operation. Iter rows are most successful for smaller data sets. Apply for complex rowise transformation and vectorz operations are the fastest and the best method at the moment. So always use vectorz method. So iter rows does not preserve the data type across the rows. Each row is converted to a series which might lead to type changes if columns have mixed data type. Less efficient for large data frames compared to the vectorz operation. And the disadvantage of iteros is it is not generally recommended to modify a data frame while iterating over it using the iter as it can lead to unexpected behavior. Right? Syntax we are very clear for index gives me the index of the values and row series in data frame. In data frame index is the current row. row series is nothing but the data of the current row in terms of tpples containing the index and data of each row. Right? So this is how I create my data frame of year and sales. And here when I run the iter rows I get the index along with the row year and sales I get the tpple value. Right? This is what we have seen practically. Yes, learners. We've already run it and seen before. Yes. Or do you want to run this code? Type in and run this code quickly. I want everybody to type this code and do this activity quickly. Yes. learners web ra quickly do this activity right so try to understand one more thing right why do we want to use this because we want the processing to be faster the other method that we have used is df do apply function now np dot sumaxis is equal to 0. What does access equal to0 mean? Tell me from arrays we understand. What does access equal to 0 mean that means we are going to take the sum of all the rows. Access equal to zero means operation on rows. Absolutely correct a right way above. So over here we are doing np dot sum which applies to all the rows and that is why it takes the value of sum and output of this is 27 and the output of q is going to be 75 clear and if I give access equal to 1 it means columns operation on columns so it will sum up all the values of the column. to get the output as for the first row 34 second as well as third row as 34. You getting my point? Clear to everybody? Got it? So we understand we also have map and apply function. I'm just trying to give the you know a comparison between map and apply function. Map works only on series but apply works on series and data frame. That is why it is more preferred method. It accepts functions and it applies also works on multiple columns. Map does not work but apply does and element wise yes. Got it? So apply function is more preferred function in pandas. Got it? Now what is sorting? Tell me what is the concept of sorting learners quickly. What is sorting in pandas or what is sorting in general? Come on, tell me. None of you understands what is sorting. Arranging the data in ascending or descending order. Right? What do we mean by ascending order? What do we mean by ascending? Nobody understands even ascending from lower to higher values. Right? And what is descending? What is descending? 10 to zero higher to lower. And if they are string values, then what does ascending mean? It means from A to Z right Aijita and if it is string then it becomes Z to A that point is also clear okay right so when I do DF dot sort values if I want to do sorting on the columns based on employee ID by default what type of a uh You know sorting do you see by default we see ascending type of sorting right by default we see ascending type of sorting from lower to higher values only this column is sorted based on the other things. Clear? You would say ma'am if I want to do it in descending order then we just have to pass this parameter. There is no thing as descending but we will pass the parameter ascending is equal to false. Right? Here we have no ascending order but here we will have ascending is equal to false. You getting my point? here here if I want to do it with two columns. Please look here. If I want to achieve sorting in multiple columns, how do we go about this? So it says first is skill skill is true. That means the ones with Java J comes before P is coming before this this uh you know uh in the order then capital Python has more value than the lower case. So capital P python is before the Python with lower case. Got my point? Now what is the role of this second uh column employee id equal to false only in the case where the values are same of the skill then we would uh you know select the second column that's employee ID and then sort it in descending order got my point learners have you understood this point Abhijita webhav praa akshhat Again I'm repeating please listen to me for multiple columns how does the sorting happens that scale by default is in ascending order. So we understand that the capital letters have more priority than the lower case. So this is in the lower case. But what is the role of this second column or the second criteria where these two skill match? Then we use the employee ID. Right? Then we do it in descending order because over here we have passed it as false. Now clear. Karthik, Jagan, Chetra, Balvik, Balvik, Krishna, Han, FZ, Hamsika, Mahir, Ready, Anika. What happens if I want to do the sorting based on the rows? df dot set_index employee id in place equal to true that means what does in place do that whatever changes are in data frame they are also reflected so by default employee id it has just set the index with employee id and now I can do the sort index also for access equal to one it will do it alphabetically age will come to pay and otherwise if I give access equal to zero then it will do like 7 12 no 7 10 12 15 and 21 clear got it learners Now let's run it practically. Nha Hamsika Haritta Malates are you all there? So sorting in pandas again it's a data frame consisting of rows and column. So the rows can also be sorted as well as the columns or the values can also be sorted. Both things can be possible as well as an ascending and descending the column. Both things are possible. So here I have created a data frame. Look at my data frame over here. I generally like to keep it separate because I can do the comparison of the output. That's the idea otherwise no other. So this is my data frame. I am creating it from the dictionary. What is the advantage of creating from dictionary that my keys will automatically become my column names. Right? And now I sort values by age. by default in ascending order and if I want to do it in descending what will I do? How will I change it to descending? I will pass the parameter ascending equal to ports. Now we see it in descending order las right and if I want to change it to salary that's also I can do it in descending I can also do it through name also clear any questions any doubt Yeah. And if I want to do it with multiple columns, how do I see it with multiple columns? Uh with one true and false, we will not be able to see it. If I want to do it with age and salary. Okay. So what I'll do is I'll add one more column with the same age. Okay, maybe here 22 salary is um 2,000 only and the name is maybe Sam. Okay, so that's the changes I made in the data frame. Learners, I'm just sending the data frame to you. Please make the changes in the data frame. So Sam here it descending Charlie that gets added but if I'm doing it for multiple criterias now let's look at over here age is true so this is the age is 22 for both all right and here is Alice and Okay. But now the salary is false. So the first one with higher salary is above than 2,000. Got it? Multiple criteria is getting understood. And now if I change it to name. So now Sam will come before that box. Is the point getting clear to everybody learners? And if I want to sort by index then I will just do df dots sort index ascending is equal to false and I can get this out. In whole of data analysis as I've been saying that statistics and the graphs are go hand in hand. So basic use of graph is to understand the distribution of the data. When I say distribution of the data, spread of the data and there is one more important term in all of data science, machine learning, AI is probability. Do we understand the term probability? What is the probability of getting head when I toss a coin? Yes, learners. Who's going to answer this? What is the probability of getting head? Okay, toss a coin. Nobody understands this. What what do we happen? What are the probable output when I toss a co to coin? It's tail or head, right? It's tail or head. And what is the probability of getting tail? That is equal to half, right? The number of success upon the total number of outcomes. And I talk about head that is also half. What is the probability of getting one when I roll a dice? What is the probability of getting one when I roll a dice? It is equal to 1 by 6 because the output are 1 2 3 4 5 6. Everybody understands this point, right? So when we are dealing with data, you know, practical data in real life, we are always talking about these two terms, the population and sample. The population here does not refer to the population of a country, city or state. Here is the number of items or group of people which are under the observation. Population refers to the group of items under observation and when we take a part of it you know or data that becomes my sample. So again this is an important concept in whole of machine learning and AI that we do not talk about the whole of the data 100% data. Why? Because it can be huge. It can it can involve a lot of cost as well as time. So when we take a part of the data sample we try to analyze through that particular sample. Getting my point right? Point over here is the statistical data distributions. Right? How my data is spread describe how the data points are spread out across values in a data set. Understanding these distributions helps in analyzing and interpreting the data. So in most of the graphs that we are going to deal with we are trying to understand the distributions we need to understand the revealing patterns trends and the underlying structures in it right now let's understand like what kind of graph are we going to draw what kind of analysis are we are we going to do so if this is my xaxis right we understand graph as twodimensional structure and this is my yaxis here it represents the probability because here we are talking about the probability distribution graph all right what is the probability of throwing one when uh in a when I throw a dice it is equal to 1 by 6 so the value is around 0.166 the probability of the two is also the same three is also same 4 five and six and therefore we get this kind of distribution which is known as uniform distribution. Have you understood this probability distribution graph that basically we are trying to find out what is going to be the output probable output getting my point? Is this point getting clear to everybody? Right? Let's understand it with another example. If I roll two dices, right? What is the probability that the sum is going to range between 2 to 12? So, the total number of outcomes when I throw two dice could be 36. Agreed learners? The minimum value of the sum is 1 comma 1 because when I throw both the dice the sum is 1 comma 1 and the maximum is 6a 6 that is 12 clear. So the probable outcomes are 1 comma 1 1 2 to 1 comma 6 right? Then we have 2 comma 1 2a 2 and then we have 2 comma six and further outputs. Clear? Is this point getting clear to everybody? Right. So what is the probability of getting the sum two? It is 1 by 36. What is the probability of getting three? It is 1 by 18. So on xaxis we have the number of events. The values can range between 2 to 12. Sum can never be 0 and 1. And on the yaxis we have the probable values. So this kind of distribution is known as a normal distribution. So how are we able to understand the graph is explaining this and this type of graph can anybody tell me what is this graph known as? Anybody who understands these graphs? Is it a bar graph or is it a histogram? Yes. Tell me do what do you think this is a bar graph or a histogram? A bar graph, a histogram. What is a histogram? Bar graph is used for categorical data. Histogram is used for continuous data where we calculate the number of class intervals. So this is the power of visualization of graphs. We understand different types of distribution. So that we have these two peaks. This is known as bi model. Here we have this comb distribution. We have this edge peak extra value coming out. This is how a normal distribution looks at. This is imbalanced distribution which is known as the skewed distribution. And this is nothing but your uniform distribution. Got it? Is this point getting clear to everybody? Marauti, Mana, Manvit, Nagendra, Vehav, Ashwaryia, Ranjan, Rahata, Shrea please let me know. So now let's understand data visualization that is visualizing the distributions of our variable is an important step in exploring and analyzing the data. And therefore the first basic library that was built was built by John D. Hunter in 2003 that is known as Matt plot lip which helps us to draw different types of graphs such as the bar bar graph, histogram, scatter plot, area plot as well as the pie plot. Clear? Right now to understand little technicalities of this mattplot lib library. If you look at the architecture of this mattplot library, it consists of three layers. At the lowermost layer, we have the back end layer which consists of figure canvas layer, renderer layer which knows how to draw on the figure canvas as well as event layer which can handle user inputs such as the keyboard, strokes and mouse etc. Above this layer we have the artist layer which is known as the primitive layer with line, rectangle, circle and text or the composite layer that is axis, text and figures. And on the topmost layer we have the scripting layer also known as the pi plot. very very important layer because this is the layer on which we as users work right at the back end uh you know the library handles all the mouse rectangle circle text etc at the top most layer we are pass commands to draw the different graphs getting my point and that is why we always do import mattplot libby piplot as PLT. We do not need to load the art artist layer or the back back end layer. We use the scripting layer. Clear. Then we have anatomy of the mattplot plot that is we have the center you know the title of the graph in a graph main figure. We can also sub up graphs. All right. With their own titles and legends. Do we understand the term legends? That refers to the key that the dark blue uh you know data points are referring to the zero value and the green turquoise one are referring to the one. And which is the command which helps us to create these subgraphs more than one graph in main figure. The command that is being or the function that is being used is the subplot. Right? The different arguments of this subplot function is n rows, n columns and index. So what do we use? We use the number of rows, number of columns and the index. Now what do I mean by that? How many graphs do you see in this one figure? How many graphs do you see in one figure? Six. Right. Can you tell me the number of rows in this particular graph? Can you tell me the number of rows in this particular graph? Yeah. So there are total number of rows are two and columns is three. Right. So every subplot function of rows and three columns. Two number of rows and three columns. Two number of rows and three columns. And this refers to the my first graph the index. This refers to my second graph. This refers to my third graph etc. Let's start with data visualization. Before that, let me give you the link for the MAD plot li. All right. So, data visualization is nothing but a very very important part of data analysis. It is a graphical representation of the data to reveal patterns, trends, insights that might not be easily apparent from the raw data. It involves creating visual elements such as charts, graphs, maps to communicate complex information. So it helps in our understanding. And we understand that mattplot li is one of the very fundamental libraries uh for data visualization built by John D. Hunter which helps us to create static, animative and interactive visualization. Different types of graph we will practically uh draw over here such as histogram, pie charts, subplots, tree chart, bubble chart, bar chart etc. All right. So one of the very basic uh you know graphs or chart is uh line plot. Yes learners you remember we had discussed the line plot on the very first day right. So how do we go about coding? Everybody has got this file. So the first step that is required to create the graph is that we need to import this mattplot lib.plot. Why piplot plot? Why do you think that mattplot li we are only importing the piplot? Tell me learners why pi plot. Tell me why piplot. Yeah, because it's the topmost application and it is the scripting layer in which we as users work. No, we are do not work on any other layer. It is this layer on which that we work. Getting my point? Got it? Right. And then importing the numpy. Do we understand the function np.linsspace? Yes, learners. Do we understand np.lindspace? What does a lens space function do? I say 1 comma 10. Do you see it creates values 1 to 10 equal values and by default it has created 1 2 3 4 100 values. So if I give a comma of five now you see what does the np.linsspace function do? It will create values between 1 to 10. the first and the second is a starting point. But well, equidistance value 1 2 3 4 5. So five values have been created. And if I give it as 10. Now we see that the values between 1 to 10 all equidistance from each other. Getting my point? Everybody is able to understand this? So here we are going to create values from 0 to 10 and it will generate 100 values and based on these 100 values I am also getting my y value for each of the x clear is this point getting clear to everybody and which is the function which I am going to use to create a graph it is plt dot plot. What is this function? It is plt do.plot. This is x and y. Label is equal to sine wave. We will get the output. Color is blue. Line style is dash and the color width is equal to two. Right? These are the certain parameters that can be passed to make the graph better. X-axis we are using the x label. Yaxis we are using the Y label to label it and the title. Okay. So this is the legend and then we have the plt dot show graph. Please look here how we are getting the output. Do you see this? This is the legend. This is sine wave. Right? The title over here is line plot is equal to sine wave. X-axis we have the x-axis. Yaxis we have the yaxis. Got it. Can I change the style? Yes, I can change this to red. Then I can uh change it to maybe a dot and a to the scatter plot. Please try to understand the significance of each graph. Why do you think we have so many graphs? Scatter, line, bar, pi. Why? Why do you think? Who's going to answer this question? Why do you think that we have so many graphs? Can we learn? Why? Why? Why everything cannot be represented with the help of line plot? Because every graph, every data we need to analyze the data differently. Every data is different. Right? Somewhere I would like to know the distribution. Somewhere I would like to know the pattern. Absolutely correct Sajj. Comparing two things we might face difficulty for better understanding. Right. And absolutely correct webhuff that due to the changes in the context of the data that is why different graphs are required. So please try to understand what is the significance of each graph. All right. So a scatter plot is used to display the relationship between two continuous variables. So scatter plot is a graph which gives relationship between two numeric values. Each point on the plot represents an observation and the position of the point is determined by the values of the two variable. It is helpful for identifying patterns, clusters and trends in the data. All right. Are we clear with this np.random.rand function? Whenever you have uh you know confusions of this kind, I would always suggest you to run it separately to understand it better. So let's run it separately. So rather than 100, let's give it as 10 value. So it is creating 10 random variables value lying between 0 to 1. Do you see this? So in this case it will generate 100 random variables and y is just the 2 into x function is not plot now we are using the scatter function in which I'm passing my x and y variable color is green marker is o and label is equal to random data along with the x y and the title of the graph. Getting my point? Abijita Saraj web can I change the style of this? Yes, I can change the style marking of it O2. I can make it as D or a triangle and then change the size. Let's understand the difference between rand and rand N. Now when I do a rand n over here, rand n stands for normally distributed data. What do we mean by normally distributed data? That it will be like looking like this uh peak centrally balanced data but mean equal to equal to zero. What does that mean? There will be few negative values and positive values. Okay. So here it was rand gives the values between 0 to 1. Random here the values can be positive and negative but the idea is to have mean equal to zero and standard deviation equal to 1. Clear? Any questions? Any doubt? Any other question? Any other doubt learners? Are we clear with line plot? Learners, are we clear with line plot? Are we clear with the scatter plot? Now let's move to the bar chart. What is the use of bar chart? That this is a common way to represent categorical data. Very very important, right? And it uses the rectangular bars of length proportional to the values they represent. This chart is useful for comparing the sizes or the frequencies of different categories. So a bar chart is a common way to represent categorical data. It uses rectangular bars of lengths proportional to the values they represent and useful for comparing the sizes or the frequencies of different categories. Getting my point right. So here we have created a list of categories along with their values. which is the function which is used to create a bar chart plt.bar bar where I pass the first parameter the x-axis as categories yaxis the y values of yaxis color is equal to orange edge color is black and the label is bar chart and now are we clear the x label gives the label for the x-axis y label gives y label and title gives that clear learners You all getting this point Las? Yes or no? Spine getting clear to everybody. Now coming on to the very important graph which is known as the box plot. Now let's understand the box plot. Very very important graph. All right. So a box plot looks like a box right and it also helps to give distribution of data. Here we are trying to compare a box plot with its probability distribution. Right? The box itself gives the data between Q3 and Q1. Q1 refers to the first quartile or the 25% of the data and Q3 refers to the third quartile that is representing the 75% of the data. And if we look at the interquartile range that refers to Q3 minus Q1. All right. So it is a very very important graph because it gives the five number summary that it gives me this minimum value. This whisker is the minimum value. This vertical line gives me the Q1. The center line gives me the median or the Q2 value. The third vertical line gives me Q3. And finally we have this last whisker which gives me the maximum value. And the values which lie beyond this that is Q1 minus 1.5 into IQR. These small circles are known as outliers. Very very important term in whole of machine learning and data science. Outliers are nothing but extreme values represented of the data. Clear? Is this point getting clear to everybody? Getting my point right? So the center point over here is like the 50% of the data right and anything that lies beyond this can be considered to be away from the median and if it is lying beyond this whisker these are known as outliers. So isn't isn't it giving more insight to the whole data set that which are the data points which are lying in the center which are away from the center and which are extremely far away from the center point getting my point where nana esaria right and the box plot also has this notch function right this notch function tells us that be a 95% confidence of the median value that the value will lie between these range. So this parameter can also be passed to the box plot function so that we can know the range of these particular values. Clear? So here we have the lower whiskers greater of the 25th percentile or the minimum value. Right? And the second is this refers to your Q1 value, the first quartile. This is your median. This is your third quartile. And the upper whisker and values lying beyond this become my possible outliers. Right? So what is the use of notch? It gives me that this is the confidence that the medians are lying within this value. Clear learners. Is this point getting clear to everybody or any questions? Any doubt? Only Se is understanding. What about others? So what is the advantage of box plot? It helps us to do comparison. If you look at A and B, the median values are lying very very close to each other that A and B are lying within this particular range. And if you look at C and D, the C value, the median value is much higher than the value of the D. The C value is much higher than the value of the D. Getting my point clear. Are you getting this point learners or not? So let's get back to it practically. So a box plot is also known as the box whisker plot which provides the graphical summary of the distribution of the data set. So box plot is a very very important graph. I'm I'm sure you might have not heard about this before. Anybody who's heard about this box plot before? Anybody who's heard about box plot before? So a box plot or a box whisker plot provides the graphical summary of the distribution of a data set. It displays the minimum, the first quartile, the median, the third quartile and the maximum value. Another important feature that it is useful for and it is generally used in almost all the analysis is useful for identifying outliers and understanding the spread and the central tendency of the data. Clear? So again I am using np.tr random normal. So again I am generating this random data. If you ever have confusion I as I always tell you open copy the code paste it into a new file and then see it creates 100 values with mean equal to zero and standard deviation equal to 1. Right? And if I change it to 10 and this to 1.5 here it is generating 10 values with mean equal to zero and standard deviation equal to 1. Clear? So here we are trying to create three different types of data with all values mean zero but standard deviation of the first one is one second one is 1.5 and the third one is two and then give the labels and here I am giving patch artist is true and notch is also equal to true therefore I get this kind of graph Right. And if I change this to false, you see the notch has been removed. Learners, do you see the notch has been removed? The moment I make notch is equal to false. So it gives me a warning over here that the labels parameter of the box plot has been renamed as tick labels. So this is not labels anymore. It is whatever changes it suggests one should make it. So now run it. You see this the warning has been removed. It says false and the patch artist is removed. The color is also removed. Can you play around with this particular graphs? Now learners, now let's understand the comparison between radar and polar chart. Practically this graph is used a little less because it's more used for navigation and for more scientific purpose. You might have not heard about this radar and polar chart but definitely web chart, radar chart, spider chart, it is used for comparison for multiple variables. Do you see this? This is used for comparison of multiple variables. Red, gray and orange. This is variable 1 2 3 7 and 8. Right? So if there are 18 variables, this this graph will be you know extended horizontally. But do you see just just imagine that this same uh you know eight variables can be drawn with the help of this radar chart. So it is again a multi-variable graph. It is used for comparisons of different variables. Getting my point right. So the elements of the radar chart are the axis. This refers to my axis over here. The circular refers to the grid grid lines or the grid points that we need to pass. Clear. These are my data points, the label axis and this is the legend that over here the the light blue is the unresolved one and the dark blue is the dissolved one. Clear? Is this point getting clear to everybody? Any questions? Any doubt till here? Right. This uh these are the elements of the radar chart. And if we look at over here another way to use radar chart especially for navigation purposes for launching of satellites, your airplanes, your ship navigation. This is how it is being used. Now the center point we have this static uh point. This is the grid grid points. How much angle are we away along with the radius? So sometimes the angle and generally the angle is in radians. Generally the angles are values are passed in radians. What is radians? 2 pi. Multiplying by 2 pi is radians. Do we understand radians? So a polar chart is represented by its radius and the angle in radians. Getting my point? So a radar chart is also known as a spider chart or a star chart. The basic uh advantage is that it helps in analysis of multivaried data with radar chart. However, each radial axis acts as a dimension allowing us to plot the multivariate data. So it acts as a dimension to plot the multivaried data. A radar chart uses