AI for Visual Data: Computer Vision in Business

Key Takeaways

[Music] [Music] o hello everyone and thank you for joining today's webinar my name is reys and I'll be your moderator today we're going to kick off today's session in about four or five minutes or so we're just waiting so everyone has a chance to join in the meanwhile though we'd love to hear from you so let us know where you're joining from using the chat or the comments depending on what platform you're watching from and yeah tell us something that you'd like to learn from today's webinar if you have any questions at all throughout the session then please let us know using the chat and yeah I will be saving them for our Q&A um if you haven't already then get registered for the event today and we will send you the recording as well as any other resources that I'll be sharing as well uh I'll be posting a link to that shortly in the chat so keep your eyes peed for that uh but if you don't catch it in the chat then you can go to datac camp.com webinars brilli I'll be back to repeat these uh messages for any new joiners shortly but until then enjoy the background [Music] music [Music] hello everyone and thank you for joining today's webinar my name is reys and I'll be your moderator today we're going to kick off today's session in about a minute or so we're just waiting so everyone has a chance to join let us know where you're joining from using the uh chat or the comments depending on what platform you're watching on and yeah let us to know that something that you'd like to learn from today's webinar if you have any questions at any point for any of our panelists or for all of our panelists uh then please let us know using the chat or the comments and I'll be saving them for the Q&A we'll be running a Q&A towards the end of the session but yeah if you have any questions at any point uh then please let us know as soon as they bring to mind if you haven't already then please register for the event and we will get the recording and resources sent to you uh via email um the link to that is in the chat I'll be posting it again shortly but if you can't find it in the chat then head over to datac camp.com webinars brilliant I think that's everything for me so now I'll hand you over to your host for today's session Richie Richie please take it away welcome to the webinar this is Richie now until recently doing data science with image and video data was incredibly difficult our advances in tools have mean have meant that visual data is becoming increasingly important for businesses as AI allows you to extract more value from it now there are still a few challenges and subtleties uh to this so today I've rounded up three experts to help you navigate making use of this data type San piter is a co-founder of acur data an AI platform for visual data where he heads up the data exploration uh business uh product and goto Market strategy he's a veteran of several startups and was previously an investment banker at be Sterns now vishaka gup clat is the co-founder and CEO of aperture data a visual data management platform and she previously worked at Intel Labs where she led the design and development of their visual data management system and uh Natan is a is the computer vision team lead at DES is a deep learning development platform he leads the uh productization of Desi's AI technology so between them they've got a tremendous amount of knowledge on computer vision and I'm getting very excited to learn from them so uh welcome to all three of you uh I'm I'm actually really excited for this discussion so I guess uh just to kick it off um can you uh tell me like one of the most common business use cases of visual data uh who wants to go first uh Nan do you want to go first yes hi everyone so um visual data of course is very very common for computer vision algorithms ACR course a wide range of business use cases H here at Desi we serve customers across uh across the world um recent uh use cases are Advanced Driver assistance systems Adas Automotive companies uh Quality Inspection for manufacturing uh also shelf monitoring and automation checkout for retail and these are just few examples that's pretty cool um I like that there's a few different Industries in there um PKA do you want to add add to that what use cases of using yeah thanks uh thanks so much hi everyone so uh in terms of retail for example you know out of um other than inventory management you have Shopper insights you have cash at lless checkout um in case of visual inspection it's also detecting defects or worker safety uh e-commerce we have we actually uh work on um visual recommendations so recommending based on the features that you see of the product and automated attribute classification there smart agriculture detect good crops bad crops pests harvests uh and then there's Medical Imaging where you help uh where you know to help experts diagnose patients faster so um in addition to what Natan said there's all these other other use cases that we uh work to work on as well see that um agriculture example is really interesting I always think of Agriculture is being low Tech you know people looking about in fields but it's absolutely not at all is it there is some very no I mean inide people end up getting uh data from satellite imagery they end up getting data from drone and then they have their own cameras and other sensors so in fact they do a lot I mean depending on of course the scale of the company there's a lot of things that happen and especially the ones that walk around and Harvest crops so there they have to detect what's the good part of it what's the bad part or if they're spraying pesticide then which is the area they should spray it on that's fascinating uh and Sanjay do you have any other examples to add uh well actually I'll flip it other way and say you know if you were to just post the question as to how many cameras are there in this world it's 45 billion in counting right and that yes that includes your C phones and laptops and things like that but even if you strip them off right the sheer amount of cameras that are deployed kind of transin across Industries right um and uh the volume produced there uh is also substantial for example if you take just a consum u consumer produced content alone which largely let's say YouTube as a proxy you're talking about a one exabyte of data and Counting right so besides the industries that they have covered uh one additional piece is also the defense um because M surveillance in that context as as well as um you know aial Inspire uh uh sort of satellite and drones which I think is covered um heavy emphasis on manufacturing too because historically you know the whole automation of manufacturing has been happening for the last 25 30 years but now there's a a shift from the classical computer vision to a more deep learning approach uh technique so uh I think pretty much there isn't an industry that is uh where visual data or computer vision does not apply okay uh that's very inter I just seem like it's completely ubiquit can you just expand on that last point about um going from classical computer vision to a deep learning approach what does that mean yeah so the classical computer vision is largely based upon a very templatized approach very bespoke models are built let's say You're Building uh your in your producing a widget and you're inspecting in line on assembly line you essentially look and you have built a model that looks for the shape and structure of the widget very purpose built right now such model works great when it when you're looking at the very stereotypical set of data compare that or contrast that with uh uh something that's mobile autonomous beat a robot walking around or autonomous cars uh you have to deal with a diverse set of environment right people place things things are moving all around and you're making decisions so uh for that the Deep learning techniques which the underpinings of that is your neural network CNN and things of that nature other the foundations for that right so it's a very different environment where uh in the latter case the data becomes even more come to prominence because models are becoming more of a black box and you just beat it with enough good data and the the models of the deep learning models uh yield better okay so it's going from just can you deal with like a few specific uh objects to can you recognize any kind of object that's an interesting development all right so um I want to talk about uh uh how data people get involved with this so uh can you tell me about what um sort of problems data scientists or machine learning um Engineers might be trying to solve with visual data so how do those people get involved um uh I guess Natan you've not spoken for a while cool yeah um perhaps if I may add to what San said H in the classic computer vision you solve the problem by designing features while in deep learning approach you just solve it by learning from a lot of examples and this is why you need vast amounts of data okay um that that makes sense um all right uh so uh just to go back to that sort of question like um where do data scientists get involved in the sort of the visual data process um so um sorry go on vaki you were ke to dive in there oh yeah no I was going to say like um uh in in terms of like you know where the data scientists come in typically like you know we when we took examples of uh inspection uh you got to start training models to detect defects on a manufacturing line uh or or you know uh warn of some hazards in uh a factory floor so there's training the models extracting and learning how uh it's performing so like how accurate it is compared to uh the data trained on you try on some validation data then very often we encounter cases where you know data scientists are working with embedding so like how good the embeddings are results in how good the Cann or the similarity searches tend to be and so you know tuning that there's a lot of working together with like understanding the data and then making use of it like making sure it's um distributed like it's representative of all the things that you want to detect and train on so um that's typically where we've seen data scientists or machine learning scientists work on and you know they are directly trying to solve for either improving the customer experience for their companies or uh worker safety or just improving the way the resources get utilized okay um you mentioned the idea of embeddings being really important there can you just explain what an embedding is should I all right whoever wants to explain what an embedding is I mean like the very straightforward way I think about embeddings is like let's say I need to describe our faces um you know there are different features on our our faces like if nose eyes the distance between the eyes uh the color everything if you start assigning numbers to them and then put that together in an array that's kind of like that feature Vector that can describe my face with numerical values and it tends to be smaller than if I were to capture the entire like all the pixels on on my face um and that's you know that's like my simplest way of thinking about Vector embeddings I you guys can make it more machine learning specific Sanjay you look like you wanted to jump in there well I'll probably give a little bit uh a simpler way of thinking uh think there are three stars in the sky right star A and B are what looks to us as visually close to each other and star see the third star is further apart in real world the second star could be really several light years behind right but our eyes perception is these two stars are close when reality is it is the other way the first star and the third star of close teacher now what I'm explaining is three dimensions and what Wasa is saying is an image could be a high Dimensions right something that we cannot see it um but how do you represent uh the Salient features right which is truth if I take a picture you will have a false illusion of A and B being closed but in reality I need to surface the fact that A and C are actually the dots that are close to each other and B the furthest apart right now extrapolate this example into 2,000 different dimensions and that is sort of the representation of feature vectors and the Salient aspects of that presented in a 2d edings and I think you combine both our explanations to really start thinking about how they get used so if I understood this it's basically it's about turning your your visual data your images or whatever into numbers so then you've got you can do like your um your numeric computations to see which things are similar and which things are different um all right okay so uh can we get to some examples of like what what success looks like so have you got any examples of companies that use visual data successfully um who wants to go first no one's looking excited um sanj you're looking like well you know let's start with auto yes autonomous vehicles are the Holy Grail that every company is chasing for however uh Auto you know has been uh evolving these driver assist functionalities right some many of us who are buying in the recent Years cards you're probably starting to see uh adaptive cruise controls Lane shifting and all those things all those things really happens with some sort of a computer vision that right and of course you probably on the other Spectrum you seeing the Whos and Crews who have sort of uh you know pretty close to self-driving cars which is you know which is true representation of a a transformation just rewind 20 years ago and see what the cars that you own today or see in the market um it's a big transformation that people uh can associate with um of course there are several examples but um I'll let my friends speak to it all right uh yeah self- driving car is definitely a cool example uh Nan you've not spoken for a while do you want to give us some examples of um what constitutes a success with visual data yes of course um so I think one example from new angle is uh responsive PDF files I guess we all know what PDFs are our cell phones and we know that it's not very con convenient to read PDFs on cell phones right but in recent years we have responsive websites that act responsibly but not PDFs and uh this is something that uh we did recently and it works great and uh I think you can use it now by the way with adobe's liquid mode on their acrobat app so this is uh one example that comes from the engineering part the computer science part where you use visual data although it's a bit synthetic and so this is one example and another example explain so the effect is it's what it makes it easier to read if you're on mobile versus if you're on um web is that the idea yeah especially if you read the scientific uh papers they are like column papers when when you have them responsibly arrange it's much easier ah yes uh multicolumn pages on mobile that's horrible to read I agree okay uh you had one more example I believe um yeah we can also talk about uh Applied Materials which they inspect uh chips so this is another example of using deep learning models to in insect chips where you don't know where the uh defects are so this is yet another example of using visual data ah okay so you're looking at images or video of chips and see where the problems and is that more uh broadly applicable to other kinds of manufacturing yeah yeah it's starting to be okay that sounds cool uh and vaka do you have more examples of um who's doing things well yeah so we uh one of the companies I can think of is SAA Technologies uh they're scaling Inventory management and planogram compliance so you can like you know in in the Northeast particularly you can see their robots walking down grocery store aisles every morning kind of like figuring out are the shelves stocked are they correctly uh the items correctly placed then there are companies like zien that are growing their Casher lless checkout stores uh so that you know people can quickly zoom in and out so it's um along the lines of Amazon go but in in a lot more places like stadiums and things like that um and and you know a lot of some of the large retailers are having uh good luck doing visual recommendations on the e-commerce side and shoper insights on the smart retail side um you know you can see papers and talks out of Home Depot Walmart uh then companies like Merc uh some in energy Aerospace and Medical Imaging Insurance in fact Insurance analysis like companies like CCC intelligence Solutions so there's quite a few of them that have managed to do very well uh despite whatever challenges we are going to discuss about um and and yeah and they've deployed this in uh production I would say okay that's very cool I like that well I'm not sure whether I I be like the idea whether it's kind of creepy having robots walk through your super market with you but yeah Inc useful they've made quite the fun experiences you know they call their robot Marty with the eyes on it and it even like goes around parking lots to say hi to people you should check out the videos nice okay yeah that sounds like something worth checking out um all right so um I don't really have a strong sense of like what are the easy things to do what difficult things to do with computer vision so um for organizations who want to get started with this like what's a good first project to do um uh Natan do you want to go again uh yeah sure so um generally image classification tasks can be relatively simple while detection or segmentation tasks are considered a bit more advanced if if you also think about it h for image classification the output is one scalar or the class name which basically means that you to uh consume the the image and output something very uh small and specific for object detection you output bounding boxes and for segmentation you output a segmentation mask so image classification generally is very simple task relatively simple task but um nonetheless every task will be difficult if your data is biased if some classes are under represented H for example for object detection if you have very small objects then this is also hard and I think that one step before uh you do any task you need to make sure that your data is fine because then your chances to fail are higher okay it does seem like we do need to have a longer discussion about like uh getting the right data and make sure that you haven't got biased um uh biased results um I do find that interesting that you said that um image classification is kind of the simplest thing because you know you're just outputting text rather than uh doing something more complicated um all right uh Sanjay do you have any ideas for like a good first project yeah I concur that you know probably classification is the easiest one to get going and probably but today you know U there's so much um off-the-shelf models that one could get going that's a beauty about it uh Gone other days you had to build something from scratch uh take some you know you probably heard of YOLO which is a very popular model they're lightweight versions you can just log in you know to hugging face and pull a model a lightweight like a YOLO egg uh Yello X detect you know you can potentially run it in your own laptop put some images and see the response out of it right so that's a great starting point for many place but uh really getting down to work it for uh your environment your business case uh will drive the choice of models that you want to go with right um now that requires uh some level of knowledge of what each models that off the shelf models are uh where designed and trained for um and then you choose the closest model that relates to your domain of data there's one thing that's detecting a coffee mug and uh my laptop and things of that nature in the table versus uh doing a detection of the grocery store example that vishaka was mentioning right you have wide range of sauce sauces to jam bottles to toothpaste so which one are you going to pick right and but U as I said earlier you there's a easy way to jump start your project and then choosing the right data uh from your set and being able to train and uh adjust the models is going to kind of yield pretty good results of the shop okay so it sounds like sort of agreed the image classification is the way to start and the tricky part is deciding what you can to do image classification on so um vishaka do you want to talk about like how you get from I'm going to do an image classification project so I'm going to get to I'm going to get some value out of this yeah um and in fact yeah that's what the codal along session is going to be uh in a few weeks but uh you essentially like you know I was even going to say a step further so when sanj alluded to the fact that you're GNA have to see your use case in fact I would say uh you have to see what data you have that is accessible to you that's usable the quality of that data so like the step like that's like sometimes the challenges start from even like collecting or gathering together because it's usually scattered across a lot of different sources in a especially in a large company so just getting a sense of what the data that you have is how well is it represented how much is it labeled um so you know like you would essentially have to start from there and then like San was saying there are now models that are very easy to you know access and run through uh for so for a small example this like literally step by step okay this is where you put your data set this is where you unzip it this is the training set this is the validation set and you take you know install ports choose a model and get going um it the tricky part starts to be when you want to do this at scale then you have to start first with like um like storing the data someplace you can access being able to search creating data sets and then um you know you see how an off-the-shelf model would perform on this data and then you start to see like okay well this is not the desired accuracy so now you got to train it on your data so that it starts to understand more of your uh whatever images or videos you have in your organization and that kind of takes you back to okay do we need more label data so like kind of get it labeled if it's not a lot uh that you need labeled then you can use some open source tools like you know we been building demos with label studio for example um you can label some certain things it lets you label bounding boxes polygons and then you can create data sets that you know put data along all these things so let's say you want to recognize Jam bottles and uh serial boxes and whatnot so you'll have like enough images in each category labeled that way and then you retrain the model and then you start seeing so you have to be able to like look at the data before and after see if your model is um uh you know how well it's doing and it sounds like a lot of steps but I think like there's now enough tooling along this entire Spectrum to kind of simplify and and a lot of it is pretty easily available that is great that you can just get started with off the-shelf tools and get some value pretty quickly and then you sort of worry about it sounds that you worry about performance a bit later on just trying to get like the best performance um after you when you have to deploy in scale or when you have to do it companywide and you actually go from you know research project to um more of a okay we're getting good results now can we see this on our entire data and then you're talking in some cases like millions of images or millions of videos that you've collected over time and that's when you start to get into like uh scale and how can you launch things in parallel and the classic systems problems show up for ML infra then okay yeah so deploying is the trick right sorry I would also argue that um while off the shelf models do give you a decent performance you should care about production from Day Zero because otherwise you will end up with some Legacy of the Shelf model that you're stuck with it and you will get uh decent uh uh accuracy and this is only because you started with off the shelf and then you kind of rolled with it also um as sanj and vishaka mentioned data is very important but you also need to uh suit the architecture the neural network architecture to the data itself and this is also a crucial part basically if you think of it as a a logical gate it is an end gate you need good data and good model yeah all right so since good data has been mentioned a few times Let's uh get into that so um where do you get this good data from like who should be responsible in your organization for collecting data for ensuring data quality things like that um I guess it's going to vary a lot depending on the use case does anyone got um any tips on this uh sanj do you want to go sure uh so actually depends what we have seen in uh companies big and small the behavior is slightly different right uh the but uh the indisputable fact is most often these data are produced in a remote area compared to a data center or a cloud where all the data is stored uh pre-processed things of that nature but if you start from the source be it a factory floor where you have a camera deployed or a camera in a retail store um they all have produce way too much data right and not all data is actually relevant to you so you there in in a larger organizations there are dedicated teams that are managing from both from the infrastructure point of view where do I store this data what portion of data can I move through the network to a central place uh and then there is whole set of data engineering data curation teams that are charted with what data have I collected of which what is that I'm looking for because most of them are garbage and when I'm trying to teach a model to detect something X I need to find enough examples of X and variations of X and picking that and having it labeled and labeling is typically done by outside vendors but for sensitive projects people do it in-house uh so there are separation between a labeling team and curation teams in some cases they're all bundled in one right so that is kind of your uh the the source to data preparation phase before your training kicks in okay um sorry I can add um so you know combining what Natan and San said here good data also depends on a lot of um understanding so you know as he as he was alluding to it comes from somewhere else in case of e-commerce maybe not an edge collection uploaded by merchants or you know you take pictures with uh uh yeah I guess in some ways that's Edge but there are a lot of teams involved and an understanding of what you're collecting and uh what do the metadata descriptions mean about all these things so like you know for any of these images of videos there always be some descriptive metadata even some things as simple as we collected this at factory 5 from camera 10 at this time that's like very fundamental example of metadata that goes but we've seen cases where um you know the data engineering team defines that one of the fields as a string and the data science team assumes it's in integer it's like the classic you know problem around the the understanding of data getting messed up or uh literally when people are passing around Cloud buckets across these stages then you know did you name the bucket correctly or someone named the bucket that person left what happened and so good like the the the quality of the data how much do you collect how much do you replicate it are you working on the right version of it um those things start to become like something you have to pay attention to especially like Nan was saying if to think about production from the beginning otherwise you have to plan a lot of time to actually like redo all the things that you did but now in production and then you wonder like oh it's not what we expected um so that's s like I I would say like solving for the good data problem should kind of span across not just like is it distributed in all the classes that I wanted to be represented do I have enough examples correctly labeled um there's also like are my labelers doing a good job so um there's a lot of like validation steps that should go in along the way um it just have was a lot of room for error there and you do need to be very careful I guess in terms of just like simple like data governance type stuff uh just make sure that like all your processes make sure that um yeah you can even find the images that you've stored somewhere in a ploud bucket um so actually earlier Natan you mentioned the idea that um you need to make sure there's no bias in your data set can you expand on that and tell us what you mean there um yes so uh I think that the most common uh mistakes is when you split the data into training and validation sets I assume that everyone is uh knows what training and validation sets are so when when you split the data um you need to make sure that the distributions match and not only the pixel distribution but also class distribution and also location distribution also a bounding box size distribution if we're talking about uh object detection um for example when we do uh autonomous vehicles so it's also important that the distributions of the colors and the brightness will be the same across training and validation sets because especially for autonomous vehicles when you drive at night night then there are no bright colors of course rather than lights and when you drive a day then most of the image is bright so you also need to carefully split these images correspondingly and for videos I think that you care about the temporal continu which is also something that people tend to miss um that's kind of interesting the idea that um something like with the autonomous vehicles it's going to work during day but not at night because all the colors are different and therefore yeah the model breaks that's uh fascinating um actually uh sanj do you have any uh more examples of like where buyers can creep into data sets and affect your model performance Yes actually one of our customer which is a very large uh tech company um they are actually having a dedicated team to focus on a buyer and there particular case um there a set of information the user about the user is posted so the individuals uh who are partaking as a customer uh in the website and implicitly the the racial bias uh is uh uh is there because the population distribution is of that nature uh but when you have uh certain um feedbacks from the market being let's say I'm giving five stars to you after engaging with you or maybe two stars and if there is a implicit pattern as how let's say mention if you so happen to pick a a training data set that inadvertently captures or has the skewed information what's really going to happen is when you are dealing with the real world data you're going to make certain adverse choices and in this world with social media and other stuff one mistake is enough and next thing you know uh company a is you know doing are taking certain actions that are racially skew which is incorrect which was not their intent but this also goes back to the fact that I mentioned models are becoming too uh complex for people to be able to pin it down what it's doing right so many Transformations are happening H and thereby the only uh way for you to control is feed it with the right set of data it really codes down to selection of data will have a cascading effect on how your model is going to eventually behave so to highlight the fact that this particular company has a team of computer vision guys and the job is to look for bias that is cool that there's sort of a dedicated resource there just to figure out um what's going on with bias and I like the idea um the example you gave about um when you're dealing with people um then you can end up with things like racial bias and discrimination um vka do you have any more examples of where bias can cause problems with computer vision um yeah I I I could think of um a medical image example like you end up training a lot in like you know your specific lab samples um and and you don't get to see every kind of like you know maybe every kind of X-ray or every kind of CD scan and then suddenly all these Corner cases start appearing uh and and the model is not ready for it so like you have to keep introducing like every time and that's why you know at at one point the general thing was oh you trained your model and you're done uh but now it's become more like okay well we got all these cases where it didn't work correctly and uh that's where like you know you have to retrain them to introduce all these Corner cases uh and then I think uh naan mentioned something about the lighting and you know earlier we were talking about the uh agriculture example like most of the times hopefully you only see healthy crops which means to see like you know crops that have pests or some problems you have to make sure like you're putting bringing that extra effort to find like you know one one of the questions I got as one at asked at one point is you know everyone talks about finding similar data we want to find dissimilar data because we want to make sure it's represented well enough um it's so that we're training on like a wide set of nonsimilar data and things like that so H so it seems like um the sort of Comon thre from all this is that um when you have rare cases of things then that seems to be um the hard thing to train on you get worse model performance is that sort of a good summary one of the one of the cases sometimes you just not collecting in one section right like what San was alluding to the face thing if you just don't have representations of people even though they exist you just don't have that shown in in in the data that has been collected so far then it's it's not because it's a rare case it's just because you don't have it it's just rare in your data set okay rare in your data set yeah so rare in that sense all right just actually for anyone in the audience who is interested in this idea of bias and computer vision the next episode of the data frame podcast coming out on Monday uh does deal with this topic so we've got uh Joy B and weie uh coming on the show so please do look out for that on Monday uh but back to the task in hand um so we talked a bit about some of the challenges of um actually getting these uh models to work are there any other challenges that we've not discussed yet like what else can go wrong what mes to people make um all right yeah we've only got like uh what 20 minutes left I'll narrow it down and then I'll I'll stop speaking on that I think uh like you know I think we already referred to the whole uh think about going into production from the very beginning so you you need to be able to uh manage like Define your data sets and manage them and not in a very ad hoc manner but like you know easy to replicate manner like often times when you're training let's say you apply certain augmentation to the data uh to get like you know to improve the quality of the model you want to be you want to make it repeatable it's not like only one data scientist can reproduce it you should like it should be uh representative across and you know how you do that is one of the challenges uh we already talked about consistency across but um uh the two big things I'll mention is when you go from a scale of let's say training with a thousand videos to you want to start training with 100,000 that's when you start running into systems challenges um that's when you're talking about how do you access these in parallel how do you deal with your network how do you deal with the costs associated with those trainings um so those are some of the other challenges and uh then keeping this data secure and uh you know privacy of this data so all these things kind of I choose those out of my list of 10 okay yeah I can certainly see how once you start getting to hundred of thousands of images you are going to have real big data problems to deal with particularly for videos and you know you have to split them and then you have to combine because your events run through across the splits that you did and then you get into dealing with all these computer vision libraries which sometimes can be a pain to install and deal with all right uh fantastic um sanay have you got any more problems that people need to deal with yes and actually just giving an analogy from my own customer one of our top customers produce a petabyte of data a month and run it through our system right that's a petabyte that run goes to our system I don't know how much more they actually generate a petabyte for people not too familiar it's kind of what 10 to the power of 18 bytes right and today uh a terabyte of data uh would you can pack about 500 movies in a terabyte th000 terabytes better and you can go from that right so so the sheer volume uh they went from a fraction of the volume that were generating to a petabyte within a year and now they're producing a petabyte a month right so that really tests everything that you had designed for which is um where do I uh collect this data what is the most efficient one petabyte in any even in the cloud would be exorbitant cost uh so the choices one has to make or made will be put to test from scalability of uh data store data inje and data processing side of things and you will be forced to pick a very small fraction of that data for your training purpose Because unless you're Google and apples who can afford to spend uh multi-millions of dollars in training uh most companies don't have that Liberty right So eventually you have to reduce down to a small sub data and this data depending upon the organization you could be collecting it from different GEOS could be coming from different factories and different GEOS or other use cases uh you have to worry about data residency because there are regulatory requirements as to uh the data Shall Not Be Moved uh from Europe to your central location in us right uh privacy because often visual data whether it's a storefront camera or a surveillance camera you're registering people's faces right and if you just go to Google Maps and look for the street view they off fisticated the license plate all those things we take it for granted but somebody has thought through that made sure that privacy is installed there's no buyas there is data residencies all of those things have to be factored in uh yeah just there's a lot of issues once you really at scale and um yeah I can certainly imagine how sort of trimming down your data set a bit unless you're running your own cloud platform like uh Amazon or Google or Microsoft okay all right Natan have you got any more um examples of things that can go wrong yeah of course um like everyone else here I will add another layer of of complexity so you have a lot of data eventually you want to train models with this data and training models with a lot of data is also very complicated you don't uh train your models only once if that is the case then you are a very lucky person most of the times you train tens or hundreds of models per day every model has different hyperparameter or architecture considerations and you also need to Version Control all of your experiments to visualize them to quickly understand which ones are more promising which ones are less promising and one very important thing is I think that vishak mentioned it earlier is that you need to have these experiments reproducible otherwise you will end up with one employee who is responsible for this model and another employee who is responsible for that model and you somehow end up with having a model that you don't know on which data it was trained on with what parameters it was train and you can't change it you can't uh modify it you can't do anything so this is one thing another thing is that people should I think think uh several times before trying to throw more data or to throw more parameters in terms of model size um one image is worth a thousand words and I think that a good image is also worth a thousand of bad images so you should also consider that before you uh start training on petabytes of data with a I don't know seven billion parameter model so a good approach will save you a lot of data and a lot of comput absolutely I mean certainly uh the idea of just tracking what you've done and trying to realize it uh you need to manage everything um uh just sound like great advice all right so uh we've got ton great questions from the audience which we'll get to in a minute um I do want to have a couple of last questions though before we get there so uh quick answers on these I think um so for people who are interested in getting into this field like they want to be um a computer vision professional what sort of skills and tools um do you need uh vka do you want to go first um I would say I mean just you know there is a lot of good uh there's a lot of good courses on deep learning that that you can get started with especially Hands-On anytime you can do some Hands-On uh trial of these things because it's all these collab notebooks that you can launch and start trying um but you know in general I think it just like everything if you're going to fundamental what the tools that you should be good at just get really good at python that's not that that's something you're just going to need for everything uh and uh and then just start trying like you know go to like there's a lot of models that you can try from hugging face there's a lot of data you can get from C just combine those things together and start working uh and start building the examples and of course when you get to you know all these challenges we talked about that's when you're doing like a companywide like you know you're doing something for the company but if you're learning I think just getting started from uh depends on how deep you want to go so doing some of these classes on uh and I think data Camp has a lot of resources to learn as well so just learning through that and as much as you can try Hands-On workshops or like some of your codal along sessions and stuff absolutely yes please everyone please do come back to the code along uh yeah uh and if you want to Learn Python you're definitely in the right place here on data Camp uh all right Natan uh any other skills or tools you think people need to know about um no actually I I think that vishaka summarized it perfectly you need to know python you need to have a a basically a will to to learn new stuff everything is online everything is very accessible so just go ahead and try absolutely um Sanjay uh anything to add on skills or tools or is that it it's just a load of python I I guess so uh I think I know py talk just mentioned earlier is that an important thing in this context so yes it is one of the most popular uh Frameworks that you go with but if all of these things are a bit overwhelming where do I start if I don't know python on if I don't know the foundational mathematics behind it all those things you can take it from a business first approach or let's say I want to just kick the ties on what is this all models and things of that nature uh you would be surprised just go to chat GPT and can actually be a great assistant to just say walk me through the steps of how to uh install a model for example right it actually does a pretty good job um and It'll point you to some of the popular models and the next step could be well you can go to the as extent of I don't even know how to install a model right or or where and if you're not familiar with python then you need to install packages and things of that nature every one of these steps today we have that information not to mention gazillion Publications in uh medium in fact when I Googled up about YOLO I recall seeing about a good public uh depth of material uh in data Camp itself so too much information uh but if you want to start off small I would encourage just uh and like I mentioned sign up for an collab account um and if you start off with some basic instructions that are given in a uh jumpstart python code if you don't know that just keep a GPT on the side Buton P phrase it actually gives you the the you can learn probably python in a faster way um having a GPD as a assist and I do that with my daughter so that's how I'm able to relate to these things I was just thinking with the hackathon we did with a team you're trying to use the segment anything model and like you know con like get polygons and then like out of that chat GPD was quite useful to get started excellent yes uh andone perhaps if I can add add something very short so um the mathematical background is important but it is not the most important thing this is a second order thing that you should uh learn uh so don't be overwhelmed of the amount of math there this is only when you want to take your skills to the next level and actually um improve yourself improve your models and improve your understanding absolutely so my background is originally mathematics and I like people to know things about mathematics but yeah on a day-to-day basis it's slightly less important than understanding the code and the concept um so uh just on your point San about using collab and GPT on the side for people who are interested in uh a notebook then please do use data Camp workspace which is a hosted jupyter notebook and does have GPT built into it to uh help you uh uh write your code all right so with that uh we've got uh 10 minutes left for audience questions and we got a lot of cool questions here uh the first question comes from Muhammad saying what's the best AI server I guess do you want to pick I'm not sure whether it's server or models tell me about models or platforms that you think are good for this what should people know about uh does anyone want to go first uh Natan what's your favorite computer vision model no so we have have our own Desi models that are by the way open source uh so one of the recent models we uh have released is yolas which outperforms in terms of accuracy to latency trade off all the other YOLO that are there around um very easy to just uh take online use in production it's also production oriented Day Zero so uh this is what I would suggest but of course I'm a bit biased all right y us maybe with check things out I guess don't be shy about selling things uh sanj uh do you have any recommendations yeah so that I mean uh if you want to just go from our uh um sort of the foundational models I would say a clip is a great one that's from Dino V2 which was recently open service all vit this the visual Transformer model all of these things are state-of-the-art models um but I do want to call out while these are all great models if you really trying to pin it down to U your use case then you know you need to think through like what Natan was mentioning or what Desi offers which is how do I take these uh models to fit into my target resources because one of the common thing is you're eventually going to be resource constrained in terms of inferences that that you have to make in let's say in an inspection line you don't have a massive server running behind that right so uh while all of the other models that I mentioned are going to yield very good results it may not be transferable uh to run on a small lightweight inference so you like to factor that in all right yeah so um you've really got to pick the right model for your for your use case um Bea have you got any favorites you want to tell us about yeah I can think of two that we recently use in making our demos one is the clip um you know it's made it really easy to combine the text so the multimodal aspect of it like you know combine be able to search for pizza and get images it's pretty easy uh now when we build our demos and um the other one is segment anything that we like extract the regions of interest in an IM okay those are the ones that come top of the mind I like the idea of searching for feature and uh yeah getting good results back um all right uh so next question comes from uh alawa saying uh how do you get uh from classical image processing methods to the fancy deep burning ones you're talking about uh yeah so what if if you got a background in classical image processing like what do you need to know for the the Deep learning stuff is it back to what we talked about before py to and hugging face or or is there anything more to it um I I think that uh the bridge between uh classical computer vision and uh deep learning computer vision is in the you can think of it as when you have a convolutional neural networks this network consists of convolutional layers and every layer is basically a metrix and these uh layers parameters are learned for example um if um edges horizontal edges are the best feature to solve your task then eventually the network will learn that this is the best feature weights of the metrix of this convolutional layer will look more like a classic um Edge horizontal horizontal or vertical Edge detection same goes uh to Fier which is basically uh sign and cosign basis and to to other things as well okay so it just sound like there's a sort of mathematical link between the classical stuff and the new stuff that's that's cool to know uh all right uh next question comes from H H asks what should we do to protect our data I think maybe there's two different different uh options for this is like whether it's you're an individual uh person or whether you're a business that that's sort of two different questions I guess take your Pickers to which one you want to answer uh anyone who wants to go here I can give an answ