Part 7 | Deploy ML Model on Kubernetes | Scalable options, Testing, Debugging & Optimization

Key Takeaways

So welcome back to the continuation of our tutorial series on deploying a machine learning model on Kubernetes. In this part, we'll be seeing uh a few other Kubernetes commands to like debug the deployment issues and also we'll discussing about optimizing the resource allocation and uh scaling strategies. And now uh we have several other commands that we should be aware of like uh we can check if the ports are running and ready. like using this command. Let me clear this. So this command like retries the list of all ports like associated with the ML services. So in our case we have uh like three replicas of an application which is running. So the output just displays the status of each port and including whether it is running bending or in a failed state. So in our case it's running. So if the ports are like not ready or or stuck in an non- running state then this is the first step to identify the issue okay with your running uh service and we can also do describe command okay for example so this command cubectl describe code with the port name so this command provides a detailed information about a specific port so just like events container resources and status ation and if the port is not starting and this command will help to like identify the root cause such as like resource limits permission issues or other errors in the port setup and we can use as well like cubectl log and this command which shows the logs of the specific port and if the port is like running but not behaving as expected the like the model is failing to load for example so inspecting the logs can like help us to diagnose the issues such there's like configuration errors or like runtime exceptions. So in our case it's running properly. So we are getting 20 0. For example, if your model loading is failed. Okay. For example, so this command like cubectl get PVC. This command checks the persistent value claims. uh and PVC is used for like so this command uh will check the status of the persistent volume claims and PVCs are like used for like persistent storage and our ML services likely depend on one want to like load the model. So if the model fails to load the issue might be related to a PVC not being correctly bound to the persistent value. So this command will show all our PVCs and their statuses. And we can do as well cubectgl describe PVC I mean the name of our PVC. So this command provides the detailed information about the specific PVC like in our case uh it will shows the status and events. It helps to check if the PVCs is currently bound to the persistent volumes and whether it is able to like provide the necessary storage for the ML model. So if your HP is not scaling properly then you can do this command. Let me clear this. So this provides the details about the horizontal port out autoscaler associated with your ML service and it shows the HPA configuration and the metrics it is used using to like scale the number of ports and if the HPA is not scaling as expected. This command can like help determine whether the target metrics like CPU utilization are being met or if there is an issue with the HP configuration or not. So for example if uh the matrixes are not showing in the protheenthesis for example uh as we have already put forward our uh promthesis here. So so once we have put forwarded to like promthesis we can like open the web and navigate to the target as I showed earlier. This will uh show the list of uh targets. So if there is any issues or the state is down, it will show the error message here and you can debug accordingly as well. To check the promthesis log uh basically uh you can use this command. Let me go back here and uh QCDL logs log level with app hypothesis. It shows the logs of the promthesis port. Uh this is filtered by the app promthesis label you see here. So if there is an issue with promthesis scrapping matrixes from ML service or if the prothesis itself isn't it functioning properly and reviewing these logs can like reveal uh some error messages or warning. So that's it guys. We are at the end of this part like uh so far what we have done is like we have tested and tried to like debug the Kubernetes deployment issues and also see how to optimize the resource allocation and scaling strategies and congratulations for finishing this course and you have successfully completed this end to end tutorial series on deploying a machine learning model on Kubernetes with autoscaling and monitoring. This was a deep dive into the standardizing the ML deployment covering everything from model training to deployment scaling and observability. So throughout this journey like we explored several key concepts and technologies like ensuring you gain like hands-on experience like for setting up the deployment environment. So we have configured our environment with port man or docker and kubernetes with kind and installed all necessary dependencies. So this foundational like step prepared us for subsequent uh stages of de development and also we have been building a machine learning model and API. We trained a sentiment analysis model using scikit learn and developed a first API based rest API to serve predictions. We also ensured the API was functional by testing it locally and then we have containerized the ML API. So we have created the docker file like build a image and ran locally to test its functionality and this step like ensures that our application was portable and ready for deployment and then we have deployed everything on Kubernetes like we have created the kind cluster which is Kubernetes and docker cluster and we have wrote essential like Kubernetes manifest files like deployment service config map PVC and applied them to deploy our ML service and we have also like implemented autoscaling using horizontal port autoscaler which is HPA. We ensure that our ML service could like dynamically scale based on workload demands and improving the performance and resource utilization and we have also added monitoring service with promthesis. We integrated promthesis to collect and visualize the performance metrics and providing the insights into the resource usage, request rates and system health. And finally we have explored different ways to like test AV end points debug the deployment issues and also how to optimize the resource allocation for better scalability. So far we have like built a fully functional scalable and monitoring machine learning deployment which is crucial for like setting up a deploying a real world a application in production. If you found this tutorial helpful then do subscribe to our channel in the way you'll be get notified whenever I upload new tutorial. So until then this is Aona. See you all in our next session. Thanks for watching. [Music]