AI is Coming for Your Toaster

MLOps.community · Advanced ·🧬 Deep Learning ·1y ago
Skills: ML Pipelines90%

Key Takeaways

Qualcomm AI Hub is revolutionizing AI deployments across various devices, making it easy for developers to bring the latest AI innovations to edge devices within five minutes and five lines of code, using tools like Qualcomm's neural processor and automated model optimization and deployment systems.

Full Transcript

talk to me a bit about what you're doing at like Qualcomm AI Hub because I think there's some fun stuff there for the developers that want to develop on the edge yeah and they want to put models onto the edge yeah I can kind of summarize that very very quickly so uh you know for for those who are unaware uh quom is the largest manufacturer of silicon in the world and we manufacture silicon for phones for PCS for cars and um you know industrial um automation um iot devices so we manufacture silicon for a lot of you know power constrained uh settings and uh one of the one of the things that we're trying to solve is to make it extremely easy for a developer whether that be a manufacturer of these devices or cars or phones or PCS or someone who's building an application on these devices we want to make their job of being able to bring the latest and greatest AI Innovations to our devices as easily as possible um and you know we have a little saying which is they have to be able to do this within um within five minutes within five lines of code so we built a system uh which allows people to you know as soon as they finished training they can say hey this is my model this is the device I want to run it on and go and what our system does is it'll take the model it'll translate it to um how to run it most efficiently on our newal processor it'll optimize it um we actually even have physical devices in in a cloud where uh we'll measure performance right there we'll run it we'll measure accuracy and we'll give back a result in five minutes and to the developer saying okay this model will run in you know 60 milliseconds on the on on these devices um it here's how here's how you should run it on the device uh here's the model for you to download and run and uh and these are the performance characteristics of the model if you want to tweak it some more and here's a link so you can look at it and you know share it with your colleagues so that they also understand what you're deploying and and here's an automation for you to do this programmatically in your in your uh in your system so we've sort of automated the process of being able to deploy models on all our devices and that we believe is the key to more and more Innovation and more and more um iteration and more and more um deployments of more complex things across all our different devices so that in a nutshell is what qu on me I have is [Music]

Original Description

Efficient Deployment of Models at the Edge // MLOps Podcast #284 with Krishna Sridhar, Vice President of Qualcomm. Big shout out to @qualcomm for sponsoring this episode! We explored Qualcomm AI Hub with Krishna Sridhar, diving into how Qualcomm is revolutionizing AI deployments across various devices. Qualcomm streamlines the process, enabling developers to easily deploy machine learning models to edge devices - in just five minutes and five lines of code. They've automated the deployment process, ensuring models run efficiently on their processors while providing real-time performance feedback. This innovation opens a gateway for continuous development and deployment, fostering more advanced AI applications. It's truly transformative for developers in power-constrained settings such as mobile phones and IoT devices. // Abstract Qualcomm® AI Hub helps to optimize, validate, and deploy machine learning models on-device for vision, audio, and speech use cases. With Qualcomm® AI Hub, you can: Convert trained models from frameworks like PyTorch and ONNX for optimized on-device performance on Qualcomm® devices. Profile models on-device to obtain detailed metrics including runtime, load time, and compute unit utilization. Verify numerical correctness by performing on-device inference. Easily deploy models using Qualcomm® AI Engine Direct, TensorFlow Lite, or ONNX Runtime. The Qualcomm® AI Hub Models repository contains a collection of example models that use Qualcomm® AI Hub to optimize, validate, and deploy models on Qualcomm® devices. Qualcomm® AI Hub automatically handles model translation from source framework to device runtime, applying hardware-aware optimizations, and performing physical performance/numerical validation. The system automatically provisions devices in the cloud for on-device profiling and inference. The following image shows the steps taken to analyze a model using Qualcomm® AI Hub. // Bio Krishna Sridhar leads engineering for Qualcomm™ A
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from MLOps.community · MLOps.community · 0 of 60

← Previous Next →
1 Our 1st MLOps Meetup // Luke Marsden // MLOps Meetup #1
Our 1st MLOps Meetup // Luke Marsden // MLOps Meetup #1
MLOps.community
 2 Remote Collaboration as a Data Scientist
Remote Collaboration as a Data Scientist
MLOps.community
 3 MLOps Manifesto with Luke Marsden from Dotscience
MLOps Manifesto with Luke Marsden from Dotscience
MLOps.community
 4 MLOps lifecycle description
MLOps lifecycle description
MLOps.community
 5 What Does Best in Class AI/ML Governance Look Like in Fin Services? // Charles Radclyffe // MLOps #2
What Does Best in Class AI/ML Governance Look Like in Fin Services? // Charles Radclyffe // MLOps #2
MLOps.community
 6 Life purpose and too many spreadsheets
Life purpose and too many spreadsheets
MLOps.community
 7 Explainability, Black boxes and EU white paper on reproducibility
Explainability, Black boxes and EU white paper on reproducibility
MLOps.community
 8 Hierarchy of Machine Learning Needs // Phil Winder // MLOps Meetup #3
Hierarchy of Machine Learning Needs // Phil Winder // MLOps Meetup #3
MLOps.community
 9 Automatically Retrain Machine Learning Models? Are best practices worth it?
Automatically Retrain Machine Learning Models? Are best practices worth it?
MLOps.community
 10 Building an MLOps Team? Key ideas to keep in mind
Building an MLOps Team? Key ideas to keep in mind
MLOps.community
 11 Hierarchy of MLOps Needs
Hierarchy of MLOps Needs
MLOps.community
 12 Bare necessities for getting an ML model into production
Bare necessities for getting an ML model into production
MLOps.community
 13 MLOps and Monitoring
MLOps and Monitoring
MLOps.community
 14 How Phil Winder got into Data Science and Software Engineering
How Phil Winder got into Data Science and Software Engineering
MLOps.community
 15 Provenance and Reproducibility in Machine Learning; what is it and why you need it?
Provenance and Reproducibility in Machine Learning; what is it and why you need it?
MLOps.community
 16 Friction Between Data Scientists and Software Engineers
Friction Between Data Scientists and Software Engineers
MLOps.community
 17 MLOps Problems in different size companies
MLOps Problems in different size companies
MLOps.community
 18 ML tooling in large companies
ML tooling in large companies
MLOps.community
 19 ML Platforms - The build vs buy question
ML Platforms - The build vs buy question
MLOps.community
 20 ML Services Gateway at SurveyMonkey
ML Services Gateway at SurveyMonkey
MLOps.community
 21 Message buses, Async and sync architecture
Message buses, Async and sync architecture
MLOps.community
 22 MLOps #4: Shubhi Jain - Building an ML Platform @SurveyMonkey
MLOps #4: Shubhi Jain - Building an ML Platform @SurveyMonkey
MLOps.community
 23 Hybrid Data Science Teams @SurveyMonkey
Hybrid Data Science Teams @SurveyMonkey
MLOps.community
 24 How do you handle ML version control at SurveyMonkey
How do you handle ML version control at SurveyMonkey
MLOps.community
 25 Doing ML with Personal Information
Doing ML with Personal Information
MLOps.community
 26 Evolution of the ML feature store @SurveyMonkey
Evolution of the ML feature store @SurveyMonkey
MLOps.community
 27 Developing a Machine Learning Feature Store
Developing a Machine Learning Feature Store
MLOps.community
 28 Auto retrain ML models is not the question
Auto retrain ML models is not the question
MLOps.community
 29 3 key parts to Machine Learning monitoring
3 key parts to Machine Learning monitoring
MLOps.community
 30 MLOps Meetup #6: Mid-Scale Production Feature Engineering with Dr. Venkata Pingali
MLOps Meetup #6: Mid-Scale Production Feature Engineering with Dr. Venkata Pingali
MLOps.community
 31 MLOps meetup #5 High Stakes ML: Active Failures, Latent Factors with Flavio Clesio
MLOps meetup #5 High Stakes ML: Active Failures, Latent Factors with Flavio Clesio
MLOps.community
 32 MLOps: Airflow Pros and Cons
MLOps: Airflow Pros and Cons
MLOps.community
 33 Specific challenges in Machine Learning
Specific challenges in Machine Learning
MLOps.community
 34 Current State Of Machine Learning
Current State Of Machine Learning
MLOps.community
 35 Humans in the Loop are a defining factor in Machine Learning
Humans in the Loop are a defining factor in Machine Learning
MLOps.community
 36 Learning from real life Machine Learning failures
Learning from real life Machine Learning failures
MLOps.community
 37 Survivorship Bias in machine learning tutorials
Survivorship Bias in machine learning tutorials
MLOps.community
 38 Swiss Cheese model in Machine Learning
Swiss Cheese model in Machine Learning
MLOps.community
 39 Resume driven development in Machine learning & software engineering
Resume driven development in Machine learning & software engineering
MLOps.community
 40 Who has the highest standards in ML?
Who has the highest standards in ML?
MLOps.community
 41 Venkata Pingali of Scribble Data Thoughts on the Current State of Machine Learning
Venkata Pingali of Scribble Data Thoughts on the Current State of Machine Learning
MLOps.community
 42 Dependable data and being able to Trust in your Data with Venkata Pengali of Scribble Data
Dependable data and being able to Trust in your Data with Venkata Pengali of Scribble Data
MLOps.community
 43 Speed, Trust, Evolution and Scale in MLOps
Speed, Trust, Evolution and Scale in MLOps
MLOps.community
 44 More difficult transition for data scientists to become ML engineers
More difficult transition for data scientists to become ML engineers
MLOps.community
 45 How many models in prod til I need a dedicated ML platform?
How many models in prod til I need a dedicated ML platform?
MLOps.community
 46 Deeper thinking from data scientists around platform blackholes
Deeper thinking from data scientists around platform blackholes
MLOps.community
 47 Checkpointing, metadata, and confidence in your data
Checkpointing, metadata, and confidence in your data
MLOps.community
 48 Adjacent usecases and multistep feature engineering
Adjacent usecases and multistep feature engineering
MLOps.community
 49 Standardization of Machine Learning tools like in Software Engineering with Venkata Pingali
Standardization of Machine Learning tools like in Software Engineering with Venkata Pingali
MLOps.community
 50 Reproducability flaws in end to end Machine Learning debugging
Reproducability flaws in end to end Machine Learning debugging
MLOps.community
 51 3rd wave of data scientists
3rd wave of data scientists
MLOps.community
 52 MLOps meetup #7 Alex Spanos // TrueLayer 's MLOps Pipeline
MLOps meetup #7 Alex Spanos // TrueLayer 's MLOps Pipeline
MLOps.community
 53 MLOps Meetup #8 Optimizing Your ML Workflow with Kubeflow 1.0
MLOps Meetup #8 Optimizing Your ML Workflow with Kubeflow 1.0
MLOps.community
 54 Are Kubeflow and Airflow complementary?
Are Kubeflow and Airflow complementary?
MLOps.community
 55 Why Kubeflow gained so much traction=open community
Why Kubeflow gained so much traction=open community
MLOps.community
 56 Who decides the dirrection of Kubeflow
Who decides the dirrection of Kubeflow
MLOps.community
 57 What do Kubeflow and Arrikto do and how do they work together?
What do Kubeflow and Arrikto do and how do they work together?
MLOps.community
 58 Versioning your ML steps with Kubeflow
Versioning your ML steps with Kubeflow
MLOps.community
 59 Machine Learning Lifecycles//Perception vs Reality
Machine Learning Lifecycles//Perception vs Reality
MLOps.community
 60 Kubeflow vs SageMaker in Machine Learning
Kubeflow vs SageMaker in Machine Learning
MLOps.community

Qualcomm AI Hub is making it easy for developers to deploy AI models at the edge, using automated optimization and deployment systems, and providing tools for monitoring and updating model performance. This is achieved through a simple and efficient process that takes only five minutes and five lines of code. By leveraging Qualcomm's neural processor and AI hub, developers can bring the latest AI innovations to edge devices, enabling more innovation and iteration in the field.

Key Takeaways
  1. Train an AI model
  2. Use Qualcomm AI Hub to optimize and deploy the model
  3. Select the target device and configure the deployment settings
  4. Automate the deployment process using Qualcomm's automated systems
  5. Monitor and update model performance using Qualcomm's tools
  6. Share and collaborate with colleagues on model deployments
  7. Use Qualcomm's neural processor to run models efficiently on edge devices
💡 Qualcomm AI Hub is automating the process of deploying AI models at the edge, making it possible for developers to bring the latest AI innovations to edge devices quickly and efficiently, and enabling more innovation and iteration in the field.

Related Reads

Up next
RNNs Explained in 60 Seconds #ai #coding #machinelearning
Ascent
Watch →