deeplearning.ai's Heroes of Deep Learning: Yuanqing Lin

DeepLearningAI · Beginner ·📐 ML Fundamentals ·9y ago

Key Takeaways

The video features an interview with Yuanqing Lin, discussing his experiences and insights in the field of deep learning, covering topics such as neural network basics, sequence models, and generative models, with a focus on ml fundamentals.

Full Transcript

well commuting and I'm really glad you could join us today so um so you know today you're the head of I do research and when the Chinese government the government of China was looking for someone to start up and build a national deep learning research lab they tapped you to help start this thing so you know arguably I think maybe other number one deep learning person in the entire country of China so before so I'd love to ask you a lot of questions about your work but before that I want to hear about your personal story so how do you end up yeah I'm getting to do this work that you do yeah so so I actually I before my ph.d program my major was in kind of optics so it's more like in physics I think I had a fairly good color background like very good Packer on math the outer I can do us and I was thinking like what kind of major gonna take for my ph.d program and I was thinking that well I guess go for a vertex are go for something else because a black to like early 2000 I think not new technology was really really hot yeah and but I was thinking probably I should look at something like a even more exciting and the leaves a good chance that I I was taking some classes that you pay and I got to know Tenley so later he became my PhD advisor so and I was thinking like machine learning was a great thing to do and I got really excited and I kind of I changed my major so I I did my hippie idea open like a majoring in like in machine learning yeah and so let that kind of lecture is I was left for five yes and then I was kind of really exciting and I think I got I learned lots of being from scratch Hey well good and even like a pcasi I didn't I didn't know Louis before yeah it was kind of I feel I was learning new things every day so he was very very exciting experience for me this was one of those things of a lot of stars although you know just did a lot of work and was underappreciated for this time right right yeah so I think and she was exciting place and I I was that at beginning as a researcher I mean I I also like to feel that wow I learned lots of things and and I just laser energy I kind of started working on computer vision I actually started working on communism very late compared I mean relatively late yeah and the first thing i did was i participated in image in a challenge that's your first year of image net challenge i was kind of a managing a team to work on a project it was lucky we're quite lucky later we were quite strong and and we at the end we actually got the number one place overwhelming number one place you know in the context so you're the first ever in the world in which their competition with like yeah and i was the prisoner did a presentation at the level shop yeah so i was always really nice appearance for me and that actually gave me into Lisa well Alaska computer vision tasks yeah and and so I had been working on Lisa well a skill Robin since they're like yeah and so when he least your cat Haven New York time cat they come out and also later like at least Alex nekima you really broke my mind I try I think it's not on oh wow the tubulin is so powerful and and and I think since then I think West LA so I for I think to work on laws so as a head of China's National Lab national research that want deep learning there must be a lot of exciting activities going on there so for the for the global audience you're watching this what should they know about what's happening with this National Lab the mission awful is a national engineering lab which to build a really large deep learning platform and probably over to be the biggest one at least the biggest one in China and unleased perform we would offer people like a deep learning framework like pedal and we offer people last well a scale computing a resource and also other people will hatch like well it big data right and and if people are able to kind of develop our research or develop a kind of good technology on this platform we're all so awful and like a big applications so formally the technican be probably into some big applications if I do so later the technology up get insulate and improved yeah so so we believe that the combining loss of a kind of a combining loss will be sauce altogether I think is gonna be a really powerful a profile and also give you one example is for example like right now if we publish a paper say little researcher published a paper if someone want to reproduce it and the needle cannot the best thing to do in probably the person who produce provided up the color somewhere and you could tell no like all the 2-yard it were computer and you also try to find that the date has said somewhere and and that we probably don't you probably also need to get a good comic relation of your kind of completing this awesome long and smoothly right so it should be easily kind of taking some efforts at the leisure tip National Lab things will become much easier so so if we someone using this platform to write a paper to do the work and I buy the paper and the lay who have look code on a unleash on this profile and and the computing structure is already set up follow for this code and a date has led to so basically you just need a common line to Lee producer Lisa so this is a huge I think is a big big a kind of belief of a lot of a kind of reproducibility she was in computer science so you easily we just kind of just a few seconds you start learning something like you see in a paper yeah so Liz Cheney powerful so we so this is just one example we we are working on like it to make sure that we are providing providing like a very powerful platform a tulip community and tuna industry that's amazing that really speed up even yeah can you um give a sense of how much measles is you know the Chinese government is putting to thank you for this deep learning National Lab so I think Alicia national engineering lab I think the government that we can invest some funding into here to build up infrastructure but I think I'm more important with the be a least gonna be a flagship inviting in China gonna lead a loss of kind of people in the efforts like including like a national project and and and the loss of kind of policies and things yeah so Lisa actually is Cheney powerful and I think of app idea we are really honored to get in his life you know someone at the heart of deep learning in China and so there's a lot of activity in China that the global audience you know isn't aware of here then hasn't seen yet so what should people outside China know about deep learning in China yeah I think in China especially in the past few years I think the C de belem empowered a product service is really booming like coming like ranging from like a search engines to like a face no condition to like to surveillance to like a ecommerce like lots of praise I think that they are investing big ever in the deep learning and also like they are really make it yourself it was technology to make the business much more powerful yeah and and the this actually is a very important for for for developing AI technology in general I think for myself I and also lots of people Shelley's we will believe that actually it's well in Holland to how Lisa we often call the puzzle group but for example you you you probably when we start out to think of buildings and technology that we have some initial data and we try to do our initial algorithm they would launch is an initial product right for the our service the other letter would get the data front users and in the other weekend model has no developer more will deliver more solid we would develop better algorithms because we see Monday how we know like a will be a better algorithm so we have more data and better algorithm were able to have better technology for a product service and then definitely we hope it will be able to achieve more users right because the product that I know it's better and and they will get more data so this is a really good positive loop and he's very special I sheriff or for AI related technologies right I mean for all for like its traditional thing not like a laser like a laser that I was working I left before so so it isn't isn't more like it it look closer but I'm not gonna be more linear but the vault this AI technology actually because of this positive loop you can imagine that definite certain part you'll come with like really fast girls are full of Technology yeah and in the least actually super important and when we designing a research research interim with only desire handy but we work on the direction that we able to get to Nisha quick kind of improving period right if we are not able to even our business i evil to the whole kind of our whole business is not able to fund this positive loop even if we are not able to fund is this it has strong positive loop then did this direction probably should not work out because someone else who holds down business too funny strong loop let me get to Lisa kind of a phrase much more quicker than you are so so so this actually when important logic for for us when we looking at the legacy hey you need a company what direction we should work on at what direction should not work our wisdom is a winning point of fact I wasn't okay today both in China and in the US and globally there are no other people wanting to enter a deep learning and wanting to enter AI but what advice would you have for someone that wants to get into this field so another day is definitely I think people who start with a was open salsa friend looks I think unless Cheney powerful to convince the others I think when I was studying my deep learning a study there was not much like a oven sauce the sauce I think nowadays like in India especially in tabular knee he's a very good community and all that right and they are multiple like a really good people and in frameworks it's always like that I got tension flow a cafe like now I call coffee to write and I've got in China we had a good pedo pedo and the closer I under in even like Formosa follows online they a lot like a causes to continue to teach you how to use those and and also nowadays also the other many like a publicly available pinch mark and the people whose see hey I can really kind of skill for really spilling people like how how well they could do a lot benchmark so so so basically time to get familiar with the deeper learning I think those are very good a starting point how did you gained an understanding so actually I do it you know obviously the way I kind of learned I'm long TCA I learned like I'd be almost like a develop learning actually right so and so so basically I so I inside also is a good way I feel like we cannot lay down lots of foundations right and we learn graphic your mother alright so so these are all kind of alright right now he didn't expand a little bit but annoying laws I should give you was a good intuition about how deep learning works and in the one day probably is a connection of deep learning to loss like a framework or approach I think the ladies already lost our connections and the laws actually make deepening which always out to India and yeah so I feel like a good stylist a open source alongside to finish cheney powerful resource but mingle allows suggest like a you also didn't learn those basic things about machine learning so thank you that was fast even though I've known you for a long time there are a lot of details you're thinking that I didn't realize until now so thank you very much thank you so much for having me
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from DeepLearningAI · DeepLearningAI · 2 of 60

1 Forward and Backward Propagation (C1W4L06)
Forward and Backward Propagation (C1W4L06)
DeepLearningAI
deeplearning.ai's Heroes of Deep Learning: Yuanqing Lin
deeplearning.ai's Heroes of Deep Learning: Yuanqing Lin
DeepLearningAI
3 deeplearning.ai's Heroes of Deep Learning: Ruslan Salakhutdinov
deeplearning.ai's Heroes of Deep Learning: Ruslan Salakhutdinov
DeepLearningAI
4 deeplearning.ai's Heroes of Deep Learning: Yoshua Bengio
deeplearning.ai's Heroes of Deep Learning: Yoshua Bengio
DeepLearningAI
5 deeplearning.ai's Heroes of Deep Learning: Pieter Abbeel
deeplearning.ai's Heroes of Deep Learning: Pieter Abbeel
DeepLearningAI
6 deeplearning.ai's Heroes of Deep Learning: Ian Goodfellow
deeplearning.ai's Heroes of Deep Learning: Ian Goodfellow
DeepLearningAI
7 deeplearning.ai's Heroes of Deep Learning: Andrej Karpathy
deeplearning.ai's Heroes of Deep Learning: Andrej Karpathy
DeepLearningAI
8 Using an Appropriate Scale (C2W3L02)
Using an Appropriate Scale (C2W3L02)
DeepLearningAI
9 Gradient Checking (C2W1L13)
Gradient Checking (C2W1L13)
DeepLearningAI
10 Gradient Checking Implementation Notes (C2W1L14)
Gradient Checking Implementation Notes (C2W1L14)
DeepLearningAI
11 Learning Rate Decay (C2W2L09)
Learning Rate Decay (C2W2L09)
DeepLearningAI
12 Understanding Mini-Batch Gradient Dexcent (C2W2L02)
Understanding Mini-Batch Gradient Dexcent (C2W2L02)
DeepLearningAI
13 Mini Batch Gradient Descent (C2W2L01)
Mini Batch Gradient Descent (C2W2L01)
DeepLearningAI
14 The Problem of Local Optima (C2W3L10)
The Problem of Local Optima (C2W3L10)
DeepLearningAI
15 Exponentially Weighted Averages (C2W2L03)
Exponentially Weighted Averages (C2W2L03)
DeepLearningAI
16 Tuning Process (C2W3L01)
Tuning Process (C2W3L01)
DeepLearningAI
17 Understanding Exponentially Weighted Averages (C2W2L04)
Understanding Exponentially Weighted Averages (C2W2L04)
DeepLearningAI
18 Bias Correction of Exponentially Weighted Averages (C2W2L05)
Bias Correction of Exponentially Weighted Averages (C2W2L05)
DeepLearningAI
19 Gradient Descent With Momentum (C2W2L06)
Gradient Descent With Momentum (C2W2L06)
DeepLearningAI
20 Normalizing Activations in a Network (C2W3L04)
Normalizing Activations in a Network (C2W3L04)
DeepLearningAI
21 Hyperparameter Tuning in Practice (C2W3L03)
Hyperparameter Tuning in Practice (C2W3L03)
DeepLearningAI
22 Adam Optimization Algorithm (C2W2L08)
Adam Optimization Algorithm (C2W2L08)
DeepLearningAI
23 RMSProp (C2W2L07)
RMSProp (C2W2L07)
DeepLearningAI
24 Fitting Batch Norm Into Neural Networks (C2W3L05)
Fitting Batch Norm Into Neural Networks (C2W3L05)
DeepLearningAI
25 Why Does Batch Norm Work? (C2W3L06)
Why Does Batch Norm Work? (C2W3L06)
DeepLearningAI
26 Batch Norm At Test Time (C2W3L07)
Batch Norm At Test Time (C2W3L07)
DeepLearningAI
27 Softmax Regression (C2W3L08)
Softmax Regression (C2W3L08)
DeepLearningAI
28 Deep Learning Frameworks (C2W3L10)
Deep Learning Frameworks (C2W3L10)
DeepLearningAI
29 Neural Network Overview (C1W3L01)
Neural Network Overview (C1W3L01)
DeepLearningAI
30 Training Softmax Classifier (C2W3L09)
Training Softmax Classifier (C2W3L09)
DeepLearningAI
31 Why Deep Representations? (C1W4L04)
Why Deep Representations? (C1W4L04)
DeepLearningAI
32 Gradient Descent For Neural Networks (C1W3L09)
Gradient Descent For Neural Networks (C1W3L09)
DeepLearningAI
33 Neural Network Representations (C1W3L02)
Neural Network Representations (C1W3L02)
DeepLearningAI
34 TensorFlow (C2W3L11)
TensorFlow (C2W3L11)
DeepLearningAI
35 Activation Functions (C1W3L06)
Activation Functions (C1W3L06)
DeepLearningAI
36 Explanation For Vectorized Implementation (C1W3L05)
Explanation For Vectorized Implementation (C1W3L05)
DeepLearningAI
37 Getting Matrix Dimensions Right (C1W4L03)
Getting Matrix Dimensions Right (C1W4L03)
DeepLearningAI
38 Understanding Dropout (C2W1L07)
Understanding Dropout (C2W1L07)
DeepLearningAI
39 Building Blocks of a Deep Neural Network (C1W4L05)
Building Blocks of a Deep Neural Network (C1W4L05)
DeepLearningAI
40 Why Non-linear Activation Functions (C1W3L07)
Why Non-linear Activation Functions (C1W3L07)
DeepLearningAI
41 Computing Neural Network Output (C1W3L03)
Computing Neural Network Output (C1W3L03)
DeepLearningAI
42 Backpropagation Intuition (C1W3L10)
Backpropagation Intuition (C1W3L10)
DeepLearningAI
43 Train/Dev/Test Sets (C2W1L01)
Train/Dev/Test Sets (C2W1L01)
DeepLearningAI
44 Deep L-Layer Neural Network (C1W4L01)
Deep L-Layer Neural Network (C1W4L01)
DeepLearningAI
45 Random Initialization (C1W3L11)
Random Initialization (C1W3L11)
DeepLearningAI
46 Other Regularization Methods (C2W1L08)
Other Regularization Methods (C2W1L08)
DeepLearningAI
47 Normalizing Inputs (C2W1L09)
Normalizing Inputs (C2W1L09)
DeepLearningAI
48 Derivatives Of Activation Functions (C1W3L08)
Derivatives Of Activation Functions (C1W3L08)
DeepLearningAI
49 Parameters vs Hyperparameters (C1W4L07)
Parameters vs Hyperparameters (C1W4L07)
DeepLearningAI
50 Vectorizing Across Multiple Examples (C1W3L04)
Vectorizing Across Multiple Examples (C1W3L04)
DeepLearningAI
51 What does this have to do with the brain? (C1W4L08)
What does this have to do with the brain? (C1W4L08)
DeepLearningAI
52 Dropout Regularization (C2W1L06)
Dropout Regularization (C2W1L06)
DeepLearningAI
53 Vanishing/Exploding Gradients (C2W1L10)
Vanishing/Exploding Gradients (C2W1L10)
DeepLearningAI
54 Basic Recipe for Machine Learning (C2W1L03)
Basic Recipe for Machine Learning (C2W1L03)
DeepLearningAI
55 Bias/Variance (C2W1L02)
Bias/Variance (C2W1L02)
DeepLearningAI
56 Forward Propagation in a Deep Network (C1W4L02)
Forward Propagation in a Deep Network (C1W4L02)
DeepLearningAI
57 Weight Initialization in a Deep Network (C2W1L11)
Weight Initialization in a Deep Network (C2W1L11)
DeepLearningAI
58 Numerical Approximations of Gradients (C2W1L12)
Numerical Approximations of Gradients (C2W1L12)
DeepLearningAI
59 Regularization (C2W1L04)
Regularization (C2W1L04)
DeepLearningAI
60 Why Regularization Reduces Overfitting (C2W1L05)
Why Regularization Reduces Overfitting (C2W1L05)
DeepLearningAI

This video features an interview with Yuanqing Lin, discussing his experiences and insights in the field of deep learning, covering topics such as neural network basics, sequence models, and generative models. The video provides a beginner-friendly introduction to ml fundamentals and deep learning concepts. By watching this video, viewers can gain a deeper understanding of the field and its applications.

Key Takeaways
  1. Watch the video to understand Yuanqing Lin's experiences in deep learning
  2. Take notes on key concepts such as neural networks, sequence models, and generative models
  3. Research and explore the topics discussed in the video
  4. Apply the concepts learned to real-world problems or projects
  5. Experiment with building simple neural networks or sequence models
💡 The video highlights the importance of understanding the basics of neural networks and sequence models in order to build more complex and effective deep learning models.

Related Reads

📰
I regret reviewing for AAAI [D]
A machine learning researcher regrets reviewing for AAAI due to non-reciprocal effort and potential negative impact on authors
Reddit r/MachineLearning
📰
GenTrace: Automating Telecom Network Fault Diagnosis from Raw Packet Traces
Automate telecom network fault diagnosis using GenTrace, which leverages AI to analyze raw packet traces and provide actionable recommendations
Medium · Machine Learning
📰
GPU Rental Guide: H100 vs A100 vs L40S vs RTX4090
Learn to choose the right GPU for your AI workloads by comparing H100, A100, L40S, and RTX4090 in this rental guide
Dev.to · YingSuan AI
📰
Kafka Consumer Lag Is Growing. Your API Is Healthy While the Business Falls Behind.
Kafka consumer lag can secretly sabotage your business despite healthy API metrics, learn to identify and address it
Medium · Programming
Up next
Machine Learning with Rust and Candle: Part 3
Stephen Blum
Watch →