deeplearning.ai's Heroes of Deep Learning: Yuanqing Lin
Key Takeaways
The video features an interview with Yuanqing Lin, discussing his experiences and insights in the field of deep learning, covering topics such as neural network basics, sequence models, and generative models, with a focus on ml fundamentals.
Full Transcript
well commuting and I'm really glad you could join us today so um so you know today you're the head of I do research and when the Chinese government the government of China was looking for someone to start up and build a national deep learning research lab they tapped you to help start this thing so you know arguably I think maybe other number one deep learning person in the entire country of China so before so I'd love to ask you a lot of questions about your work but before that I want to hear about your personal story so how do you end up yeah I'm getting to do this work that you do yeah so so I actually I before my ph.d program my major was in kind of optics so it's more like in physics I think I had a fairly good color background like very good Packer on math the outer I can do us and I was thinking like what kind of major gonna take for my ph.d program and I was thinking that well I guess go for a vertex are go for something else because a black to like early 2000 I think not new technology was really really hot yeah and but I was thinking probably I should look at something like a even more exciting and the leaves a good chance that I I was taking some classes that you pay and I got to know Tenley so later he became my PhD advisor so and I was thinking like machine learning was a great thing to do and I got really excited and I kind of I changed my major so I I did my hippie idea open like a majoring in like in machine learning yeah and so let that kind of lecture is I was left for five yes and then I was kind of really exciting and I think I got I learned lots of being from scratch Hey well good and even like a pcasi I didn't I didn't know Louis before yeah it was kind of I feel I was learning new things every day so he was very very exciting experience for me this was one of those things of a lot of stars although you know just did a lot of work and was underappreciated for this time right right yeah so I think and she was exciting place and I I was that at beginning as a researcher I mean I I also like to feel that wow I learned lots of things and and I just laser energy I kind of started working on computer vision I actually started working on communism very late compared I mean relatively late yeah and the first thing i did was i participated in image in a challenge that's your first year of image net challenge i was kind of a managing a team to work on a project it was lucky we're quite lucky later we were quite strong and and we at the end we actually got the number one place overwhelming number one place you know in the context so you're the first ever in the world in which their competition with like yeah and i was the prisoner did a presentation at the level shop yeah so i was always really nice appearance for me and that actually gave me into Lisa well Alaska computer vision tasks yeah and and so I had been working on Lisa well a skill Robin since they're like yeah and so when he least your cat Haven New York time cat they come out and also later like at least Alex nekima you really broke my mind I try I think it's not on oh wow the tubulin is so powerful and and and I think since then I think West LA so I for I think to work on laws so as a head of China's National Lab national research that want deep learning there must be a lot of exciting activities going on there so for the for the global audience you're watching this what should they know about what's happening with this National Lab the mission awful is a national engineering lab which to build a really large deep learning platform and probably over to be the biggest one at least the biggest one in China and unleased perform we would offer people like a deep learning framework like pedal and we offer people last well a scale computing a resource and also other people will hatch like well it big data right and and if people are able to kind of develop our research or develop a kind of good technology on this platform we're all so awful and like a big applications so formally the technican be probably into some big applications if I do so later the technology up get insulate and improved yeah so so we believe that the combining loss of a kind of a combining loss will be sauce altogether I think is gonna be a really powerful a profile and also give you one example is for example like right now if we publish a paper say little researcher published a paper if someone want to reproduce it and the needle cannot the best thing to do in probably the person who produce provided up the color somewhere and you could tell no like all the 2-yard it were computer and you also try to find that the date has said somewhere and and that we probably don't you probably also need to get a good comic relation of your kind of completing this awesome long and smoothly right so it should be easily kind of taking some efforts at the leisure tip National Lab things will become much easier so so if we someone using this platform to write a paper to do the work and I buy the paper and the lay who have look code on a unleash on this profile and and the computing structure is already set up follow for this code and a date has led to so basically you just need a common line to Lee producer Lisa so this is a huge I think is a big big a kind of belief of a lot of a kind of reproducibility she was in computer science so you easily we just kind of just a few seconds you start learning something like you see in a paper yeah so Liz Cheney powerful so we so this is just one example we we are working on like it to make sure that we are providing providing like a very powerful platform a tulip community and tuna industry that's amazing that really speed up even yeah can you um give a sense of how much measles is you know the Chinese government is putting to thank you for this deep learning National Lab so I think Alicia national engineering lab I think the government that we can invest some funding into here to build up infrastructure but I think I'm more important with the be a least gonna be a flagship inviting in China gonna lead a loss of kind of people in the efforts like including like a national project and and and the loss of kind of policies and things yeah so Lisa actually is Cheney powerful and I think of app idea we are really honored to get in his life you know someone at the heart of deep learning in China and so there's a lot of activity in China that the global audience you know isn't aware of here then hasn't seen yet so what should people outside China know about deep learning in China yeah I think in China especially in the past few years I think the C de belem empowered a product service is really booming like coming like ranging from like a search engines to like a face no condition to like to surveillance to like a ecommerce like lots of praise I think that they are investing big ever in the deep learning and also like they are really make it yourself it was technology to make the business much more powerful yeah and and the this actually is a very important for for for developing AI technology in general I think for myself I and also lots of people Shelley's we will believe that actually it's well in Holland to how Lisa we often call the puzzle group but for example you you you probably when we start out to think of buildings and technology that we have some initial data and we try to do our initial algorithm they would launch is an initial product right for the our service the other letter would get the data front users and in the other weekend model has no developer more will deliver more solid we would develop better algorithms because we see Monday how we know like a will be a better algorithm so we have more data and better algorithm were able to have better technology for a product service and then definitely we hope it will be able to achieve more users right because the product that I know it's better and and they will get more data so this is a really good positive loop and he's very special I sheriff or for AI related technologies right I mean for all for like its traditional thing not like a laser like a laser that I was working I left before so so it isn't isn't more like it it look closer but I'm not gonna be more linear but the vault this AI technology actually because of this positive loop you can imagine that definite certain part you'll come with like really fast girls are full of Technology yeah and in the least actually super important and when we designing a research research interim with only desire handy but we work on the direction that we able to get to Nisha quick kind of improving period right if we are not able to even our business i evil to the whole kind of our whole business is not able to fund this positive loop even if we are not able to fund is this it has strong positive loop then did this direction probably should not work out because someone else who holds down business too funny strong loop let me get to Lisa kind of a phrase much more quicker than you are so so so this actually when important logic for for us when we looking at the legacy hey you need a company what direction we should work on at what direction should not work our wisdom is a winning point of fact I wasn't okay today both in China and in the US and globally there are no other people wanting to enter a deep learning and wanting to enter AI but what advice would you have for someone that wants to get into this field so another day is definitely I think people who start with a was open salsa friend looks I think unless Cheney powerful to convince the others I think when I was studying my deep learning a study there was not much like a oven sauce the sauce I think nowadays like in India especially in tabular knee he's a very good community and all that right and they are multiple like a really good people and in frameworks it's always like that I got tension flow a cafe like now I call coffee to write and I've got in China we had a good pedo pedo and the closer I under in even like Formosa follows online they a lot like a causes to continue to teach you how to use those and and also nowadays also the other many like a publicly available pinch mark and the people whose see hey I can really kind of skill for really spilling people like how how well they could do a lot benchmark so so so basically time to get familiar with the deeper learning I think those are very good a starting point how did you gained an understanding so actually I do it you know obviously the way I kind of learned I'm long TCA I learned like I'd be almost like a develop learning actually right so and so so basically I so I inside also is a good way I feel like we cannot lay down lots of foundations right and we learn graphic your mother alright so so these are all kind of alright right now he didn't expand a little bit but annoying laws I should give you was a good intuition about how deep learning works and in the one day probably is a connection of deep learning to loss like a framework or approach I think the ladies already lost our connections and the laws actually make deepening which always out to India and yeah so I feel like a good stylist a open source alongside to finish cheney powerful resource but mingle allows suggest like a you also didn't learn those basic things about machine learning so thank you that was fast even though I've known you for a long time there are a lot of details you're thinking that I didn't realize until now so thank you very much thank you so much for having me
Playlist
Uploads from DeepLearningAI · DeepLearningAI · 2 of 60
1
▶
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Forward and Backward Propagation (C1W4L06)
DeepLearningAI
deeplearning.ai's Heroes of Deep Learning: Yuanqing Lin
DeepLearningAI
deeplearning.ai's Heroes of Deep Learning: Ruslan Salakhutdinov
DeepLearningAI
deeplearning.ai's Heroes of Deep Learning: Yoshua Bengio
DeepLearningAI
deeplearning.ai's Heroes of Deep Learning: Pieter Abbeel
DeepLearningAI
deeplearning.ai's Heroes of Deep Learning: Ian Goodfellow
DeepLearningAI
deeplearning.ai's Heroes of Deep Learning: Andrej Karpathy
DeepLearningAI
Using an Appropriate Scale (C2W3L02)
DeepLearningAI
Gradient Checking (C2W1L13)
DeepLearningAI
Gradient Checking Implementation Notes (C2W1L14)
DeepLearningAI
Learning Rate Decay (C2W2L09)
DeepLearningAI
Understanding Mini-Batch Gradient Dexcent (C2W2L02)
DeepLearningAI
Mini Batch Gradient Descent (C2W2L01)
DeepLearningAI
The Problem of Local Optima (C2W3L10)
DeepLearningAI
Exponentially Weighted Averages (C2W2L03)
DeepLearningAI
Tuning Process (C2W3L01)
DeepLearningAI
Understanding Exponentially Weighted Averages (C2W2L04)
DeepLearningAI
Bias Correction of Exponentially Weighted Averages (C2W2L05)
DeepLearningAI
Gradient Descent With Momentum (C2W2L06)
DeepLearningAI
Normalizing Activations in a Network (C2W3L04)
DeepLearningAI
Hyperparameter Tuning in Practice (C2W3L03)
DeepLearningAI
Adam Optimization Algorithm (C2W2L08)
DeepLearningAI
RMSProp (C2W2L07)
DeepLearningAI
Fitting Batch Norm Into Neural Networks (C2W3L05)
DeepLearningAI
Why Does Batch Norm Work? (C2W3L06)
DeepLearningAI
Batch Norm At Test Time (C2W3L07)
DeepLearningAI
Softmax Regression (C2W3L08)
DeepLearningAI
Deep Learning Frameworks (C2W3L10)
DeepLearningAI
Neural Network Overview (C1W3L01)
DeepLearningAI
Training Softmax Classifier (C2W3L09)
DeepLearningAI
Why Deep Representations? (C1W4L04)
DeepLearningAI
Gradient Descent For Neural Networks (C1W3L09)
DeepLearningAI
Neural Network Representations (C1W3L02)
DeepLearningAI
TensorFlow (C2W3L11)
DeepLearningAI
Activation Functions (C1W3L06)
DeepLearningAI
Explanation For Vectorized Implementation (C1W3L05)
DeepLearningAI
Getting Matrix Dimensions Right (C1W4L03)
DeepLearningAI
Understanding Dropout (C2W1L07)
DeepLearningAI
Building Blocks of a Deep Neural Network (C1W4L05)
DeepLearningAI
Why Non-linear Activation Functions (C1W3L07)
DeepLearningAI
Computing Neural Network Output (C1W3L03)
DeepLearningAI
Backpropagation Intuition (C1W3L10)
DeepLearningAI
Train/Dev/Test Sets (C2W1L01)
DeepLearningAI
Deep L-Layer Neural Network (C1W4L01)
DeepLearningAI
Random Initialization (C1W3L11)
DeepLearningAI
Other Regularization Methods (C2W1L08)
DeepLearningAI
Normalizing Inputs (C2W1L09)
DeepLearningAI
Derivatives Of Activation Functions (C1W3L08)
DeepLearningAI
Parameters vs Hyperparameters (C1W4L07)
DeepLearningAI
Vectorizing Across Multiple Examples (C1W3L04)
DeepLearningAI
What does this have to do with the brain? (C1W4L08)
DeepLearningAI
Dropout Regularization (C2W1L06)
DeepLearningAI
Vanishing/Exploding Gradients (C2W1L10)
DeepLearningAI
Basic Recipe for Machine Learning (C2W1L03)
DeepLearningAI
Bias/Variance (C2W1L02)
DeepLearningAI
Forward Propagation in a Deep Network (C1W4L02)
DeepLearningAI
Weight Initialization in a Deep Network (C2W1L11)
DeepLearningAI
Numerical Approximations of Gradients (C2W1L12)
DeepLearningAI
Regularization (C2W1L04)
DeepLearningAI
Why Regularization Reduces Overfitting (C2W1L05)
DeepLearningAI
More on: Neural Network Basics
View skill →Related Reads
📰
📰
📰
📰
I regret reviewing for AAAI [D]
Reddit r/MachineLearning
GenTrace: Automating Telecom Network Fault Diagnosis from Raw Packet Traces
Medium · Machine Learning
GPU Rental Guide: H100 vs A100 vs L40S vs RTX4090
Dev.to · YingSuan AI
Kafka Consumer Lag Is Growing. Your API Is Healthy While the Business Falls Behind.
Medium · Programming
🎓
Tutor Explanation
DeepCamp AI