Learner feedback on Vizuara's LLM Inference Engineering Workshop

Key Takeaways

[music] [music] >> My name is Sita Sitaraman. I lead the innovation digital transmission practice for one of the largest investment banks. And this program from VishwaLab on inference engineering workshop is not just about pre-training, it's also how do these LLMs serve? In fact, we were exposed to multiple concepts right from KV cache and and and many other images and so many other possibilities of attention mechanisms. >> I also appreciated the exposure to experts from different AI ecosystem. We had opportunities to interact with engineers and researchers who had worked at organizations like Apple, Ray, and other frontier AI labs. Listening to their experiences provide a valuable industry perspectives that are difficult to get from traditional courses. Sessions from creators and contributors building real AI products made the learning experience even more practical and inspiring. Um like for example, we had a talk with the creator of small chat which has around close 2,000 GitHub stars. >> Hello everyone. My name is David. An AI entrepreneur and engineer based out in Tokyo, Japan. I have taken many AI courses throughout my career. But the inference engineering program by Dr. Raj and team VishwaLab Labs absolutely outstanding and stands out because it focuses on what truly matters in production AI. Most programs teach you [music] how to use AI model. This program teaches you how to optimize, scale, and deploy them efficiently in the real world applications. But, what really impressed me the most is Dr. Raj's ability who can explain complex concepts with clarity and depth in all perspective. Now, if you're serious about becoming an AI engineer, understands not just only model the infrastructures and system behind them, I highly recommend you to join this program and I hope to see you soon in the next Visual AI Labs AI program. See you. Bye-bye. >> Hi, everyone. I've taken several courses from Visual AI in the past, and Dr. Raj and Dr. Sridhar and Dr. Rajat, they have also included a lot of practical [music] use cases as well that currently applied in industry. I particularly enjoyed the AI inference course. I was looking for a good resource for a while, and when I went through the course by Dr. Raj, I found it be really, really helpful to understand many of the concepts. >> Hello, friends. Nice to meet you. My name is Suchit Smith. I recently took Visual AI's inference engineering workshop. I very much liked the course given its comprehensiveness and attention to detail, and I very highly recommend it to anyone who wants to learn more about inference in general, particularly with regard to LLM. >> Hi. My name is Sriharsha. I took inference engineering course from Visual AI. It was taught by Raj Dandekar. I would say the course was fantastic. I learned a lot. It taught us from all the way from basics to like advanced stuff where we kind of learned how do we kind of tweak the knobs to do the inference serving better. I think I really liked Raj's way of teaching where he was like, instead of presenting us some facts and asking us to remember, he would present a problem statement and ask us to come up with ideas like what could you do differently to solve this problem. And that kind of helped us build this intuition of the problems to solution from all the way from ground up to like the advanced methods. I think in the course there was like a lot of stuff that was actually covered all the way from like basics to like KV cash management to different attention mechanisms to like more advanced serving methodologies. And he even covered like most important stuff like for example the v l l m and how do you kind of tune the knobs in v l l m and what it's not just about how do you tune because the knobs and these things are available in the documentation but the reasons behind it like why should you kind of increase for example why should you go for a higher GPU or a bigger GPU versus like when would you do like a tweaking of like quantization for example like what is the rationality behind it and also the math involved behind it. For example he kind of really dwelt uh dwelt deep into like calculating like what would be the overall estimated KV cash size for example from first principles. So I think this was really good in terms of getting that intuition. Also accompanying the course is like the book that he wrote which he generously shared with us. It has about more than 20 plus chapters about inference engineering. It kind of helps us like the kind of format that we took was like take the course and now I'm also like reading this book this book which is kind of helping me to reinforce the ideas. So I think that was really really great. Besides we also had like guest lectures where there was focus on llama.cpp. There was like open claw model session. There's a multi model inference session and more to come. I think overall it was like a great learning experience for me. I would like highly recommend anyone who's taking [music] this course. Actually this is actually a second part of the trilogy. The first one was the GPU based training and now this is like inference engineering and the third one is basically kernel engineering. So, I really wish him all the best for the third course. I would be definitely enrolling for the third third course. And [music] and also this course, I think one other point I would say is this is really really something that who is who is who wants to become an inference engineer, I think I would for them it is obvious that they have to take this course. I think it really helps. But even if you're not into inference engineering like as a as a career goal, but let's say you're trying to become a general machine learning model developer, I think having an understanding of what the inference stack is like, what knobs to tune that is an important part. Because earlier I was under the conception that [music] oh, if you just train a model and hand it off to an inference engineer, they will just kind of just put it on the server and that's it. But actually there's a so much of science and engineering and math that is involved behind the screens and having an understanding of all these things helps you during the model development phase itself because for example, what kind of attention mechanism has to be decided in the training time itself. What kind of quantization could help because if you're doing quantization away training, that would also be like something based on the inference, you'd kind of pick it up. Similarly, not just for these two people, one is like inference or general ML model developers, but also for like management role, people who are in like management role and who makes the decisions also for them it is a good course because it helps them understand like what kind of amount of importance and how how do should they allocate resources and stuff for in for inference, which is in fact a major part, right? For a lot of companies, sure they might be using close source, but also for a lot of companies, hope they tend to use open source models and they tend to serve their own models. So, I think having a good understanding of this saves them cost, time, and also gives great performance for the users. So, yeah, I would highly recommend this course for anyone who wants to have general experience as well as a very focused understanding on on inference stack. Thank you so much and I wish best of luck for Raj for his third course, the kernel engineering course as well. Thank you. >> Hello. So, I just finished the phase two of the inference engineering workshop by Visual AI. Dr. Raj has done an amazing job design of this course and the delivery as well. There are so many resources that he has curated right from the book, the lectures, the handwritten notes. What I like most about this course is the way Dr. Raj has delivered it. So, starting from the fundamentals or what he calls as the first principles of thinking and then taking into the depths of each concept. We looked at the many attention mechanisms and how it affect inference. He has also provided bunch of codes and you know, he has created so many visualization that it make makes it very easy for you to understand all these concepts. Amazing work so here. One more thing that I would like to point out is when you look at the price you might feel that is expensive and but I would say that once you go through the course will set you up for understanding inference engineering that is used in real life. That will definitely give you you know, 10x return on investment. So, I would definitely recommend this course to anyone who wants to get into inference engineering because that's where the future lies in my opinion because open source models are going to be the future and then you have to serve them in the right way so that it is accessible for everyone. >> Hi everyone. My name is Manoj and I recently completed the inference engineering boot camp at Visual AI. I have taken many courses over years in machine learning and data engineering and AI. But what made this boot camp different was [music] its focus on the inference engineering which is a topic that is becoming increasingly important as AI systems move from research to production. What impressed me most was the depth and the relevance of the curriculum. The content wasn't limited to concepts concepts that were popular a year ago. The instructors continuously incorporated the latest developments from the AI industry. For example, technologies and techniques that had only recently released we [music] were already being discussed and analyzed in the boot camp. So that level of effort and commitment to staying current and updated is something I genuinely appreciate. Another aspect I found extremely valuable was the way complex concepts were taught. The visual walkthroughs especially which is which I felt really great helped me bridge the gap between the theory and intuition. Topics that initially seemed difficult became much easier to understand because the instructors invested especially Dr. Raj invested a lot of time in explaining not just how things [music] work but why they work. The capstone projects deserve a special attention. These were not the kind of projects where you just simply [music] prompt the AI like use chat GPT or Claude and submit the result. They were carefully designed to force you to think deeply about the concepts being taught and learned. Solving them required a strong understanding of in terms optimizing techniques, deployment [music] strategies, and kind of engineering trade-offs, some knobs to tune basically is what I call. They challenged us to become more like a problem solvers rather than tool users. So I also appreciated the exposure to experts from different AI ecosystem. We had opportunities to interact with engineers and researchers who had worked at organizations like Apple, Ray, and other frontier AI labs. Listening to their experiences provide a valuable [music] industry perspectives that are difficult to get from traditional courses. Sessions from creators and contributors building real AI products made the learning experience even more practical and inspiring. Like for example, we had a talk with the creator of small chat which has around close to 2,000 GitHub stars or maybe more. What I like most is that this boot camp doesn't simply teach you the collection of topics. It gives you a framework for thinking about modern AI systems, whether it's attention mechanisms or quantizations or serving systems or edge deployment, more like optimization techniques or emerging inference architectures. The boot camp provides both breadth and depth. Ultimately, the value you get depends on how much willingly you to you engage with the material. The information and the mentorship and opportunities are all there. The potential is enormous and the more effort you invest, the more you gain from it. For anyone serious about understanding how modern AI systems are actually working and actually built and optimized and deployed, not just how to call an API, I would strongly recommend this boot camp. And a huge thank you to an entire We are in team, especially Dr. Raj, Dr. Sridhar, and Dr. Rajat because they had few involvement in some of the lectures and for the dedication and the quality of the instruction they they went to creating this experience. So, I would really thank and I would encourage >> Hi, I'm Ishani, a senior machine learning engineer. I recently completed the We are in AI inference workshop live. I generally want to share my experience. The depth, [music] that's what stood out. Uh Dr. Raj didn't just skim the surface and he went deep into every topic and more importantly, he took his time to answer the questions that you didn't even know you had. Um as someone already working in a production ML and edge deployment, like uh I had a I came in thinking that I have a >> [music] >> I had a decent uh foundation and but this workshop filled the critical gaps for me how to do GPU utilization, inference optimization, and that's very important topic these days. And concepts that were fuzzy for me suddenly clicked. Um Dr. Rajat and Dr. Sridhar brought the same depth to embodied AI and vision engineering at inference level. And these two areas are honestly not covered well anywhere else at inference level. >> Uh I really like the structure of the course, the way it's been taught, and also in-depth um knowledge that was shared. Yes, I would definitely >> Hi, I'm currently working as an AI/ML engineer at an MNC, and I recently completed the inference engineering course. This course has significantly strengthened my research mindset in the field by taking me through uh both foundational concepts and the latest advancements in the inference system. I now have much clearer understanding of how and where meaningful improvements can be made, and I am actively pursuing research in similar directions. I'm especially grateful to Raj for his dedication. Although the schedule class duration was just 2 hours, he often [music] extended his classes to 3 hours, even sometimes 4 hours, to ensure every topic was covered thoroughly from first principles. I truly appreciate his level of commitment and the depth of learning it enabled. Thank you. >> Hey everyone. Uh my name is Aman Kesharwani. I am based in New York. I'm here to give a a review about the inference course by Dr. Rajat here. I currently tried this Visual AI lab. I was part of the whole phase one and phase two. And the couple of things that I want to highlight is there was a lot of interaction during the live lectures with Dr. Raj. This really helped in clarifying the doubts then and there. The other thing was that the lectures were designed in a way so that they covered the entire topic from like front to end and the sessions may have been really long, but that also made sure that you are actually not losing your train of thought when you're actually learning those concepts, so which I also really appreciated. The other thing which I also want to highlight is the visual walkthrough. So Dr. Raj has put like significant amount of time to generate these visual walkthroughs, which can really help you understand the topics very well and they can be also a good revision helper when you want to like revise for the exam. So that was also amazing. And further the capstone projects were like designed very well. They covered like most of the concepts which were actually taught in the class. So they were very well done. They are very different from the toy projects which typical people do. So they also add lot of value to their learning. So I also appreciate that. I also enrolled for the for the research part as well and they have recently provided me for the full road map for the research and I'm actually very excited about that because it's like very well done and lot of information about the separate milestones which needs to be done. So very structured. I would definitely encourage in future for the candidates to go for that and I wish Visual AI lab all the best in all their future workshops and I also look forward to enroll in their our workshops as well. Thank you very much. >> Hi, my name [music] is Sita Sitaraman. I lead the innovation digital transmission practice for one of the largest investment [music] banks. While I was familiar with application oriented AI based application development and doing a lot of stuff around building AI specific applications. I really wanted to go beyond that and this program from Vishwara on inference engineering workshop completely transformed. In fact, this is the second of the series. I was also part of the GPU engineering 5D [music] parallelism and this workshop inference engineering bridge that in terms of it's not just about pre-training, it's also >> [music] >> how do these LLMs In fact, we were exposed to multiple concepts right from KV cache [music] and and and many other images and so many other possibilities of attention mechanisms and it What it helped essentially is it's not about pure theory. We We understood the concepts by at times going into the depth by holding a pen and a paper. There were visuals which was very nicely prepared by Professor Raj which helped us to get that sort of an understanding, go back after the [music] classes and after the workshops and then look at how it actually means visually. So that was quite brilliant. There were capstone projects as well to apply what we have learned to really simulate and act in the capacity of an inference like an inference engineer. Before the workshop, I would say everything looked to me like a black box on inference engineering and after this session, I feel quite comfortable and confident to go about not just training large language models, figuring out all the strategies, but also serving the models. [music] So this is something which is very very useful in the in the evolution of AI that we are seeing. If anybody wants to look at what's behind um, the inference, what's what's behind the working of these LLMs, I would recommend them to go ahead and look at this inference engineering workshop. Brilliant stuff. Thanks to Vishura and thanks to Dr. Raj for helping us get to and get get lot of clarifications through these workshops. Thank you so much. Cheers. Bye. >> Hi, my name is Vineeth and I currently work in the AI and machine learning domain. I first came across Vishura through Dr. Ross building LLMs from [music] scratch series on YouTube. So, what impressed me was that the series did not simply show how to use existing APIs or frameworks. It explained [music] what is actually happening inside a large language model step-by-step by relating the concepts with real world examples. That learning experience increased my interest in areas such as LLM inference, model optimization, and production AI systems. One thing I really appreciate about Vishura is its strong focus on fundamentals. AI tools and frameworks may change very quickly, but a clear understanding of the underlying concepts will always remain important. I also find the live course format very valuable. Unlike only watching recorded videos, live classes provide a structured learning path, direct interaction with the instructors, opportunities to ask questions, and better accountability throughout the course. The discussions, hands-on implementations, guidance on the capstone projects help me connect theoretical concepts with real world production-ready applications. It makes Vishura's learning approach unique. I need to thank Dr. Ross and the entire Vishura team for creating such valuable and practical learning resources. >> If If serious about AI inference and have a genuine enthusiasm for learning, not just following along, this is a workshop for you. I would absolutely recommend it. Thank you. >> It was like a great learning experience for me. I would like highly recommend anyone >> I would highly recommend Ashwara to anyone who is interested in any topics in AI. >> And I very highly recommend it to anyone who wants to learn more about LLM inference and the different techniques involved in optimizing performance. >> I would really encourage anybody who is very serious about their AI career to take this course. >> [music]