📰 Lilian Weng's Blog

Special thanks to John Schulman for a lot of super valuable feedback and direct edits on this post. Test time compute ( Graves et al. 2016 , Ling, et al. 2017 ,

Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely l

Hallucination in large language models usually refers to the model generating unfaithful, fabricated, inconsistent, or nonsensical content. As a term, hallucina

The use of large language models in the real world has strongly accelerated by the launch of ChatGPT. We (including my team at OpenAI, shoutout to them) have in

Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT , GPT-Engineer and Ba

Prompt Engineering , also known as In-Context Prompting , refers to methods for how to communicate with LLM to steer its behavior for desired outcomes without u

Many new Transformer architecture improvements have been proposed since my last post on “The Transformer Family” about three years ago. Here I did a

[Updated on 2023-01-24: add a small section on Distillation .] Large transformer models are mainstream nowadays, creating SoTA results for a variety of tasks. T

Neural networks are well known to be over-parameterized and can often easily fit data with near-zero training loss with decent generalization performance on tes

This is part 2 of what to do when facing a limited amount of labeled data for supervised learning tasks. This time we will get some amount of human labeling wor

[Updated on 2022-03-13: add expert choice routing .] [U

The goal of contrastive representation learning is to learn such an embedding space in which similar sample pairs stay close to each other while dissimilar ones

Large pretrained language models are trained over a sizable collection of online data. They unavoidably acquire certain toxic behavior

[Updated on 2020-06-17: Add “exploration via disagreement” in the “Forward Dynamics” section . Exploitation versus ex

[Updated on 2020-01-09: add a new section on Contrastive Predictive Coding ]. [Updated on 2020-04-13: add a “Momentum Contra

Stochastic gradient descent is a universal choice for optimizing deep learning models. However, it is not the only option. With bl

In my earlier post on meta-learning , the problem is mainly defined in the context of few-shot classificati

In Robotics, one of the hardest problems is how to make your model transfer to the real world. Due to the sample inefficiency of deep RL algorithms and the cost

[Updated on 2019-02-14: add ULMFiT and GPT-2 .] [Updated on 2020-02-29: add ALBERT .] <span class="updat

[Updated on 2019-10-01: thanks to Tianhao, we have

So far, I&rsquo;ve written about two types of generative models, GAN and VAE . Neither of them explicitly learns the probability density function of real da

The full implementation is available in lilianweng/deep-reinforcement-learning-gym In the previous two posts, I have introduced the algorithms of many deep rein

[Updated on 2018-06-30: add two new policy gradient methods, SAC and D4PG .] [Updated on 2018-09-30: ad

[Updated on 2020-09-03: Updated the algorithm of SARSA and Q-learning so that the diffe