Foundations

Reinforcement Learning

RL algorithms, reward modelling, RLHF, policy gradients, Q-learning and multi-agent RL

858
lessons
Skills in this topic
View full skill map →
RL Foundations
beginner
Formalise a problem as an MDP
Policy Gradient Methods
intermediate
Implement REINFORCE from scratch
RLHF & Alignment
advanced
Describe the RLHF pipeline end-to-end
All Reads (218) Articles (48)Blog Posts (66)Tutorials (6)Research Papers (98)
“The Hidden Problem with Pass-Rate Rewards in Reinforcement Learning for Code Generation”
Medium · AI 🎮 Reinforcement Learning ⚡ AI Lesson 17h ago
“The Hidden Problem with Pass-Rate Rewards in Reinforcement Learning for Code Generation”
If you’ve trained reinforcement learning models for code generation, you’ve probably used pass rate as the reward. Continue reading on Medium »
“The Hidden Problem with Pass-Rate Rewards in Reinforcement Learning for Code Generation”
Medium · Machine Learning 🎮 Reinforcement Learning ⚡ AI Lesson 17h ago
“The Hidden Problem with Pass-Rate Rewards in Reinforcement Learning for Code Generation”
If you’ve trained reinforcement learning models for code generation, you’ve probably used pass rate as the reward. Continue reading on Medium »
How Machines Learn to Make Decisions: A Practitioner’s Guide to Reinforcement Learning
Medium · Machine Learning 🎮 Reinforcement Learning ⚡ AI Lesson 21h ago
How Machines Learn to Make Decisions: A Practitioner’s Guide to Reinforcement Learning
Imagine a thermostat that has to decide, right now, whether to turn the heating on. A simple version just checks the current temperature… Continue reading on Me
When Machines Learned to Learn: A Brief Story of Reinforcement Learning
Medium · AI 🎮 Reinforcement Learning ⚡ AI Lesson 1d ago
When Machines Learned to Learn: A Brief Story of Reinforcement Learning
How a theory of reward became the key to both artificial intelligence and understanding the human brain Continue reading on Medium »
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 4d ago
Deconstructing Off-Policy Ratios: Entropy-Scaled Trust Regions for Asynchronous Reinforcement Learning
arXiv:2607.22186v1 Announce Type: new Abstract: Asynchronous reinforcement learning (RL) accelerates large language model (LLM) post-training by overlapping rol
I Taught an Agent to Act Directly - No Q-Values Needed (Day 6: REINFORCE)
Dev.to · Madhumitha Kolkar 🎮 Reinforcement Learning ⚡ AI Lesson 6d ago
I Taught an Agent to Act Directly - No Q-Values Needed (Day 6: REINFORCE)
SERIES: Learning RL and JAX in Public - from zero to DeepMind :) Days 4 and 5 were value-based...
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 1w ago
It Takes 8 Tokens: Weak-to-Strong Off-Policy RL via Auxiliary Branches
arXiv:2607.16205v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards has emerged as a standard approach for enhancing reasoning in lar
A Practical Guide to Implementing the REINFORCE Algorithm in Python (Part 5)
Medium · Machine Learning 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
A Practical Guide to Implementing the REINFORCE Algorithm in Python (Part 5)
Learn how to build the REINFORCE algorithm from scratch using Python, PyTorch, and Gymnasium with a step-by-step, beginner-friendly… Continue reading on Medium
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 3w ago
Gimitest: A Comprehensive Tool for Testing Reinforcement Learning Policies
arXiv:2607.07029v1 Announce Type: cross Abstract: Reinforcement learning (RL) policies can be unsafe and vulnerable to attacks. Ensuring their reliability is of
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 3w ago
RLVP: Penalize the Path, Reward the Outcome
arXiv:2607.07435v1 Announce Type: cross Abstract: Agents acting on our behalf in the real world (e.g. placing phone calls) must learn online from costly, often
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 3w ago
Self-Review Reinforcement Learning (SRRL) with Cross-Episode Memory and Policy Distillation
arXiv:2607.05541v1 Announce Type: cross Abstract: Reinforcement Learning is commonly used to train large language models using environmental feedback. In applie
Offline Reinforcement Learning — Learning When You Can’t Try Things
Medium · Python 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
Offline Reinforcement Learning — Learning When You Can’t Try Things
Algorithms in Python — Advanced Reinforcement Learning, Part 7 Continue reading on Medium »
A Practical Guide to Implementing the REINFORCE Algorithm in Python(Part 3)
Medium · Machine Learning 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
A Practical Guide to Implementing the REINFORCE Algorithm in Python(Part 3)
Learn how to build the REINFORCE algorithm from scratch using Python, PyTorch, and Gymnasium with a step-by-step, beginner-friendly… Continue reading on Medium
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 3w ago
Ensemble Elastic DQN: A Step Dependent Ensemble Approach for Reducing Overestimation in Deep Value-Based Reinforcement Learning
arXiv:2506.05716v2 Announce Type: replace-cross Abstract: Deep Q-Networks (DQN) can suffer from overestimation bias because bootstrapped targets use a maximisat
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 3w ago
The Three Regimes of Offline-to-Online Reinforcement Learning
arXiv:2510.01460v4 Announce Type: replace-cross Abstract: Offline-to-online reinforcement learning (RL) has emerged as a practical paradigm that leverages offli
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 3w ago
Safe RLHF Beyond Expectation: Stochastic Dominance for Universal Spectral Risk Control
arXiv:2603.10938v2 Announce Type: replace-cross Abstract: Safe Reinforcement Learning from Human Feedback (RLHF) typically enforces safety through expected cost
Soft Actor-Critic — Exploration Becomes Part of the Objective
Medium · Machine Learning 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
Soft Actor-Critic — Exploration Becomes Part of the Objective
Algorithms in Python— Advanced Reinforcement Learning, Part 5 Continue reading on Medium »
Soft Actor-Critic — Exploration Becomes Part of the Objective
Medium · Python 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
Soft Actor-Critic — Exploration Becomes Part of the Objective
Algorithms in Python— Advanced Reinforcement Learning, Part 5 Continue reading on Medium »
A Practical Guide to Implementing the REINFORCE Algorithm in Python(Part 2)
Medium · Machine Learning 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
A Practical Guide to Implementing the REINFORCE Algorithm in Python(Part 2)
Learn how to build the REINFORCE algorithm from scratch using Python, PyTorch, and Gymnasium with a step-by-step, beginner-friendly… Continue reading on Medium
RLHF vs Agent RL: A Model Weight Diff Study
Medium · Machine Learning 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
RLHF vs Agent RL: A Model Weight Diff Study
Weight Watchers: I put open models on the scale. This one started as a family-fork story and turned into something better: a… Continue reading on Medium »
RLHF vs Agent RL: A Model Weight Diff Study
Medium · LLM 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
RLHF vs Agent RL: A Model Weight Diff Study
Weight Watchers: I put open models on the scale. This one started as a family-fork story and turned into something better: a… Continue reading on Medium »
RLHF: How AI Learned to Listen to Us
Medium · Machine Learning 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
RLHF: How AI Learned to Listen to Us
A quick moment of recognition Continue reading on Medium »
RLHF: How AI Learned to Listen to Us
Medium · Data Science 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
RLHF: How AI Learned to Listen to Us
A quick moment of recognition Continue reading on Medium »
RLHF: How AI Learned to Listen to Us
Medium · ChatGPT 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
RLHF: How AI Learned to Listen to Us
A quick moment of recognition Continue reading on Medium »
Reinforcement Learning from Scratch (Part 4): Monte Carlo Methods and Temporal Difference…
Medium · Data Science 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
Reinforcement Learning from Scratch (Part 4): Monte Carlo Methods and Temporal Difference…
A beginner-friendly guide to Monte Carlo Methods in Reinforcement Learning, covering prediction, control, ε-greedy exploration, importance… Continue reading on
Twin Delayed DDPG — Fixing the Value That Lied
Medium · Python 🎮 Reinforcement Learning ⚡ AI Lesson 3w ago
Twin Delayed DDPG — Fixing the Value That Lied
Algorithms in Python — Advanced Reinforcement Learning, Part 4 Continue reading on Medium »
Snapshot Once, Rollout a Thousand Times: A Practical RL Setup for Coding Agents
Dev.to · Sebastian Buzdugan 🎮 Reinforcement Learning ⚡ AI Lesson 1mo ago
Snapshot Once, Rollout a Thousand Times: A Practical RL Setup for Coding Agents
Your GPUs aren't the RL bottleneck, rebuilding the environment is. Snapshot the world once, fork it into thousands of rollouts. Real numbers, runnable harness.
Trust Region Policy Optimisation — The Exact Method PPO Approximates
Medium · Python 🎮 Reinforcement Learning ⚡ AI Lesson 1mo ago
Trust Region Policy Optimisation — The Exact Method PPO Approximates
Algorithms in Python — Advanced Reinforcement Learning, Part 2 Continue reading on Medium »
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 1mo ago
Hierarchical Reinforcement Learning in StarCraft Micromanagement with Influence Maps and Cluster-based Scripts
arXiv:2606.30092v1 Announce Type: new Abstract: Real-time strategy (RTS) games present significant AI challenges, characterized by expansive state-action spaces
Proximal Policy Optimisation — The Clip That Made Policy Gradients Reliable
Medium · Machine Learning 🎮 Reinforcement Learning ⚡ AI Lesson 1mo ago
Proximal Policy Optimisation — The Clip That Made Policy Gradients Reliable
Algorithms in Python — Advanced Reinforcement Learning, Part 1 Continue reading on Medium »
Proximal Policy Optimisation — The Clip That Made Policy Gradients Reliable
Medium · Python 🎮 Reinforcement Learning ⚡ AI Lesson 1mo ago
Proximal Policy Optimisation — The Clip That Made Policy Gradients Reliable
Algorithms in Python — Advanced Reinforcement Learning, Part 1 Continue reading on Medium »
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 1mo ago
Tandem Reinforcement Learning with Verifiable Rewards
arXiv:2606.28166v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has significantly improved the reasoning capability of lar
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 1mo ago
Retroactive Advantage Correction: Closed-Form V-Trace Bias Correction for Delay-Aware RLHF
arXiv:2606.27580v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) in production does not always have a synchronous reward sign
Q-Learning From Scratch: Reinforcement Learning in a Gridworld
Dev.to · Devanshu Biswas 🎮 Reinforcement Learning ⚡ AI Lesson 1mo ago
Q-Learning From Scratch: Reinforcement Learning in a Gridworld
No labels, no "correct answer" — just rewards. Reinforcement learning lets an agent figure out the...
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 1mo ago
Deterministic Pareto-Optimal Policy Synthesis for Multi-Objective Reinforcement Learning
arXiv:2606.26397v1 Announce Type: cross Abstract: Real-world decision-making often requires balancing multiple conflicting objectives, a challenge that standard
Deep Q-Networks — When the Q-Table Won’t Fit
Medium · Python 🎮 Reinforcement Learning ⚡ AI Lesson 1mo ago
Deep Q-Networks — When the Q-Table Won’t Fit
Algorithms in Python— Reinforcement Learning, Part 3 Continue reading on Medium »
Why Reinforcement Learning Is Harder in Trading Than in Games Like Chess
Medium · Deep Learning 🎮 Reinforcement Learning ⚡ AI Lesson 1mo ago
Why Reinforcement Learning Is Harder in Trading Than in Games Like Chess
The Fundamental Difference Between Beating a Grandmaster and Beating the Market Continue reading on InsiderFinance Wire »
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 1mo ago
Auto-exploration for online reinforcement learning
arXiv:2512.06244v2 Announce Type: replace-cross Abstract: The exploration-exploitation dilemma in reinforcement learning (RL) is a fundamental challenge to effi
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 1mo ago
An Introduction to Causal Reinforcement Learning
arXiv:2606.24160v1 Announce Type: new Abstract: Causal inference provides a set of principles and tools that allow one to combine data and knowledge about an en
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 1mo ago
Learning to Trigger: Reinforcement Learning at the Large Hadron Collider
arXiv:2606.23993v1 Announce Type: cross Abstract: High-throughput scientific facilities such as the Large Hadron Collider depend on real-time event filtering (\
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 1mo ago
EMAgnet: Parameter-Space EMA Regularization for Policy Gradient Self-Play in Large Games
arXiv:2606.23995v1 Announce Type: cross Abstract: Recent work has established that regularized policy gradient methods such as PPO, when used in self-play, can
Reinforcement Learning with Verifiable Rewards: Why AI is Learning to Grade Its Own Homework
Dev.to · Shrijith Venkatramana 🎮 Reinforcement Learning ⚡ AI Lesson 1mo ago
Reinforcement Learning with Verifiable Rewards: Why AI is Learning to Grade Its Own Homework
Hello, I'm Shrijith Venkatramana. I'm building git-lrc, an AI code reviewer that runs on every...
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 1mo ago
IRumAI: Reinforcement Learning for Indian Rummy
arXiv:2606.21975v1 Announce Type: new Abstract: Despite its massive player base and complex hidden-information dynamics, Indian Rummy has received no reinforcem
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 1mo ago
Select-to-Act: Hierarchical Reinforcement Learning via Adaptive Language Guidance
arXiv:2606.22350v1 Announce Type: cross Abstract: Reinforcement Learning (RL) has been widely applied to sequential decision-making, yet it often suffers from p
ArXiv cs.AI 🎮 Reinforcement Learning 📄 Paper ⚡ AI Lesson 1mo ago
Hierarchical Reinforcement Learning for Sparse-Reward Search in Commutative Algebra
arXiv:2606.22922v1 Announce Type: cross Abstract: Applying machine learning techniques to solving long-standing mathematical conjectures can be particularly cha
Medium · LLM 🎮 Reinforcement Learning ⚡ AI Lesson 1mo ago
Reward hacking in Reinforcement learning
A field guide to reward hacking in GRPO — why it happens, how it hides, and what actually fixes it Continue reading on Medium »
SARSA — The On-Policy Sibling That Learns to Play It Safe
Medium · Machine Learning 🎮 Reinforcement Learning ⚡ AI Lesson 1mo ago
SARSA — The On-Policy Sibling That Learns to Play It Safe
Algorithms in Python — Reinforcement Learning, Part 2 Continue reading on Medium »
SARSA — The On-Policy Sibling That Learns to Play It Safe
Medium · Python 🎮 Reinforcement Learning ⚡ AI Lesson 1mo ago
SARSA — The On-Policy Sibling That Learns to Play It Safe
Algorithms in Python — Reinforcement Learning, Part 2 Continue reading on Medium »