9 Examples of Specification Gaming

AI systems do what you say, and it's hard to say exactly what you mean. Let's look at a list of real life examples of specification gaming! How to Help: https://aisafety.info/questions/8TJV/... https://www.aisafety.com/ Related Videos from me: Reward Hacking: https://youtu.be/92qDfT8pENs Reward Hacking Reloaded: https://youtu.be/46nsTFfsBuc What Can We Do About Reward Hacking?: https://youtu.be/13tZ9Yia71c Sources: The list: http://tinyurl.com/specification-gaming The blogpost this video is based on: https://vkrakovna.wordpress.com/2018/04/02/specification-gaming-examples-in-ai/ The newer blogpost that happened while I was making this video: https://deepmind.com/blog/article/Specification-gaming-the-flip-side-of-AI-ingenuity Links: Eliciting latent knowledge from AI: https://www.alignmentforum.org/posts/rxoBY9CMkqDsHt25t/eliciting-latent-knowledge-elk-distillation-summary Do current AI models show deception: https://aisafety.info/questions/9AL4/ What is deceptive alignment: https://aisafety.info/questions/8EL6/ Scaling Laws: https://aisafety.info/questions/7750/ LLMs as simulators: https://aisafety.info/questions/9FQK/ (Explosion graphic from videezy.com) Thanks to my wonderful patrons: https://www.patreon.com/robertskmiles Gladamas James Steef Scott Worley Chad Jones Chris Canal David Reid Francisco Tolmasky Frank Kurka Jake Ehrlich JJ Hepboin Kellen lask Michael Andregg Pedro A Ortega Peter Rolf Said Polat Teague Lasser Allen Faure Bryce Daifuku Clemens Arbesser Eric James Erik de Bruijn Jason Hise jugettje dutchking Ludwig Schubert Qeith Wreid Andrew Harcourt anul kumar sinha Ben Glanton Benjamin Watkin Cooper Lawton Duncan Orr Eric Scammell Euclidean Plane Ian Munro Igor Keller Ingvi Gautsson James Hinchcliffe Jeroen De Dauw Jon Halliday Jonatan R Julius Brash Jérôme Beaulieu Laura Olds Luc Ritchie Lupuleasa Ionuț Michael Greve Nathan Fish Nicholas Guyett Paul Hobbs Sean Gibat Sebastian Birjoveanu Shevis Johnson Taras Bobrovytsky Tim Neilson Tom O'C