Limits of Transformers on Compositionality

In this video, we dive into the paper: 'Faith and Fate: Limits of Transformers on Compositionality'. The authors question whether transformer-based models are truly capable of compositional reasoning or are merely performing analog pattern matching when solving complex problems. They suggest that transformers may not be capable of compositional reasoning, evidenced by common trivial mistakes. Chapters: 00:00 - Introduction 00:26 - Questioning transformer-based models 00:43 - Lack of ability to perform compositional reasoning 01:00 - Three tasks to assess capabilities 01:33 - Hypotheses for the paper 02:17 - Formulating compositional tasks as computation graphs 03:00 - Three tasks explained 03:13 - Zero shot accuracy and complexity metrics 03:48 - Poor generalization and partial solutions 04:03 - Ratio of correct nodes and decreasing correct answers 04:13 - Mathematical argument for reducing correct answers 04:39 - Conclusion and final thoughts 🔗 Links: - Paper: https://arxiv.org/abs/2305.18654 - Subscribe!: https://www.youtube.com/channel/UCu8WF59Scx9f3H1N_FgZUwQ - Join Medium: https://lucas-soares.medium.com/membership - Tiktok: https://www.tiktok.com/@enkrateialucca?lang=en - Twitter: https://twitter.com/LucasEnkrateia - LinkedIn: https://www.linkedin.com/in/lucas-soares-969044167/ - Music from www.epidemicsound.com Thanks for watching and see you next time! Cheers :) . #transformers #artificialintelligence #chatgpt #gpt4 #compositionality #problem-solving