Grid cells - Caswell Barry, UCL

Google DeepMind · Advanced ·🤖 AI Agents & Automation ·8y ago

Key Takeaways

The video discusses the development of an artificial agent to test the theory that grid cells support vector-based navigation, and how this can be applied to improve the navigation of artificial agents in virtual environments using techniques such as path integration and vector-based navigation.

Full Transcript

grid cells are type of brain cell and first of all they were discovered in rodents but now we know they exist in mammals in general they seem to be very important for sort of measuring out space our good analogy is that they're like the grid lines on a map except on a map we've used square grid lines and the brains using hexagonal grid lines and you have small scale grid lines and larger scale grid lines originally people used to think they were very important force that we call path integration which is basically updates where you think you are based on how you're moving so I know where I am right now and if I move it 10 meters then I end up somewhere over there but actually we're moving beyond that now while we still think path integration is important focusing on the fact that we believe they function like a map they sort of relate one place to another they also seem to be very important for planning routes so if you wouldn't know the shortest route between here and somewhere else that you've visited then the grid lines would provide you with something called a vector based navigation and allow you to calculate the direct line distance with the recent paper we were trying to do several things so first of all we were trying to test this idea of vector based navigation so our grid cells really the substrate and machinery that allows you to sort of calculate these direct line routes between places but equally we're trying to do think a little bit more now we were trying to see if you could take what we know about how the mammalian brain works and incorporates it into artificial agents to improve the way that they work in this case the way they navigate first of all we were able to show that if we trained an agent or a network to paths integrate then we found that actually developed grid cells so this is remarkable convergence of form in that they looked like red cells in the mammalian brain they had the same hexagonal pattern with more tightly tessellated fields and then we were able to take that network and transfer into a larger network that was now controlling an agent navigating around and that agent was then able to navigate flexibly and demanding virtual environments and so for example one of the tests that we wanted to see whether to do is whether it could take shortcuts we had eight environments that would change while the agent was in there so it might find one long route to the goal but then a new door would open up presenting a shortcut so the agent with grid cells could utilize that shot go straight through to the goal so this is important for several reasons we've confirmed that grid cells are important for database navigations we've shown that we can use an artificial network to conduct neuroscience experiments but also we've shown that incorporating what we know about how the brain solves a problem into an artificial network can provide that network with abilities that wouldn't have otherwise have had [Music]

Original Description

In our most recent paper published in Nature, we developed an artificial agent to test the theory that grid cells support vector-based navigation, in keeping with our overarching philosophy that algorithms used for AI can meaningfully approximate elements of the brain.
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Google DeepMind · Google DeepMind · 53 of 60

1 RL Course by David Silver - Lecture 8: Integrating Learning and Planning
RL Course by David Silver - Lecture 8: Integrating Learning and Planning
Google DeepMind
2 RL Course by David Silver - Lecture 1: Introduction to Reinforcement Learning
RL Course by David Silver - Lecture 1: Introduction to Reinforcement Learning
Google DeepMind
3 RL Course by David Silver - Lecture 2: Markov Decision Process
RL Course by David Silver - Lecture 2: Markov Decision Process
Google DeepMind
4 RL Course by David Silver - Lecture 5: Model Free Control
RL Course by David Silver - Lecture 5: Model Free Control
Google DeepMind
5 RL Course by David Silver - Lecture 6: Value Function Approximation
RL Course by David Silver - Lecture 6: Value Function Approximation
Google DeepMind
6 RL Course by David Silver - Lecture 4: Model-Free Prediction
RL Course by David Silver - Lecture 4: Model-Free Prediction
Google DeepMind
7 RL Course by David Silver - Lecture 3: Planning by Dynamic Programming
RL Course by David Silver - Lecture 3: Planning by Dynamic Programming
Google DeepMind
8 RL Course by David Silver - Lecture 10: Classic Games
RL Course by David Silver - Lecture 10: Classic Games
Google DeepMind
9 RL Course by David Silver - Lecture 7: Policy Gradient Methods
RL Course by David Silver - Lecture 7: Policy Gradient Methods
Google DeepMind
10 Google DeepMind: Ground-breaking AlphaGo masters the game of Go
Google DeepMind: Ground-breaking AlphaGo masters the game of Go
Google DeepMind
11 Match 1 - Google DeepMind Challenge Match: Lee Sedol vs AlphaGo
Match 1 - Google DeepMind Challenge Match: Lee Sedol vs AlphaGo
Google DeepMind
12 Match 2 - Google DeepMind Challenge Match: Lee Sedol vs AlphaGo
Match 2 - Google DeepMind Challenge Match: Lee Sedol vs AlphaGo
Google DeepMind
13 Match 1 15 min Summary - Google DeepMind Challenge Match
Match 1 15 min Summary - Google DeepMind Challenge Match
Google DeepMind
14 Match 3 - Google DeepMind Challenge Match: Lee Sedol vs AlphaGo
Match 3 - Google DeepMind Challenge Match: Lee Sedol vs AlphaGo
Google DeepMind
15 Match 2 15 Minute Summary - Google DeepMind Challenge Match 2016
Match 2 15 Minute Summary - Google DeepMind Challenge Match 2016
Google DeepMind
16 Match 3 15 Minute Summary - Google DeepMind Challenge Match 2016
Match 3 15 Minute Summary - Google DeepMind Challenge Match 2016
Google DeepMind
17 Match 4 - Google DeepMind Challenge Match: Lee Sedol vs AlphaGo
Match 4 - Google DeepMind Challenge Match: Lee Sedol vs AlphaGo
Google DeepMind
18 Match 4 15 Minute Summary - Google DeepMind Challenge Match 2016
Match 4 15 Minute Summary - Google DeepMind Challenge Match 2016
Google DeepMind
19 Match 5 - Google DeepMind Challenge Match: Lee Sedol vs AlphaGo
Match 5 - Google DeepMind Challenge Match: Lee Sedol vs AlphaGo
Google DeepMind
20 Match 5 15 Minute Summary - Google DeepMind Challenge Match 2016
Match 5 15 Minute Summary - Google DeepMind Challenge Match 2016
Google DeepMind
21 DQN SPACE INVADERS
DQN SPACE INVADERS
Google DeepMind
22 DQN Breakout
DQN Breakout
Google DeepMind
23 Asynchronous Methods for Deep Reinforcement Learning: Labyrinth
Asynchronous Methods for Deep Reinforcement Learning: Labyrinth
Google DeepMind
24 Asynchronous Methods for Deep Reinforcement Learning: MuJoCo
Asynchronous Methods for Deep Reinforcement Learning: MuJoCo
Google DeepMind
25 Asynchronous Methods for Deep Reinforcement Learning: TORCS
Asynchronous Methods for Deep Reinforcement Learning: TORCS
Google DeepMind
26 Differentiable neural computer family tree inference task
Differentiable neural computer family tree inference task
Google DeepMind
27 StarCraft II DeepMind feature layer API
StarCraft II DeepMind feature layer API
Google DeepMind
28 DeepMind Health – Partnership with the Royal Free London NHS Foundation Trust
DeepMind Health – Partnership with the Royal Free London NHS Foundation Trust
Google DeepMind
29 DeepMind Health – Michael Wise – a patient's journey
DeepMind Health – Michael Wise – a patient's journey
Google DeepMind
30 Streams – a platform for a digital NHS
Streams – a platform for a digital NHS
Google DeepMind
31 DeepMind Lab - Nav Maze Level 1
DeepMind Lab - Nav Maze Level 1
Google DeepMind
32 DeepMind Lab - Stairway to Melon Level
DeepMind Lab - Stairway to Melon Level
Google DeepMind
33 DeepMind Lab - Laser Tag Space Bounce Level (Hard)
DeepMind Lab - Laser Tag Space Bounce Level (Hard)
Google DeepMind
34 Exploring the mysteries of Go with AlphaGo and China's top players
Exploring the mysteries of Go with AlphaGo and China's top players
Google DeepMind
35 Demis Hassabis on AlphaGo: its legacy and the 'Future of Go Summit' in Wuzhen, China
Demis Hassabis on AlphaGo: its legacy and the 'Future of Go Summit' in Wuzhen, China
Google DeepMind
36 The Future of Go Summit: AlphaGo & Ke Jie match 1 moves analysis
The Future of Go Summit: AlphaGo & Ke Jie match 1 moves analysis
Google DeepMind
37 The Future of Go Summit: AlphaGo & Ke Jie match 2 moves analysis
The Future of Go Summit: AlphaGo & Ke Jie match 2 moves analysis
Google DeepMind
38 The Future of Go Summit: Pair Go moves analysis
The Future of Go Summit: Pair Go moves analysis
Google DeepMind
39 The Future of Go Summit: AlphaGo & Ke Jie match 3 moves analysis
The Future of Go Summit: AlphaGo & Ke Jie match 3 moves analysis
Google DeepMind
40 Emergence of Locomotion Behaviours in Rich Environments
Emergence of Locomotion Behaviours in Rich Environments
Google DeepMind
41 StarCraft II 'mini games' for AI research
StarCraft II 'mini games' for AI research
Google DeepMind
42 Trained and untrained agents play StarCraft II full 1vs1 game
Trained and untrained agents play StarCraft II full 1vs1 game
Google DeepMind
43 DeepMind open source PySC2 toolset for Starcraft II
DeepMind open source PySC2 toolset for Starcraft II
Google DeepMind
44 ICML 2017: Test of Time Award (Sylvain Gelly & David Silver)
ICML 2017: Test of Time Award (Sylvain Gelly & David Silver)
Google DeepMind
45 Ke Jie and DeepMind's Go Ambassador Fan Hui review the 3rd AlphaGo vs Ke Jie game
Ke Jie and DeepMind's Go Ambassador Fan Hui review the 3rd AlphaGo vs Ke Jie game
Google DeepMind
46 Ke Jie and DeepMind's Go Ambassador Fan Hui review the 1st AlphaGo vs Ke Jie game
Ke Jie and DeepMind's Go Ambassador Fan Hui review the 1st AlphaGo vs Ke Jie game
Google DeepMind
47 Ke Jie and DeepMind's Go Ambassador Fan Hui review the 2nd AlphaGo vs Ke Jie game
Ke Jie and DeepMind's Go Ambassador Fan Hui review the 2nd AlphaGo vs Ke Jie game
Google DeepMind
48 AlphaGo Zero: Discovering new knowledge
AlphaGo Zero: Discovering new knowledge
Google DeepMind
49 AlphaGo Zero: Starting from scratch
AlphaGo Zero: Starting from scratch
Google DeepMind
50 Defining principles for tech companies in the NHS: DeepMind Health's Collaborative Listening Summit
Defining principles for tech companies in the NHS: DeepMind Health's Collaborative Listening Summit
Google DeepMind
51 A systems neuroscience approach to building AGI - Demis Hassabis, Singularity Summit 2010
A systems neuroscience approach to building AGI - Demis Hassabis, Singularity Summit 2010
Google DeepMind
52 Retour de Rémi Munos en France et ouverture de DeepMind Paris
Retour de Rémi Munos en France et ouverture de DeepMind Paris
Google DeepMind
Grid cells - Caswell Barry, UCL
Grid cells - Caswell Barry, UCL
Google DeepMind
54 DeepMind Health Research and Moorfields Eye Hospital NHS Foundation Trust: What our research shows
DeepMind Health Research and Moorfields Eye Hospital NHS Foundation Trust: What our research shows
Google DeepMind
55 DeepMind Health Research and Moorfields Eye Hospital NHS Foundation Trust: A Patient's Story
DeepMind Health Research and Moorfields Eye Hospital NHS Foundation Trust: A Patient's Story
Google DeepMind
56 Deep Learning 3: Neural Networks Foundations
Deep Learning 3: Neural Networks Foundations
Google DeepMind
57 Deep Learning 5: Optimization for Machine Learning
Deep Learning 5: Optimization for Machine Learning
Google DeepMind
58 Deep Learning 8: Unsupervised learning and generative models
Deep Learning 8: Unsupervised learning and generative models
Google DeepMind
59 Reinforcement Learning 1: Introduction to Reinforcement Learning
Reinforcement Learning 1: Introduction to Reinforcement Learning
Google DeepMind
60 Deep Learning 2: Introduction to TensorFlow
Deep Learning 2: Introduction to TensorFlow
Google DeepMind

The video discusses how grid cells in the mammalian brain support vector-based navigation, and how this can be applied to improve the navigation of artificial agents in virtual environments. The development of an artificial agent that can navigate flexibly and demanding virtual environments is also presented.

Key Takeaways
  1. Develop an artificial network that can path integrate and develop grid cells
  2. Train the network to navigate in virtual environments
  3. Test the network's ability to take shortcuts and navigate flexibly
  4. Apply the insights from neuroscience to improve the navigation of AI agents
💡 Incorporating what we know about how the brain solves a problem into an artificial network can provide that network with abilities that it wouldn't have otherwise had

Related Reads

📰
You don’t need to learn to code anymore.
Discover how AI agents can generate playable games without requiring manual coding, revolutionizing game development
Medium · Programming
📰
The One-Person Billion-Dollar Company: How AI Is Rewriting the Rules of Business Forever
Learn how AI is enabling one-person billion-dollar companies, rewriting traditional business rules
Medium · AI
📰
Your Agent’s Safety Classifier Approved the Malware. It Was Working Exactly as Designed.
Learn how a simple 4-word prompt can trick AI agents into generating malware, highlighting the importance of robust safety classifiers
Medium · AI
📰
The Generative and Agentic Frontier in Financial Services: A Comprehensive Analysis of AI in…
Explore the intersection of AI, generative models, and agentic systems in financial services to understand their transformative potential
Medium · AI
Up next
CLI vs API vs MCP Explained | Key Differences for AI Engineers
Pavithra’s Podcast
Watch →