George Hotz | Programming | can we fit a LLaMA inside a tinygrad? | $1499 comma.ai/shop/comma-three

Date of stream 08 Mar 2023. from $1499 buy https://comma.ai/shop/comma-three Live-stream chat added as Subtitles/CC - English (Twitch Chat) - three-dot menu icon - Show transcript Source files: - https://github.com/geohot/tinygrad/blob/llama/examples/llama.py - https://github.com/geohot/tinygrad/tree/llama - https://github.com/geohot/tinygrad Follow for notifications: - https://twitch.tv/georgehotz Support George: - https://twitch.tv/subs/georgehotz Programming playlist: - https://www.youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:01:30 guitar tuning and playing 00:04:30 giving advice to elon, woke 00:07:25 llama, trip to india, bengaluru, bike to mumbai 00:08:40 when software was cool 00:11:20 llama, privilege 00:13:45 discord ban, pressure, power 00:16:05 prime, blue crown, money 00:18:00 tiny corp, making value, food 00:22:25 capitalism 00:23:50 facebook LLaMA, apply for access, torrent 00:25:10 ai safety, enjoy the decline 00:28:00 deepmind, google, rants 00:28:40 thinking for yourself, jesus, george is christian, god 00:30:30 a progression to christian 00:31:50 big text, copilot 00:35:10 running on gpu m1 00:39:30 using tinygrad tiny ram usage, float16 00:46:00 METAL=1 00:47:25 numpy size of datatype 00:49:12 interesting, np.frombuffer 00:52:10 problem raw buffers to have a datatype? 00:52:55 python read into buffer, not making copies = fast 00:59:40 loading from disk, making it lazy 01:00:50 tinygrad is the future, replacing programing with dsp paradigm 01:02:17 tinygrad has grown 01:03:50 metal buffer, float16 hard 01:05:20 interview question how big is a float 01:06:40 making metal buffer, supporting dtypes 01:09:40 tinygrad recognize llama 01:10:25 facebook llama arxiv, unbiased model 01:13:25 class Attention, FeedForward 01:21:40 washed heads 01:24:00 class RMSNorm 01:30:25 testing inference 01:35:30 error and slow loading because torch 01:39:00 wasting 0.6 seconds 01:40:45 how transformers work, Linear 01:47:2