Nat Friedman (Github CEO) — Reading ancient scrolls, open source, & AI

Key Takeaways

we have 600 plus kind of roughly in tax Scrolls that we can't open and I heard I heard about this and I thought that was incredibly exciting like the idea that there was information from 2000 years in the past we don't know what's in these things we could read all of them then that would give us approximately a doubling of the total tax that we have from Antiquities there are thousands more Papyrus schools in there and we now have the techniques to read them then there's gold in that mud and you know it's got to be dug out I just fundamentally don't believe the world is efficient and so if I see an opportunity to do something I used to but I no longer have a reflexive reaction that says oh that must not be a good idea if it were a good idea someone would already be doing okay today I have the pleasure of speaking with Matt Friedman who was the CEO of GitHub from 2018 to 2021 before that he started and sold two companies companies Simeon and xamarin and he is also the founder of AI Grant and California and most recently he is the organizer and funder of the squirrel prize which is what we'll start this conversation so Nat do you want to tell the audience about what the score prize is well we're calling it The Vesuvius challenge okay and uh this is just this crazy and exciting thing I feel like incredibly honored to have gotten caught up in but uh a couple of years ago I was reading it was the midst of covid and I think we were in lockdown and like everybody was sort of falling into internet rabbit holes and I just started reading about the eruption of Mount Vesuvius in Italy about 2 000 years ago and it turns out that when Vesuvius erupted uh it was ad79 it destroyed all the nearby towns everyone knows about Pompeii but there was another nearby town called Herculaneum and Herculaneum was sort of like the Beverly Hills to Pompeii so big Villas big houses fancy people and in Herculaneum there was one Villa in particular it was enormous and it had once been owned by the father-in-law of Julius Caesar and so well connected guy and it was full of beautiful statues and Marbles and art but it was also the home to a huge library of Papyrus Scrolls and so when the Villa was buried the volcano actually it spit out enormous quantities of mud and Ash and it buried Herculaneum in particular in something like 20 meters of material so it wasn't like a thin layer it was a very thick layer those towns were buried and forgotten for hundreds of years no one even knew exactly where they were until the 1700s and so in 1750 a farm worker who was digging a well kind of in the outskirts of of Herculaneum struck this marble Paving Stone of a path it had been at this huge Villa and of course he was pretty far down when he did that he was you know 60 feet down and then subsequently this Swiss engineer came in and started digging tunnels from that well shaft and they found all these treasures and that was sort of the spirit at the time was like looting you know they were taking out like incredible they would if they encountered a wall they would just bust through it and they were taking out these beautiful Bronze Statues that had survived and along the way they kept encountering these lumps of what looked like charcoal they weren't sure what they were and many were apparently thrown away until someone noticed a little bit of writing on one of them and they realized they were Papyrus Scrolls and there were hundreds there were even thousands of them and so they had uncovered really the this enormous library is the only library ever to have sort of survived in any form even though it was badly damaged you know they were sort of carbonized very fragile deformed the only one that survived since Antiquity in in in the open air these Papyrus Scrolls and like a Mediterranean climate they rot and they declare Decay quickly and so they'd have to be recopied by monks like every 100 years or so maybe even less and so we only have it's estimated you know something less than one percent less than one percent of all the writing from that period and so to find underground hundreds of definitely not in good condition but still present you know Papyrus Scrolls were on a few of them you can make out the lettering was like this enormous Discovery people immediately in a well-meaning attempt to read them started trying to open them but they're they're really fragile like they you know they're like they turn to Ash in your hand and so hundreds were destroyed people did things like cut them with daggers down the middle and you know a bunch of little pieces would flake off and they'd try to like get a few letters off of a couple of pieces and then eventually there was a monk named piagio who's an Italian Monk and he devised this machine kind of under the care of the Vatican to unroll these things very very slowly like half a centimeter a day something like that in a typical scroll I think could be 15 or 20 or 30 feet long and manage tube successfully enroll a few of these and they found on them Greek philosophical texts in the Epicurean tradition by this little-known philosopher named philodimus but but we got kind of new text from Antiquity which is you know not a thing that happens all the time eventually people stopped trying to physically unroll these things because so many were destroyed in fact some attempts to physically unroll the Scrolls continued even into like the 2000s like 1990s 2000s and and they were destroyed so the current situation is we have 600 plus kind of roughly in tax Scrolls that we can open and I heard I heard about this and I thought that was incredibly exciting like the idea that there was information from 2000 years in the past we don't know what's in these things and obviously people are trying to develop new ways and new technologies to open them and I read about a professor at the University of Kentucky Brent seals who had been trying to scan these using increasingly Advanced Imaging techniques and then use computer vision techniques and machine learning to kind of virtually unroll them without ever opening them and they tried a lot of different things but their most recent attempt in 2019 was to take the Scrolls to a particle accelerator in Oxford England called The Diamond flight source and to make essentially an incredibly high resolution CT scan of the sort of 3D X-Ray Scan and they needed really high energy photons in order to do this and they were able to take scans at eight microns so these these really quite tiny voxels which they thought would be sufficient and I thought this was like the coolest thing ever you know we're using technology to read this lost information from the past and I sort of waited for the news that they had been decoded successfully um so that was you know 2020 and then I think covet hit everybody got a little bit slowed down by that and last year I found myself wondering I wonder what happened to you know Dr seals and his scroll project and I reached out and it turned out they had been making really good progress you know they'd gotten some machine learning models to start to identify ink inside of the Scrolls but they hadn't yet extracted words or passages it's very challenging and I invite him to come out to California and hang out and to my shock he did and we got to talking and decided to team up and try to crack this thing and the approach that we've settled on to do that is to actually launch an open competition we're gonna we've done a ton of work with his team to get the data into a shape where and the tools and techniques and just the broad understanding of the materials into a shape where smart people can kind of approach it and get productive easily and then I'm putting up together with Daniel gross a prize it's sort of like an X Prize or something like that for the first person or team who can actually read like substantial amounts of real text from one of these Scrolls without opening them and so we're launching that this week you know I guess maybe it's when when this airs I don't know um the stakes are kind of big like this like what gets me excited are the stakes so the six or eight hundred Scrolls that are there it's estimated that if we could read all of them and you know like somehow the technique Works in a generalizes to all the Scrolls then that would give us approximately a doubling of the total tax that we have from Antiquity this is what historians and classicists tell me so it's not like oh we would get like a five percent bumper 10 bump in the total ancient Roman or Greek text it would be like no we get all of the text that we have again yeah multiple shakespeares is sort of one of the units that I've heard so so that that would be significant I mean we don't know what's in there you know we've got a few philodina's texts those are of some interest um but there could be lost epic poems or God knows what so I'm really excited and I think you know my bet is there's like a 50 chance that someone will encounter this opportunity and get the data and get nerd sniped by it and we'll solve it this year I mean really it is something out of a science fiction novel you know it's like something you'd read in Neil stevens or something um I was talking to uh Professor seals before and apparently it should shock went both ways because the first few emails he goes like this has got to be spam like no way not Friedman is reaching out and has found out about this prize and that's really funny because he was really pretty hard to get in touch with so like I I emailed them a couple times just like didn't respond and so I was like so I asked my admin Emily to call the Secretary of his department and say like Mr Friedman requested me and then like he knew there was something like actually going on there so he finally got on the phone with me and got on zoom and uh he's like why are you interested I mean I love Brent he's fantastic and uh I think you know we're like friends now and I think we found that we think alike about this and I think he's reached the point where he just really wants to crack you know they've taken this right up to the one yard line like uh this is doable at this point they've demonstrated I think every key component but putting it all together improving the quality doing it at the scale of a whole scroll this is still very hard work and an open competition seems like the most efficient way to get it done before we get into the state of the data and the different possible solutions um I want to make tangible like what could be gained if we can unwrap these so you said there's a few more thousand scrolls are we talking about the ones in uh Fila dimas's Lair are we talking about the ones and other layers you know you'd think if you find this crazy Villa that was owned by Julius Caesar's father-in-law that we just like dig the whole thing out but in fact most of the exploration occurred in the 1700s through the Swiss Engineers tunnels underground so it was never The Villa was never dug out and exposed to the air you went down 50 60 feet and then you dig tunnels and you know again they were looking for Treasure not like a full archaeological exploration so they mostly got treasure in the 90s some additional excavations were done kind of at the edge of the villa and they discovered a couple things first they discovered this like it was a Seaside bill that faced the ocean it was right on the water before the volcano erupted the eruption actually pushed the shoreline out by depositing so much additional mud there so it's no longer right by the ocean apparently I've actually never been and they also found that there were two additional floors in the Villa that the tunnels apparently had never excavated and so at most a third of Davila has been excavated now they also know when they were discovering these Papyrus Scrolls that they found basically one little room where most of the Scrolls were and these were mostly these philodemus texts at least that's what we know and they found apparently several revisions sometimes at the same text and so they think the hypothesis is this was actually philodemus's working Library he worked here this this sort of Epicurean philosopher and in the hallways though they occasionally found other Scrolls including crates of them and the belief is at least this is what historians have told me and I'm no expert but what they have told me is they think that the mean library in this Villa has probably not been excavated and that the mean Library may be a Latin library and may contain you know literary tax historical texts other things and that it could be much larger now I don't know how prone these classicists started wishful thinking it is a romantic idea but they have some evidence you know in the presence of these uh partly evacuated sort of Scrolls that were found in hallways and that sort of thing so there are descriptions you know I've since gone and read a bunch of the uh like first-hand accounts uh of of the excavations and there are these heartbreaking descriptions of them finding like an entire case of Scrolls in Latin and like accidentally destroying it as they tried to get it out of the mud and you know there were maybe 30 Scrolls or something in there so there clearly was some other stuff that we just haven't got to I mean you made some uh yeah so Papyrus is it Papyrus and um it's a reed it's a grassy read that grows on the Nile in Egypt and for thousands of years many thousands of years they've been making paper out of it and the way they do it is they take the kind of outer rind off of the Papyrus and then they cut the inner core into these strips when they lay the strips out kind of parallel to one another and then they put another layer to 90 degrees to that bottom layer and they press it together in a press or under stones and let it dry out and that's Papyrus essentially and then they'll take some of those sheets and kind of glue them together with paste made out of flour usually and get a long scroll and um you can still buy it I bought this on Amazon and it's interesting because it's got a lot of texture you know those fibrous Ridges of the Papyrus plant and you can see when you write on it you really feel the texture and um I got it because I wanted to understand sort of what are these artifacts that we're working with and so we made an attempt to simulate carbonizing a few of these so we basically took a dutch oven because when you carbonize something and you make charcoal it's not like burning it with oxygen you sort of remove the oxygen heat it up and let it carbonize so we tried to simulate that with the Dutch oven which is probably imperfect and left it in the oven at 500 degrees Fahrenheit for maybe I don't know how long these were but our biggest attempt was like five or six hours and they really and so these things are incredibly light and if you try to unfold them they just fall apart in your hand very readily I assume these are in somewhat better shape than the ones that were found because these were not in a volcanic eruption you know and like covered in mud I think the volcano is probably maybe that mud was hotter than my oven can go so and it just flakes you know just sort of you just squeeze it it's just it's just dust in your hand and so we actually tried to replicate many of the heartbreaking 1700s 18th century unrolling techniques like they used rose water for example or they tried to use different oils to soften it and unroll it and most of them are just very destructive they poured mercury into it because they thought Mercury would slip between the layers potentially so so yeah this is sort of what they look like they you know they shrink and they turn to Ash yeah for those listening by the way it kind of looks like a black I mean just imagine sort of the ash of a cigar but uh blacker and it crumbles the same way uh it's just a blistered black piece of rolled up Papyrus yeah and they blister the layers can separate they can fuse and so this happened in 79 A.D right so we know that anything before that could be in here which I guess could include yeah so what could be in there I don't know you know uh uh you and I have speculated about this right well I think you know it would be extremely exciting not to just get more Epicurean philosophy although that's fine too um but almost anything would be interesting and additive like dreams are um I think it would maybe have a big impact to find something about early Christianity like a contemporaneous mention of early Christianity maybe there would be something that you know the church wouldn't want that would be exciting uh to Me Maybe there'd be something you know some color detail from someone commenting on Christianity or Jesus that like I think that would be a very big deal we have not no such things as far as I know uh other things that would be cool would be old stuff like even older stuff so there were several Scrolls already found in there that they know were hundreds of years old when the Villa was buried so the Villa was probably constructed about 100 years prior is my understanding and they can tell from the style of writing they can date you know they can date some of these Scrolls and so there is some old stuff in there and the Library of Alexandria was burned 80 year 90 years prior and so again maybe wishful thinking but there's some rumors that some of those Scrolls were evacuated and maybe some of them would have ended up at this substantial prominent uh you know Mediterranean Villa God knows what would be in there that would be really cool I think it'd be great to find Literature Like personally I think that would be exciting like beautiful new poems or stories we just don't have a ton uh because so little survived and uh so I think I think that would be fun um I think you're you had the best uh crazy idea for what could be in there which was text which was GPT watermarked that would be a creepy feeling uh I still can't get over just how how much of a plot of a Sci-Fi novel this is like right like potentially the biggest uh intact library from the ancient world that has been sort of stopped like a debugger and because of this volcano and I mean just the philosophers and Antiquity forgotten like the earliest gospels there's so much interesting stuff there but um let's talk about what the data looks like so you mentioned that they've been um CT scanned and that they develop these machine learning techniques to do uh segmentation and the unrolling what would it take to get from there to an actual to understand the actual content of what is within Dr seals actually pioneered this field of what he calls and is now widely called virtual unwrapping and he did it actually not with these Herculaneum Scrolls these things are like expert mode they're so difficult I'll tell you why soon um but he did it with initially with a scroll that was found in the Dead Sea in Israel it's called The End Getty scroll and it was carbonized actually under like slightly similar circumstances I think there was a temple that was burned the Papyrus scroll was in a box so it kind of it's like a dutch oven it kind of carbonized in the same way and so it was not openable um it's fall apart and so the question was could you non-destructively read the the contents of it and so he did this 3D x-ray the CT scan of the scroll and then was able to do two things first the ink gave a great x-ray signature and so it looked very different from the Papyrus it was a high contrast and then second he was able to segment the wines of the scroll you know throughout the entire body of the of the scroll and identify each layer and then just geometrically unroll it using you know fairly normal flattening computer vision sort of techniques and then read the contents of it and it turned out to be I think an early part of the Book of Leviticus you know something of the Old Testament or the Torah and uh that you know that was like a Landmark achievement and so then the next idea was to apply those same techniques to to this case and so okay this this is proven hard I think there's a couple things that make it difficult one is that the primary one is that the ink used on the Herculaneum papyri it it has it is not very absorbent of X-ray like it basically seems to be equally absorbent of X-ray as the Papyrus or very close certainly not perfectly and so you don't have this nice bright lettering that shows up kind of on your tomographic 3D x-ray so you have to somehow develop new techniques for finding the ink in there so that's sort of problem one and it's been a major Challenge and then the second problem is the scrolls are just real messed up like they're long and tightly wound highly distorted by the you know volcanic mud which not only heated them but deformed you know partly deformed them and um and so just the segmentation problem of identifying each of these layers throughout the Scrolls Act is you know it's doable but it's it's hard um those are a couple of challenges and then the other challenge of course is just getting access to Scrolls and taking them to a particle accelerator so you have to like have scroll access and you know particle accelerator access and time on those it's expensive and difficult and and uh you know Dr seals did the hard work of making all that happen and um so the good news is very recently just in the last couple of months his lab has demonstrated with a convolutional neural network the ability to actually recognize ink inside these X-rays and you know you look I look at the X-ray scans and I cannot at least in any of the renderings that we've seen I can't see the ink but the machine learning model can pick up on sort of very subtle patterns in the X-ray absorption at high resolution inside these volumes in order to identify ink and we've seen that and so you might ask okay like how do you train a model to do that because you need some kind of ground truth data to train the model so the big Insight that they had was to train on broken off fragments of the Papyrus so as people tried to open these over the years you know in Italy they destroyed many of them but they saved some of the pieces they broke off and on some of those pieces you can kind of see lettering and if you take an infrared image of the fragment then you can really see the lettering pretty well in some cases and so they I think it's 930 nanometers they take this little infrared image now you've got some ground truth then you do a CT scan of that broken off fragment and you try to align it register it with the image and then you have data that you can use potentially to train a model and that turned out to work in the case of the fragments okay so now I think this is sort of the why now this is why I think launching this challenge now um is the right time because we have a lot of reasons to believe it can work like and the core techniques the core pieces have been demonstrated it just all has to be put together at the scale of these really complicated Scrolls and so so yeah I think if you can do the segmentation which is probably a lot of work maybe there's some way to automate it and then you can figure out how to apply these models you know inside the body of a scroll and not just to these fragments then it seems it seems like you could probably read lots of text why did you do it decide to do it in the form of a prize rather than just like giving a grant to the team that was already pursuing or maybe some other team that wants to take it up we talked about that um but I I think the what we basically concluded was the search space of different ways you could solve this is pretty big and we just wanted to get it done as quickly as possible yeah and so having a contest means lots of people are going to try lots of things and you know someone's going to figure it out quickly you know many eyes may make it shallow as a task and uh so I think that's the main thing like probably someone could do it but I think this will just be a lot more efficient and it's fun too I think this is fun like I think it's interesting to do a contest and you know who knows who will solve it or how people may they may not even use machine learning you know we think that's the most likely approach for recognizing the ink but they may find some other approach that we haven't thought of one question people might have is that you have these visible fragments mapped out do we expect them to correspond to the burned off or the Ashen carbonized squirrels that you can do machine learning on the ground truth of one can correspond to the other I think there's a very legitimate concern they're different like when you have a broken off fragment there's air above the ink so when you CT scan it you have kind of ink next to air inside of a wrap scroll the ink might be next to Papyrus right because it's pushing up against the next layer and your model has to you your model you know may not know what to do with that and so um yeah I think this is one of the challenges and sort of how you take these models the retrained on fragments and translate them into the slightly different environment but maybe there's parts of the scroll where there is air on the inside and we know that to be true you can sort of see that here and so I think it should at least partly work if in clever people can probably figure out how to make it completely work yeah so you said the odds are about 50 50. what makes you think that it can be done yeah I think it can be done because we recognized Inc on from a CT scan on the fragments and I I think everything else is probably geometry and computer vision the scans are very high resolution so they're eight microns eight micrometers and they're taken if you kind of stood a scroll on end like this they're taken in these slices through it right like this so it's like this in the z-axis from bottom to top there are these slices and the way they're represented on disk is each slice is a tiff file and for the full Scrolls each slice is like 100 100 something megabytes so they're quite high resolution and then if you stack for example 100 of these they're eight microns right so 100 of these is 0.8 millimeters so you know millimeter is pretty small so that you know so they're fairly we think the resolution is is good enough or at least right on the edge good enough that it should be possible there's there's sort of like seem to be six or eight pixels uh for voxels I guess per you know across an entire layer of papyrus that's probably enough and we've also seen with the machine learning models Dr seals has got some PhD students who have actually demonstrated this add eight microns so I think that the ink recognition will work I think the data's in it the data is clearly physically in the Scrolls right the ink was carbonized the Papyrus was carbonized but not as like a lot of data actually physically survived and then the question is did the data make it into the scans and I think that's very likely based on the results that I've seen so far we've seen so far and so I think it's just about a smart person solving this and or a smart group of people or just a dogged group of people who do a lot of manual work that could also you know be true or you may have to be smart and dogged but um and I think that's where most of my uncertainty is there's uh just like weather whether somebody does it yeah I mean if uh quota million dollars doesn't motivates you yeah I think money's good yep I mean there's a lot of money in machine learning these days that's right do we have enough data in the form of Scrolls that have been mapped out to be able to train a model if that's the best way to go I guess because one question somebody might have is listen if you already have this ground truth why why hasn't Dr Seal's team already been able to just train if they will I think if we just let them do it they'll get it solved it might take a little bit longer because you know it's not a huge number of people and there's probably there's a big search space here but I mean yeah if we didn't launch this contest I'd still think this would get solved right but it might take several years and I think this way it's very it's likely to happen this year okay and what happens let's say you know the price is solved somebody figures out how to do this and we can read the first scroll you mentioned that these other layers haven't even excavated how is how how is it how is the world going to react let's say we give up one of these Maps that's my like personal hope for this I always like to look for sort of these cheap leverage hacks you know these moments where you can do like a relatively small thing and it creates you know you kick a pebble and you get an avalanche right and the theory is and you know Brent shares this Theory the theory is that if you can read one scroll like just one scroll and we only have two scan Scrolls there's hundreds of surviving skulls it's relatively expensive to use to book a particle accelerator so if you can scan one scroll and you know it works you can generalize the technique out and it's going to work on these other Scrolls then the money which is probably low Millions maybe only one million dollars to scan the remaining Scrolls will just arrive like it's just it's too sweet of a prize not for that not to happen and the urgency and kind of return on Excavating the rest of the Villa will be incredibly obvious too because if there are thousands more Papyrus schools in there and we now have the techniques to read them then there's gold in that mud and you know it's got to be dug out and um you know it's amazing how little money there is for archeology it's you know literally for decades no one's been digging there so that's my hope is that like this this is the Catalyst that you know it works somebody reads it they get a lot of Glory we all get to feel great and then the diggers arrive in Hercules and they dig out the rest I I wonder if the budget for archaeological movies and games like Uncharted or Indiana Jones is bigger than the actual budget to do real world archeology but you know I was talking to some of the people before this interview and that's one thing they emphasized is your ability to find these leverage points for example with California EMB I don't know the exact amount you see did it with but um for that amount of money it is uh and for an institution that is that new it is one of the very few institutions that has had a significant amount of political influence right like if you look at the state of UMB in California and nationally today I guess how do you identify these things like how do you see I mean there's plenty of people who have money who get into history or get into whatever subject very few do something about it right like how do you figure out where you know I'm a little bit mystified by why people don't do more things too um like I think first of all I don't know maybe you can tell me why are more people doing things like I think most rich people are boring and they should do more cool things um so I'm hoping that they do that now um but yeah I mean I don't know like um I think part of it is I just fundamentally don't believe the world is efficient and so if I see an opportunity to do something I don't have it I used to but I no longer have a reflexive reaction that says oh that must not be a good idea if it were a good idea someone would already be doing it like someone must be taking care of housing policy in California right or somebody must be you know taking care of this or that and so like I think you know first I like I don't have that filter that says the world's efficient don't bother someone's probably got it covered and then the second thing is I kind of have learned to trust my enthusiasm you know it was this gets me in trouble too but if I get like really enthusiastic about something and that enthusiasm kind of persists I just indulged and just think oh yeah I'm gonna go like you know like I like doing the things I'm enthusiastic about and so I I just kind of let myself be impulsive and so frequently what you do uh there's this great you know image that I found and tweeted which said we do these things not because they are easy but because we thought they would be easy and so yeah like that's frequently what happens is like the commitment to do it is impulsive because and it's done that of enthusiasm and then you get into it and you're like oh my God this is like really much harder than we expected but then you're sort of committed and you're stuck and you're gonna have to get it done like I thought this project would be relatively straightforward we're just going to take the data and put it up and but of course everything is and and truly 99 of the work has already been done by Dr seals and his team at the University of Kentucky I am a kind of carpet bagger I've shown up at the end here you know to like try to do a new piece of it but the last mile is often the hardest well I mean it's it turned out to be fractal anyway like that you know just like all the little bits that you have to get right to do a thing and have it work and you know I hope we got all of them but so I think that's part of it is just like yeah not believing the world's efficient then just like allowing your enthusiasm to cause you to commit to something that turns out to be a lot of work and really hard and then you just are like stubborn and don't want to fail and so you keep at it I don't know I think that's it yeah I I don't know I I feel like the efficiency point do you think that's particularly true just of things like um California emu or this where there isn't a direct monitor incentive or no I mean look certainly parts of the world are more efficient than others right and uh you can assume equal levels of inefficiency everywhere but I'm I'm like constantly surprised by how even in areas you expect to be very efficient there are things that are sort of in plain sight that no and it's not that I see them and others don't it you know there's lots of stuff I I don't see too I was talking to some Traders at a hedge fund recently and I asked them I was trying to understand the role Secrets play in the success of a hedge fund and the reason I was interested in that is because I think the AI labs are going to enter a new similar Dynamic where their secrets are very valuable like if you have a 50 training efficiency Improvement in your training Runs cost a hundred million dollars that is a 50 million dollar secret that you have that you want to keep and hedge funds do that kind of thing routinely and so I asked uh some Traders had a very successful hedge fund uh if you had maybe your smartest trade or get on Twitch for 10 minutes once a month and on that twitch stream describe their 30-day old trading strategies right so not your current ones but the ones that are a month old what would that how would that affect your business after 12 months of doing that so 12 months 10 minutes a month 30 day look back it's two hours in a year and to my shock they told me 80 reduction in their profits like it would have a huge impact and then I asked okay so how long would the look back window have to be before it would have like a relatively small effect in your business and they said 10 years so like that I think is just quite strong evidence that the world's not perfectly efficient because you know these folks make billions of dollars using secrets that could be related in like an hour or something like that and yet others don't have them or their secrets wouldn't work and so I think there are different levels of efficiency in the world but on the whole our like default estimate of how efficient the world is is far too charitable on the particular Point by the way of AI Labs potentially starting Secrets I mean you have this sort of strange Norm of different people from different AI Labs not only being friends but like often living together right so it would be like Oppenheimer living with somebody working on the Russian atomic bomb or something like that do you think those Norms will persist once the value of the secrets is realized yeah I was just wondering about that some more today I mean it's it seems to be sort of slowing you know they seem to be trying to close the valves um but I think there's a lot of things working against them in this regard so one is again that the secrets are relatively simple two is that you coming off this academic Norm of publishing and really open like the entire culture is based on sort of sharing and Publishing you know three is as you said they all live in group houses some are in molecules you know there's just a lot of um intermixing and then it's all in California and California is not you know non-compete State we don't have non-competes and so they'd have to change the culture get everybody their own house and move to Connecticut and then you know maybe work you know I I think ml engineer salaries and compensation packages will probably be adjusted to try to you know address this because you don't want your secrets walking out the door there are Engineers you know at Igor babushkin for example um who has just I believe joined Twitter I think is that right Elon hired him to train I think that's public is that I think it is it will be now I mean Igor's really really great guy and Brilliant um but he also happens to have trained state-of-the-art models at deepmind and open Ai and so you know like that's the set of people who have that sort of you know I don't know whether that's a consideration or how big of an effect that is but it's the kind of thing that it would make sense to Value if you think there are sort of valuable secrets that have not yet proliferated so I think they're going to try to slow it down publishing is certainly slowed down dramatically already but I think there's just a long way to go before you're anywhere in like hedge fund or Manhattan Project territory and probably Secrets will still have a relatively short half-life as somebody who has been involved in open source your entire life are you happy that this is the way that EI has turned out or do you think that this is less than optimal well I don't know my opinion's been changing I'm I have increasing worries about kind of safety issues like um not not the hijacked version of safety but um some industrial accident type situations or misuse and so I do think there's there's some we're not in that world and I'm not particularly concerned about it in the in the short term but in the long term I do think there are worlds that we should be a little bit concerned about although I don't know what to do about um where yeah like bad things happen Mo the probability mass of my belief is though is that is probably better in the whole for more people to get to Tinker with and use these models at least in their current state and so for example when you know gay orgy gergenov this weekend did a 4-bit quantization of the Llama model and got it you know inferencing on a M1 or M2 I was very excited and I got that running and it's like fun to play with now I've got a model that you know it's like very good it's almost gpt3 quality runs my laptop and you know I've sort of grown up in this world with the tinkerers and open source folks and more access you have the more things you can try so I I I think I do find myself you know very attracted to that uh I I guess that is the scientist uh and the ideas part of what is being shared but there's also another part about the actual substance right so like the uranium and the sort of atom bomb analogy as I guess different sources of data realize how valuable their data is for trading newer models do you think that these things will become harder to scrape libgen archive are these going to become rate Limited in some way or what are you expecting there well first there's so much data on the internet I mean the two kind of Primitives that you need to build models are you need lots of data we we have that in the form of the internet we digitize the whole world into the internet and then you and then you have you need these gpus which we have because of video games you take like the internet and video game hardware and you smash them together and you get machine learning models and they're both Commodities and so I think the data I don't I don't think anyone in the open source world is really going to be data limited for a long time there's so much that's out there probably people who have like proprietary data sets that are readily scrapable have been shutting those down you know so get your scraping in now if you uh if you need to do it but um that's just on the margin I I still think there's there's quite a lot that's out there to work with so I think look there's going to be a ton this is the Europe proliferation there's a week of proliferation like we're going to see four or five major AI announcements this week you know new models new apis new platforms new tools from all the different vendors in a way this you know they're all looking forward my Herculaneum project is looking backwards I think it's extremely exciting and cool but it is sort of a funny contrast okay so that uh I guess before I delve deeper into AI I do want to talk about GitHub so I think we should start with you are at Microsoft and at some point you realize that GitHub is very valuable and worth acquiring how did you realize that and how did you convince Microsoft to purchase GitHub well so I had started a company called xamarin together with uh Miguel de casa and Joseph Hill and we had built kind of mobile tools and platforms and Microsoft acquired the company in 2016. and uh I was excited about that I thought it was great uh but to be honest I didn't actually expect or plan to spend you know more than a kind of a year or so there but when I got in there I got exposed to what saty was doing and just the quality of his leadership team I was really impressed and um actually I think saw him in the first week or so I was there and he asked me what do you think we should do it at Microsoft and I said well I think we should buy GitHub this was like my first week it was like eight March or April of 2016. okay and then he said um yeah it's a good idea we thought about it I'm not sure we can get away with it or something like that and then it was about a year later a little more than a year later yeah I wrote it I wrote him an email just a memo you know it sort of said like I think it's time to do this there was some noise that Google was sniffing around I think that may have been manufactured by the GitHub team but it was a good Catalyst because it was something I thought made a lot of sense for Microsoft to do anyway and so I wrote an email to Satya sort of a little memo saying you know hey I think we should buy GitHub here's why here's what we should do with it and the basic argument was developers are making it purchasing decisions now it used to be the sort of I.T thing you know and now developers are leading that purchase and it's you know this sort of major shift in how software products are are acquired and Microsoft really was an I.T company it was not a developer company in in the way most of its purchases were made but it was founded as a developer company right and so you know the Microsoft's first product was a programming language um yeah I said look the challenge that we have is there's an entire new generation of developers who have no Affinity with Microsoft and the largest collection of them is a GitHub and if we acquire this and we do a merely competent job of running it we can earn the right to be considered by these Developers for all the other products that we do and to my surprise Satya replied in like six or seven minutes and said I think this is very good thinking let's meet next week or so and talk about it and I ended up at this conference room with him and Amy hood and Scott Guthrie and Kevin Scott and several other people and uh they said okay make you know tell us what you're thinking and I kind of did a little 20-minute ramble on it and Satya said yeah I think we should do it and why don't we run it independently like LinkedIn that you'll be the CEO and he said do you think we can get it for 2 billion and I said which we could try and uh three weeks later you know he said okay go go do this Scott you know Scott will support you on this three weeks later we had like assigned term shooting and announced deal um and then it was an amazing experience for me I'd been there less than two years and you know Microsoft was made up of and run by a lot of people who've been there for many years and they trusted me with this really big project and made me feel really good you know to be trusted and empowered and I had grown up in the open source world and so for me to get an opportunity to run get up it's like I don't know getting appointed mayor of your hometown or something like that it felt cool um and I really wanted to do a good job for developers and so that's that's how it happened that's actually uh one of the things I want to ask you about because often when something succeeds we kind of think it was inevitable that it would succeed but at the time I remember I mean I it was like a while back but I remember that it was a huge amount of skepticism I would go unlike Hacker News and like the top thing would be the blog post about how Microsoft's going to mess up GitHub and I guess people are have those concerns have been alleviated throughout the years but how did you get you know deal with that skepticism and deal with that distrust well I was really paranoid about it yeah and I really cared about what developers thought I think there's always this question of who are you performing for like who do you actually really care about sort of who's in who's the audience that's in your head that you're trying to you know do a good job for impress or in the respect of whatever it is and though I love Microsoft and care a lot about Satya and the everyone there I really cared about the developers you know I got grown up in this open source world and so for me to do a bad job with this Central institution and open source would have been a devastating feeling for me it was very important to me not to so that was sort of first thing is just that I cared and the second thing is that the deal leaked I was going to be announced I think in a Monday leaked on a Friday and uh Microsoft's buying GitHub and the whole weekend there were like terrible posts online you know people saying we got to evacuate GitHub as quickly as possible and and uh we're like oh my God it's terrible and then Monday we put the announcement out and we said we're acquiring GitHub it's going to run as an independent company and then I said Nat you know Nat Friedman's gonna be CEO and you know I I had I don't want to overstate or whatever like but I think a couple people were like oh now it comes from open source you know he spent some time in open source so and it's going to be run independently so I don't think they were really that that calm down but at least a few people thought like well maybe I'll give this a few months and see what happens before I migrate off and then my first day as CEO after we got the deal closed like 9 A.M the first day uh you know I was in this room and we got on zoom and all the heads of engineering and product and I think maybe I don't know what people were expecting but I think maybe they were expecting some kind of longer term strategy or something but I came in and I said there was this GitHub had no official feedback mechanism that was publicly available but there was several GitHub repos the community members had started Isaac from npm and started Juan uh where he'd just been allowing people to give GitHub feedback and people had been voting on this stuff for years and I kind of shared my screen and put that up sorted by votes and said like we're gonna pick one thing from this list and fix it by the end of the day and ship that like just one thing and you know I think we were like like this is the new CEO strategy like yeah you know and they were like I don't know we can't you know do database migrations that can't do that in a day and like and then someone's like well maybe we can do this you know we had to sort of we actually have a half implementation of this and we eventually found something that we could fix by the end of the day and what I'm thinking is what I'm thinking what I hope I said was well we need to show the world is that GitHub cares about Developers not that it cares about Microsoft like if the first thing we did after the acquisition was to add Skype integration developers would have said oh we're not your priority like you have new priorities now and so the idea was just to find ways to make it better for the people who use it and have them see that we cared about that immediately and so I said we're going to do this today and then we're going to do it every day for the next hundred days and it was cool because I think it created some really good feedback loops at least for me one was you know you ship things and then people are like oh hey I've been wanting to see this fixed for years and now it's fixed it's a relatively simple thing so you get this sort of nice dopaminergic you know feedback loop going there and then people in the team feel the you know excitement of shipping stuff um I think GitHub was a company that had a little bit of stage fright about shipping previously and sort of break that static friction and ship a little bit more I think felt good and then the other one is just the learning Loop by trying to do lots of small things I got exposed to like okay this team is really good you know or this part of the code has a lot of tech debt or hey we shipped that and it was actually kind of bad how come that design got out like where you know and so you whereas if the project had been some six month thing I'm not sure my learning would have been quite as quick about the company and there's still things I missed the mistakes I made for sure but that was part of how I think you know you know no one knows counter factually what whether that made a big difference or not but I do think that earned some trust I mean most Acquisitions don't go well not only do they not go as well but like they don't go well at all right like as we're seeing in in the last few months uh with a certain one well why do most Acquisitions fail or fail to go well yeah it is true most Acquisitions are destructive of value what is the value of a company um in an Innovative industry the value of the company a lot of it boils down to its ability culturally to produce new Innovations and is some sensitive harmonic of cultural elements that sets that up that makes that possible and it's quite fragile I think and so if you take a culture that has achieved some productive harmonic and you put it inside of another culture that's really different the kind of mismatch of that can destroy the productivity of of the company so I think that maybe one way to think about it is companies are a little bit fragile and yeah and so when you acquire them it's like relatively yeah relatively easy to break them I mean they're also more durable than people think in many cases too like I would say another another version of it is the people who really care leave and so like the people who really care about building great products and serving the customers maybe they don't want to work for the acquirer and the set of people that are really load-bearing around kind of the long-term success is small and when they leave or get disempowered you get very different behaviors and then so I want to go into the story of co-pilot because until chapter I guess it was like the most widely used application of the modern AI models whatever part of the story you're willing to share in public yeah I mean um I've talked about this a little bit I mean so uh look I uh gpt3 came out in May I think of 2020 and I saw it and it really blew my mind uh I thought it was amazing and I was CEO of GitHub at that time and I um I thought like I don't know what but we've got to build some product with this this is you know we've got to build something so Satya had at I think Kevin Scott's urging already invested in open AI like a year before gpt3 came out like this is like quite amazing and he invested like a billion dollars by the way do you know why he knew that open AI would be worth investing at that point I don't know actually I've never asked him but yeah I'm not sure that's a good question I mean I think opening I had already had some successes that were noticeable and I think with your Satya and you're running this multi-trillion dollar company you're trying to execute well and serve your customers but you're always looking for the next gigantic wave that is going to upend the technology industry it's not just about trying to win Cloud it's like okay what comes after cloud and so you want you have to make some big bets and I think he thought AI could be one um and I think Kevin Scott deserves a lot of credit for really advocating for that aggressively and I think Sam Altman did a good job of building that partnership because he knew that he needed access to the resources of a company like Microsoft to build you know large-scale Ai and eventually AGI and so I think it was some combination of those three people kind of coming together to make it happen but I still think it was very pressy and bad you know people I've said that to people and they've said well a billion dollar is not a lot for Microsoft yeah but like there were a lot of other companies that could have spent a billion dollars to do that and did not and so I still think like that deserves a lot of credit okay so GB3 comes out uh you know I pinged Sam and Greg I think Brockman at open Ai and um they were like yeah like let's we've already been experimenting with gpt3 and derivative models encoding context like let's definitely work on something and to me at least and a few other people it was not incredibly obvious what the product would be um now I think it's trivially obvious you know autocomplete my gosh isn't that what the models do but at the time actually my first thought was that it was probably going to be like a q a chat bot stack Overflow type of thing and so that was actually the first thing we prototyped um so we grabbed a couple of Engineers um uh this guy oga who had come in from a kind of acquisition that we'd done and Alex Gravely and uh started prototyping and the first prototype was a chat bot and you know what we discovered first was that demos were fabulous like every AI product has a fantastic demo you get the sort of wow moment so like that is turns out to be maybe not a sufficient condition for a product to be good because it was just at the time the models were just not reliable enough they were not good enough you know I asked you a question 25 of the time you give me an incredible answer that I love 75 of the time your answer is useless or wrong it's not a great product experience and so then we started thinking about code synthesis and our first attempts at this were actually large chunks of code synthesis like synthesizing whole function bodies and we built some tools to do that and put them in the editor and that also was not really that satisfying and so the next thing that we tried was to just do simple small scale autocomplete with the large models and we used the kind of intellisense drop down UI to do that and that was better like definitely pretty good but the UI was not quite right and we lost the ability to do this large-scale synthesis you know we still have that the UI for that wasn't good and we had it I think so that you to get a function body synthesized you would hit a key and then I don't know why this was the idea everyone had at the time but several people had this idea that it should display um multiple options for the function body and then the user would read them and pick the right one and I think the idea was that we would use that human feedback to improve the model but that turned out to be a bad experience because first you had to hit a key and explicitly request it then you had to wait for it and then you had to read you know three different versions of a block of code reading one version of a block of code takes some cognitive effort doing it three times takes more cognitive effort and then most often the result of that was like you none of them were good or you didn't know which you know like which one to pick and so that was also like you're putting a lot of energy and you're not getting a lot out it's sort of frustrating so once we had that sort of single line completion working I think Alex had the idea of saying we can use the cursor position in the AST to figure out heuristically whether you're at the beginning of a block in the code or not and if it's not the beginning of a block just complete a line if it's the beginning of a block show inline a full you know block completion so the sort of number of tokens you request and when you stop gets gets altered automatically with noise or interaction and then the idea of using this sort of gray text like Gmail had done in the editor and so we got that implemented and it was really only kind of once all those pieces came together and we started using a model that was small enough to be low latency but big enough to be accurate that we reached the point where like the median new user loved copilot and wouldn't stop using it and that took four months five months of just tinkering and sort of exploring you know there were other dead ends that we had along the way and um and then yeah I think that then it became quite obvious that it was good because we had hundreds of internal users who were GitHub engineers and I remember the first time I looked at the retention numbers they were extremely high it was like I remember 660 plus percent after 30 days from first install like if you installed it the chance that you were still using it after 30 days like over 60 percent In This Very intrusive product I mean it's sort of always popping UI up and so if you don't like it you will disable it um for if indeed 40 something percent of people did disable it but those are very high retention numbers for like an alpha first version you know of a product that you're using all day so then I was you know just incredibly excited to launch it and now it's you know it's improved dramatically since then yeah sounds very similar to the Gmail story right of it's a incredibly valuable insight and then it was obvious that it needs to go outside okay we'll go back to the ESF in a second but uh some some more you did have questions by what point will if ever will GitHub profiles replace resumes for programmers that's a good question I mean I think they're a contributing element to how people like try to understand a person now but I don't think they're like a definitive resume you know we introduced readme's on profiles when I was there and I was excited about that because I thought it gave people like some degree of personalization I think some people have you know I mean many thousands of people have have done that um yeah I don't know there's forces to push in the other direction too on that one where people like don't want their activity and skills to be as legible and there may be some adverse selection as well where the people with the most elite skills you know it's rather gauche for them to Signal their competence on their profile so there's some weird like social dynamics that feed into it too but I will say I think it effectively has this role for people who are breaking through today like one of the best ways to break through I know many people are in this situation you were born in Argentina you um you're a very sharp person but you didn't grow up in like a highly connected or or prosperous Network family Etc and yet you know you're really capable and you just want to get connected to kind of the most elite part communities in the world and so if you're good at programming you can join open source communities and contribute to them and you can very quickly accrete a global reputation for your talent which is legible to many companies and individuals around the world and suddenly you find yourself getting a job and moving maybe to the US or maybe not moving or you end up at a great startup I mean I know a lot of people who've like deliberately pursued the strategy of you know building reputation in open source and then kind of you've got the sale up and the wind catches you and you know you're you've got a career um and so I think it plays that role in that sense but in other communities like in machine learning research this is not how you you know there's there's a thousand people the reputation is more on archive you know than it is on GitHub so I don't know that it'll ever be comprehensive are there any other Industries for which proof of work of this kind will eat more into the way in which people are hired well I think there's a labor market dynamic in software where the really high quality Talent is so in demand and the supply is so much less than the demand that it shifts power onto the developers such that they can require of their employers that they be allowed to work in public and uh because and then when they do that they develop an external reputation which is this asset they can Port between companies and if the labor market dynamics weren't like that if programming well were less economically valuable then they would not labor you know the the like that they would not have companies wouldn't let them do that they wouldn't let them like publish a bunch of stuff publicly they'd say like we're not that's a rule and that used to be the case in fact and so as software's become more valuable Developers um the the leverage of like a single super talented developer has gone up and they've been able to demand over the last several decades the ability to work in public and um I think that's not going away other than that what is it I mean we talked about this a little bit but what has been the impact of developers being more empowered in organizations even ones that are not traditionally I.T organizations yeah I mean software is is it's kind of magic right I mean the you can write a for Loop and do something a lot of times and like you know like when you build large organizations at scale one of the things that does surprise you is the degree to which you need to systematize the behavior of the people who are working like when I first was starting companies and Building Sales teams I had this wrong idea coming from the world as a programmer that sales people were like hyper aggressive hyper entrepreneurial you know making Promises to the customer that the product wouldn't do and that the main challenge you had with sales people was like restraining them from going out and like you know aggressively cutting deals that shouldn't be cut and what I discovered is that does exist sometimes but like the much more common case is that you need to build a systematic sales Playbook which is almost a script that you run on your sales team where your sales reps know the process they need to follow to like exercise this repeatable sales motion and get a deal closed and so you know I just had bad ideas there I didn't know that that was how the world worked but software is a way to like systematize and scale out a valuable process extremely efficiently and I think the more digitized the world has become the more valuable software becomes and the more valuable become the developers who can create it essentially with 25 year old NAD be surprised with how well open source worked and how pervasive it is yeah I think that's true yeah I I um I think we all have this image When We're Young that these institutions are these implacable edifices that are evil and all-powerful and you know are able to like substant with Master plans substantially orchestrate the world and that is some sometimes a little bit true but like they're very vulnerable to these um yeah like new ideas and new forces and new Communications media and stuff like that so right now I think like maybe I wouldn't right now I think our institutions overall look relatively weak and certainly they're weaker than I thought they were back then so I thought Microsoft honestly I thought Microsoft could stop open source I thought that was a possibility you know they can do some patent move and kind of there's a master plan to ring fence open source in and um yeah that didn't that didn't end up being the case in fact Microsoft when we bought GitHub we um we pledged all of our patent portfolio to open source that was one of the things that we did as part of it and so that was a kind of poetic moment for me having been on the other side of patent discussions in the past to be a part of an instrumental in in Microsoft making that pledge like that was that was quite crazy oh that's really interesting it wasn't that there was like some business strategic reason more so it was just like an idea whose time had come well um GitHub had made such a pledge and so I think in part in acquiring GitHub we had to either try to annul that pledge or sign up to it ourselves and so there was sort of a moment of a forced choice but you know everyone at Microsoft thought it was a good idea too I so I think in many senses it was a moment whose time had come and the kind of GitHub acquisition was a forcing function what what do you make of um critics of modern open source like uh Richard stallman or people who look advocate for free software saying that um well corporations might advocate for open source because of like practical reasons for getting good code the real value of software um and the real way this software should be made it should be free in that you can replicate it you can um you can change it you can modify it and you can completely view it and that the ethical value is about that should be more important than the Practical values like what do you make of that critique of yeah I think those are the things that he wants right um and I think the thing that maybe he hasn't updated is that maybe not everyone else wants that um you know he has this idea that people want freedom from the tyranny of a proprietary intellectual property license but what people really want is freedom from having to configure their graphics card or sound driver or something like that you know they want their computer to kind of work right there are places where freedom is really valuable but there's there's always this thing of like I have a prescriptive ideology that I'd like to impose on the world versus this thing of like I will try to develop the best observational model for what people actually want whether I want them to want it or not and I think you know Richard is is strongly in the former Camp what is the most underrated license by the way I mean I don't know I maybe the MIT license is still underrated because it's just so simple and bare um uh Nadia Iqbal had a book recently where she argued that the key constraint on open source software and on the uh and on the time of the people who maintain it is the community aspect of software they had to deal with feature requests and discussions and you know maintaining for different platforms and things like that and it wasn't the actual code itself but rather this sort of extracurricular aspect that was the main constraint do you think that is the constraint for open source software like how do you see what is what is holding back more open sort of software yeah I mean I think by and large I would say that there is not a problem meaning open source software continues to be developed continues to be broadly used and you know there's areas where it works better in areas where it works less well but it's sort of winning in in all the areas where like um large-scale coordination and editorial control are not necessary and so it tends to be great at infrastructure Standalone components and very very horizontal things like operating systems and it tends to be worse at user experiences and like things that where you need a sort of dictatorial aesthetic or an editorial control I've had new debates with Dylan field figma as to why it is that we don't have lots of good open source applications and you know I've always thought it had something to do with this governance Dynamic of you know gosh it's such a pain to coordinate with tons of people who all sort of feel like they have a right to try to push the project in one way or another whereas in a hierarchical corporation there can be a you know head of this product or a CEO or founder or designer who just says we're doing it this way and you can really align things in One Direction very very easily um Dylan has argued to me that it might be because there's just fewer designers you know people with good design sense and open source I think that might be a contributing factor too but I I think it's still mostly the governance thing and I think that's what Nadia is pointing at all so you're running a project people you gave it to people for free for some reason giving people something for free creates the sense of entitlement and then they feel like they have the right to demand your time and push push things around and give you input and you want to be polite and it's very draining so so I think that you know where that coordination burden is lower is where open source tends to succeed more and probably software and other new forms of governance can improve that and like expand the territory that's open source can succeed in yeah I mean theoretically those two things are um consistent right like you could have very tight control over governance while the code itself is yeah open source and this happens in programming languages right languages can't be designed I mean they often are eventually sort of uh set in stone and then like Advanced by committee yeah but yeah I mean certainly you have these sort of benign dictators of languages who enforce the strongest strong set of ideas they have a sort of vision master plan um so that would be the argument that's most kind of on Dylan's side it's like hey it works for languages why can't it work for end user applications um I think the thing you need to do though to build a good end user application is not only have a good aesthetic and idea but somehow establish a tight feedback loop with a set of users where you can give it tour cash try this oh my gosh okay that's not what you need okay and so like doing that is so hard even in a company where you have total hierarchical control of the team in theory um and everyone really wants the same thing and everyone's salary and stock options depend on the product being accepted by these users it still fails you know many times in that scenario then additionally doing that in the context of Open Source I think it's just like a slightly too hard the reason you uh acquired GitHub as you said is that there seemed to be sort of complementarity between Microsoft and github's missions and I guess that's been proven out over the last few years should there be more of these uh collaborations and Acquisitions should there be more Tech conglomerates like would that be good for the system I don't know if it's good but I think um there are yes it it is certainly efficient in many ways I think we are seeing a collaboration occur because the math is sort of pretty simple like if you are a large company and you have a lot of customers then the thing that you've achieved is this very expensive and difficult thing of building distribution and relationships with lots of customers and that is as hard or harder and takes longer and more money than just inventing the product in the first place and so if you can then go and just buy the product for a small amount of money and like make it available to all of your customers then there's often like an immediate really obvious you know gain from from doing that and so in that sense like acquisition Acquisitions make a ton of sense and I've been surprised that the large companies haven't done many more Acquisitions in the past until I got into a big company and started trying to do Acquisitions and I saw that there are like strong elements of the internal dynamics that make it hard um you know there's not even you know it's like easier to spend 100 million dollars on employees internally to do a project and to spend 100 million dollars to buy a company the the dollars are treated differently the approval processes are different the kind of cultural buy-in processes are different and then to the point of the discussion we had earlier many Acquisitions do fail and when an acquisition fails it's somehow louder and more embarrassing than when like some new product effort you've spun up doesn't quite work out as well and so I think there's lots of internal reasons some somewhat Justified and some less so that they haven't been doing it but just from an economic point of view it seemed like it made makes sense to see more Acquisitions than we've seen uh well why did you leave I think as much as I loved Microsoft and certainly as much as I love GitHub I really truly like I still feel tremendous love for GitHub and everything that it means to the people who use it and I didn't really want to be a part of like a giant company anymore and you know I think building co-pilot was an example of this you know it wouldn't have been possible without openai and Microsoft and GitHub the building had also required navigating this like really large group of people and between Microsoft and nobody and GitHub and and uh you you reach a point where you're spending a ton of time you know on just just navigating and coordinating lots of people and I just find that less energizing you know back to enthusiasm like just my enthusiasm for that was was not as high and uh it was I was torn about it because I truly love GitHub the product and there was so much more I still you know knew we could do but I was proud of what we'd done and I missed the team you know and I miss uh I miss working on GitHub it was really an honor for me but yeah it was time for me to go do something you know I was always a startup guy I always like small teams and I wanted to go back to sort of the smaller more Nimble environment okay so we'll get to it in a second but first I want to ask about nat.org and the 300 words oh yeah uh the list there which is I think like one of the most interesting um uh sort of like and I guess very strassian uh list I've seen the list of 300 words I've seen anywhere but um I'm just gonna like mention some of these and get some of your commentary you should probably work on raising the ceiling not the floor yeah why yeah I mean um well first I I say probably but what does it mean to raise the ceiling or the floor I mean I I just observed a lot of projects that set out to raise the floor meaning gosh we are fine but they are not and we need to go help them with our Superior prosperity and understanding of their situation and many of those projects fail so for example there were a lot of attempts to bring internet to Africa by large and Wealthy tech companies in American universities and I won't say they all had no effect that's not true but many of them were far short of successful like there were satellites there were balloons there were you know high altitude drones there were mesh Network laptops that were pursued by all these companies and by the way by perfectly well-meaning incredibly talented people who I think did in some cases see some success but overall probably much less than they ever hoped but if you go to Africa there is internet now and the way internet got there is the technologies that we developed to raise the ceiling in the richest part of the world which were cell phones and cell towers I mean in the movie Wall Street from the 80s you know he's got that gigantic brick cell phone that thing cost like 10 grand at the time that was a ceiling raising technology it eventually uh went down the learning curve and became cheap and the cell towers and cell phones eventually you know we've got now hundreds of millions or billions of them in Africa and it was sort of it was that initially ceiling raising technology and then the sort of force of of capitalism that that made it work in the end it was not any deus ex machina technology solution that it was intended to kind of raise the floor and so I think there's something about that that's not just an incidental example um but I say on my website I say probably because there are there are some examples where I think people set out to kind of raise the floor and say no one should ever die of smallpox again right no one should ever die of guinea worm again and they succeed and I wouldn't want to discourage that from happening but I think on balance we have too many attempts to do that that look good feel good sound good and don't matter in some cases have the opposite of the effect they intend to here's another one um and this is uh under the image section in many cases it's more accurate to model the world as 500 people than 8 billion now here's my question what are the 808 billion minus 500 people doing like why why are there only 500 people yeah I mean um I don't know exactly it's it's a good question I ask people that a lot I mean when the more I've sort of done in life The more I've been mystified by this like oh somebody must be doing acts and then you kind of you hear there's a few people doing acts and then you look into it they're not actually doing X they're doing kind of some version of it that's not that and so all the kind of best you know best moments in life occur when you find something that to you is totally obvious that clearly somebody must be doing but no one is doing I mean Mark Zuckerberg says this about founding Facebook like surely the big companies will eventually do this and create this social and identity layer on the internet you know Microsoft will do this but no none of them were and and he did it so what are they doing okay so I think the first thing is many people throughout the world are optimizing local conditions so they're working in their Town their Community they're doing something there and so the set of people that are kind of thinking about kind of global conditions is just naturally narrowed by the structure of the economy that's number one I think number two is most people really are quite mimetic um and I think we all are including me you know we we get a lot of ideas from other people and so you know our ideas are not our own um we kind of got them from somebody else it's kind of copy paste and so you have to work really hard not to do that and to be de-correlated and I think this is even more true today because of the internet you know I don't know if Albert Einstein as a patent Clerk what you know wouldn't he have just been on Twitter uh just getting the same ideas as everybody else like would he have his Decor related ideas so I think the internet's correlated us more the exception would be really disagreeable people who are just naturally disagreeable and so I think the the future belongs to the autists in some sense because you know they don't care what other people think as much um those of those of those of us on the Spectrum in any sense I think are in that category then yeah I think you know I think and then we have this belief that the world's efficient and it isn't and I think that's part of it so the other thing is that the world is so fractal and so interesting I mean Herculaneum papyri right like is this is this corner of the world that I find totally fascinating but I don't have any anticipation that eight billion people should be thinking about that you know whether that should be a priority for everyone okay here's another one large-scale engineering projects are more soluble in IQ than they appear and here's my question does that make you think that the impact of AI tools like copilot will be bigger or smaller because of engineer because one way to look at copilot is like actually it's like he was probably less than the average engineer so maybe it'll have like less impact right yeah but I think it increases the productivity of like it definitely increases the productivity average engineer to bring them you know higher up and I think it increases the productivity of of the best Engineers as well um certainly a lot of the people I consider to be the best Engineers tell me that they find it increases their productivity a lot so yeah I think AI is going to completely change like it's really interesting how so much of what's happened in AI has been sort of soft fictional work you know you have mid-journey you have copywriting you have you know gosh Claude from anthropica so literary it writes poetry so well except for copilot which is this real hard area where like the code has to compile uh you know has to be syntactically correct it has to work and pass the tests and you know we see the steady Improvement curve we're now already on average more than half of the code is written by copilot I think when it shipped it was like low 20s and so it's really improved a lot as the models have gotten better and the prompting has gotten better but uh I don't see any reason why that won't be like 95 percent um like it seems very likely to me and so I think I don't know what that world looks like it seems like we might have more special purpose and less general purpose software like right now we use general purpose tools like spreadsheets and and things like this a lot but part of that has to do with the cost of creating software and so once you have you know much cheaper software do you create more special purpose software that's a possibility um every company just a custom piece of code in a sense like maybe that's the kind of future we're headed towards so yeah I think we're going to see like enormous amounts of change in software development uh another one the cultural prohibition on micromanagement is harmful great individuals should be fully empowered to exercise their judgment and um the rebuttal to this is like you know if you micromanage or preventing people from learning and to develop their own judgment yeah so imagine you go into some company yeah they hired your cash and you do a great job with the first project that they give you and so you're like everyone's really impressed man dorcash he made the right decisions he worked really hard he figured out exactly what needed to be done and he did it extremely well and so over time you get promoted into positions of Greater Authority and the reason the company is doing this is they want you to do that again but at bigger scale right do it again but 10 times bigger the whole product instead of part of the product or 10 products instead of one and so the company is telling you you have great judgment and we want you to exercise that at a greater scale meanwhile the culture is telling you as you get promoted you should suspend your judgment more and more and defer your judgment to your team and so there's some equilibrium there and I think we're just out of equilibrium right now where the cultural prohibition is too strong and you know I I think maybe in the I don't know if this is or not but maybe in the 80s I would have felt the other side of this that like we have too much micromanagement I think the other problem that people have is that they don't like micromanagement because they don't want bad managers to micromanage right so you have some bad managers they have no expertise in the area they're just kind of people managers and they're starting to micromanage something they don't understand where their judgment is bad and my answer to that is like stop empowering bad managers like don't have them just don't have bad managers promote and Empower people who have great judgment and do understand the subject matter that they're working on you know if I work for you and I just know you have better judgment and you come in and you say that like you're launching the scroll thing and I think you've got the final format wrong you know here's how you should do it I would welcome that even though it's micromanagement because it's going to make us more successful and I'm going to learn something from that I know your judgment is better than mine in this case or at least we're gonna have a conversation about it we're both going to get smarter so I think on balance yeah there are cases where people have excellent judgment and and we should encourage them to exercise it and sometimes you know things will go wrong when you do that but on balance you will get far more Excellence out of it and uh we should yeah we should Empower individuals who have great judgment yeah there's a quote about Napoleon that if he could have been in every single theater of every single battle he was part of that he would have never lost to battle um I was talking to somebody who worked with you at GitHub and she emphasized to me and it's like really remarkable to me that even the applications already being shipped out to Engineers how much of the actual suggestions and the actual design came from you directly which was kind of remarkable to me that SEO you would uh yeah you can probably find people you can talk to who think that was terrible but the question is always does that scale right and and the answer is it does not scale it doesn't like but the set of people who really do have great judgment um like the experience that I had as CEOs I was terrified all the time that there was someone in the company who really knew exactly what to do and had excellent judgment but because of cultural forces that person wasn't empowered right that person was not allowed to exercise their judgment and make decisions and so when I would think and talk about this that was the fear that it was coming from is is uh you know they were in some consensus environment where their good ideas were getting whittled down by lots of conversations with other people and a politeness and a desire not to micromanage and so we were ending up with some kind of average thing and I would rather kind of have more High variance outcomes where you either get something that's excellent because it is the you know expressed vision of a really good tour or you get a disaster and it didn't work and so now you know it didn't work and you can start over like I would rather have those more High variance outcomes and I think it's worth it's a worthy trade okay let's talk about AI yeah what percentage of the economy is basically text to text yeah I mean it's a good question we've done the sort of Bureau of Labor Statistics analysis of this um and uh yeah it's not you know the majority of the economy or anything like that we're in the low double digit percentages um the thing that I think is hard to predict is what happens over time is the kind of cost of text to text goes down and um yeah I don't know I don't know what that's going to do but uh yeah there's like plenty there's plenty of Revenue to be got now I mean one way you can think about it is okay we have all these benchmarks for machine learning models um and you know there's lombata and there's this and there's that and those are really only useful and only exist because we haven't deployed the models really at scale and so we don't have a sense of what they're actually good at the best metric would probably be something like what percentage of economic tasks can they do or like on a gig Marketplace like upwork for example like what fraction of upwork jobs can gpt4 do I think it's sort of interesting question my guess is like extremely low right now autonomously but over time it will grow and then the question is what does that do for upwork I mean it's I don't know it's probably a five guessing a five billion dollar gmv market Marketplace something like that does it grow does it become 15 billion or 50 billion um does it shrink because the cost of text-to-text tasks goes down um I don't know my bet would be that we find more and more ways to use text to text you know to sort of Advance Advance uh progress and so overall there's a lot more demand for it um so yeah I guess we'll see I know what point does it happen so I mean like gpd3 has been a sort of rounding error in terms of like overall economic impact does does it happen with gpd4 or gpd5 where we see billions of dollars of usage you know I've got Early Access to gpd4 and I've gotten to use it a lot and I honestly can't tell you the answer to that because it's so hard to discover what these things can do that the prior ones couldn't do I just was talking to someone last night who told me oh gbd4 is actually really good at Korean and Japanese and like GB3 is much worse at those and so it's actually real step change you know for those languages and um yeah I think people didn't know how good gpt3 was until it got instruction tuned for chat CPT and was put out in that format and so I think there's kind of you can imagine the pre-trained models is kind of unrefined crude oil and then once they've been kind of rhf'd and trained and then put out into the world people can they can find the value well part of the EI narrative is wrong in the over optimistic Direction the probably over optimistic case from both the people who are fearful of what will happen and from people who are expecting great economic benefits is that we're definitely in this realm of diminishing returns from scale so for example I think gpd4 is my guess is two orders of magnitude more expensive to train the gbd3 but clearly not two orders magnitude more capable um now is it too or is magnitude more economically valuable that would also surprise me and so I think it's possible when you're in these sigmoids where you kind of are going you know up this exponential and then you start to asymptote it it can be difficult to tell if that's going to happen so I think yeah the idea that we might not run into hard problems or that scaling will continue to like be worth it on a dollar's basis I think or reasons to reasons to be a little bit more pessimistic than the people of High certainty of I don't know GDP increasing by 50 per month or something like that which I think some people are predicting but on the whole I'm very optimistic so you're asking me to like make the bear case for something I'm very bullish about all right no that's why I asked her to make the bear case because I know oh yeah um I want to ask you about these Foundation models what is the um stable equilibrium you think of how many of them will there be like will it be a sort of oligopoly like uber and Lyft where there will probably be wide scale proliferation um and they sort of asked me what are the structural forces that are proproliferation the structural forces that are pro-concentration so I think the the pro proliferation case is a bit stronger so the pro proliferation case is they're actually not that hard to train you can kind of the best practices will promulgate you can kind of write them down on a couple sheets of paper and to the extent that secrets are developed that improve training those are relatively simple and they get copied around you know easily number one number two the data is mostly public it's mostly kind of data from the internet number three the hardware is mostly commodity and the hardware is improving quickly um and getting much more efficient um and then I think there's a lot of techniques that kind of overcome you know like I think some of these Labs potentially have 50 100 200 percent training efficiency Improvement techniques and so there's just a lot of low-hanging fruit on the technique side of things and so we're seeing it happen I mean it's happening this week and it's happening this year is that we're getting like a lot of proliferation the only case against proliferation is that you'll get concentration because of training costs and I don't know I don't know that that's true I you know I just like I don't I don't have confidence that trillion dollar model will be much more valuable than the 100 billion dollar model and that even it will be necessary to spend a trillion dollars training it like maybe there will be so many techniques available for improving efficiency that like how much are you willing to spend on researchers to find techniques if you're willing to spend a trillion on training right like that's a lot of bounties for new techniques and like some smart people are going to take those bounces how different will these models be will it just be sort of everybody chasing the same exact marginal Improvement leading the same marginal capabilities or well they have entirely different repertoires of skills and abilities right now back to the nomadic point they're all pretty similar right I would say I mean basically the same rough techniques right what's happened is an alien substance is sort of landed on Earth and we are trying to figure out what we can build with it and you know I think we're in this multiple overhangs here we have sort of a compute overhang where there's much more compute in the world than is currently being used to train models like much much more you know I think the biggest models are trained on maybe 10-ish thousand gpus but there's millions of gpus you know and so okay there's the compute overhang and then we have I think a capability and technique overhang where there's lots of good ideas that are coming out and we haven't figured out how best to assemble them all together but that's just a matter of time kind of until people do that and then those capabilities haven't reached because many of them are in the hands of the labs they haven't reached the Tinkers of the world and I think that is where the new like what can this thing actually do like what you know like until you get your hands on it you don't really know I think openai were themselves surprised by how explosively Chad gbt has grown I don't think they put chat DBT I would expecting that to be the big announcement I thought I think they thought gpt4 was going to be their big announcement and I think it still probably is and will be big but the tattoo gbt really surprised them and I think that's that's you know it's hard to predict what people will do with it and what they'll find valuable and what works and so you need Tinker so it basically goes from like Hardware to researchers to tinkerers to products that that's the pipe that's the Cascade yeah yeah when I was scheduling my interview with um Ilya uh it was originally supposed to be around the time that Chad came out and so that their cost person tells me um listen just so you know this interview would be scheduled around the time we're gonna make like a minor announcement it isn't it it's not the thing you're thinking it's not gbd4 but it's just like a minor thing uh so they didn't expect um uh would it end up being um have incumbents gotten smarter than before so it seems like Microsoft was able to integrate this new technology URL there's two there's been two really big shifts in the way incumbents behave in the last 20 years that I've seen the first is that it used to be incumbents got disrupted by startups all the time you had example after example of this in like the mini Computer Micro computer era Etc and then clay Christensen wrote The innovator's Dilemma and I think what happened was that everyone read it and they said oh like disruption is this thing that occurs and we have this innovator's dilemma where we get disrupted because the new thing is cheaper and we can't let that happen and they became determined not to let that happen and they mostly learned how to avoid it they learned that you have to be willing to do some cannibalization and you have to be willing to set up separate sales channels for the new thing and so forth and so we've had a lot of stability and incumbents for the last you know 15 years or so and I think that's maybe why that's my theory so that's the first major step change and then the second one is man they are paying a ton of attention to AI you know if you look at the prior platform revolutions like Cloud mobile internet web PC all the incumbents derided the new platform and said you know gosh like no one's going to use web apps like everyone will use full desktop apps Rich applications and uh and so there was always this sort of laughing at the new thing that the iPhone was laughed at by incumbents and and that is not happening at all with AI now we may be at Peak hype cycle and we're going to enter the trough of despair I kind of don't think so though I think people are taking it seriously and you know every live player CEO is adopting it aggressively in their company um so yeah I think incumbents have gotten smarter all right um so let me ask you some questions that we got from Twitter okay um this is uh former guest and I guess mutual friend Austin Vernon oh yeah um Nat is one of those people that seems unreasonably effective what parts of that are innate and what did he have to learn well it's very nice of Austin to say um I don't know I mean I think you know we talked a little bit about this before but I think I just have a high willingness to try things and get caught up in new projects and then I don't want to stop doing it and so I think I just have a relatively low activation energy to try something and I'm willing to sort of impulsively jump into stuff and many of those things don't work but enough of them do that I guess you know I've been able to accomplish a few things the other thing I would say to be honest with you is that I do not consider myself accomplished or successful like my self-image is that I haven't really done anything of tremendous consequence and um I don't feel like I have this giant you know uh sort of bed of achievements that I can go to sleep on every night um I think and I try and I you know I think I've I think that's truly how I feel I'm kind of an insecure overachiever I don't really feel good about myself unless I'm doing good work but I also have kind of cultivated a tried to cultivate a forward-looking view you know where I try not to be incredibly nostalgic about the past I don't keep like lots of trophies or anything like that you go into some people's offices and there's like things in the wall and trophies of like all the things they've accomplished and I'd always seemed really icky to me so I I don't know just had a sort of revulsion to that is that we took down your vlog yeah yeah I just wanted to move forward um Simeon asks uh for your takes on alignment he's uh quote he seems to invest both in capabilities and Alignment which is the best move under a very small set of beliefs uh so he's curious to hear a reasoning there yeah well I'm not you know I guess we'll see I'm not sure capabilities alignment end up being these opposing forces um it may be the capabilities are very important for alignment maybe that alignment is you know very important for capabilities I mean I think I believe I think a lot of people believe and I think I'm included in this that AI can have tremendous benefits but there's like a small chance of really bad outcomes um maybe some people think it's a large chance and so I think the solutions if they exist are likely to be technical they're probably some combination of like Technical and prescriptive so it's probably a piece of code and a readme file that says if you want to build like aligned AIS use this code and don't do this you know or something like that and um and so yeah I think that's I think that's really important and more people should try to actually build Technical Solutions I think one of the big things that's missing that sort of perplexes me is there's no open source technical alignment Community there's no one actually just implementing in open source the best alignment tools there's a lot of philosophizing and talking and then there's a lot of behind closed doors interpretability and Alignment work and I think we're going to end up because the aligned people have this belief that they shouldn't release their work in a world where there's a lot of Open Source pure capabilities work and no open source alignment work for a little while and then hopefully that'll change so yeah I wanted to on the margin invest in people doing alignment it seems like that's important I thought Sydney was kind of an example of this you had you know Microsoft essentially released an unaligned AI and I think the world sort of said hmm sort of started threatening its users that seems a little bit strange I mean if Microsoft can't put a leash on this thing like who can and so I I think there will be more interest in it and yeah I hope there's open communities that was so endearing for some reason it like threatening you just made it so much more lovable yeah I think it's like the only reason it wasn't scary is because it wasn't hooked up to anything you know like if it was hooked up to HR systems or if it could like post jobs or something like that then I think it could have been scary yeah all right before we go uh where can people learn more about the Vesuvius challenge yeah so Vesuvius challenge is at scrollprize.org s-c-r-o-l-l-p-r-i-z-e dot org um yeah check it out and uh I I think it's very likely that somebody listening to this could be the one who wins the grand prize and decodes the Scrolls okay excellent awesome okay well now this is a true pleasure thanks for coming thanks so much for coming on the podcast thanks for having me hey everybody I hope you enjoyed that episode just wanted to let you know that in order to help pay for the bills associated with this podcast I'm turning on paid subscriptions on my step stack at warcashpatel.com no important content on this podcast will ever be paywalled so please don't donate if you have to think twice before buying a cup of coffee but if you have the means and you've enjoyed this podcast or gotten some kind of value out of it I would really appreciate your support as always the most helpful thing you can do is just share the podcast send it to people you think might enjoy it put it in Twitter your group chats Etc just splits the world appreciate your listening I'll see you next time cheers foreign