George Hotz | Programming | exploring the radeon 9070XT | AMD | tinycorp.myshopify.com

Key Takeaways

Don [Music] donkey donkey don donkey well I'm streaming we're singing the don don Donkey song on stream and we're eating pills good morning everybody have you had pills this morning these are stale pills they're not fresh pills but they're still pills oh there we go yeah pills L streamer rip all right is that good is is is it live [Music] now yeah we're going to Raw Dog the internet let's go all right camera's good now no one got me I don't know why my vpn's being shit I don't know uh okay let's go we're going to start things off this week by reading some comments from last week no I mean this this always sends me down a rabbit hole in a spiral so I'm sure you guys are here for it while we eat pills but just while we eat pills all right because there's some comments here that are ridiculous what a wonderful start of the stream conversation no that's a good comment we only focus on the negativity George ranting streams are the best no no no no no we gotta we gotta George is right on with the Roman Empire debasing their silver coin with ler medals to increase the amount of money money no you don't understand we have to increase the amount of money you understand how stupid that is I think that everybody here against the gold standard is planted by the CIA um oh this guy had reasonable critiques of Hong Kong uh what a banger beginning wow it's only it's only the it's only the positive comments you're the first guy I know that pretends to be an anarchist but despises activism oh yeah yeah I'm a real I'm a real see you got to make a difference between cyber information Anarchy and real life Anarchy information Anarchy great crypto Anarchy great real life Anarchy fuck that shit you know why because in real life that's when they pull the gats out in Internet you can just close your eyes like Tyler Creator said so that's what I'm saying uh so we can have Anarchy but only on the internet you can't have Anarchy in the real world because I live in the real world okay does that make does that be clear oh yes love how your POV sounds so naive sometimes that trust feeling of yours regarding interview by Chinese officials I actually really like um this comment uh well this comment this guy's an idiot but um the trust feeling is definitely one of Georgia's correct ticks unlike the US China has an actual culture that influences the way everyone acts that's a gross oversimplification because no capturing culture with words anyway people aren't defined by politics you have to experience a place firsthand to understand and on this small bit of public trust George has experience and I'd say he's correct my brother got caught attempting to take a knife onto a bullet train and they simply confiscated the knife marked the stuff on record and we were ch all throughout officials don't really have any animosity and if they do they don't show it and they friendly and understanding this this is a much better way to put the distinction than I did um and then like you know yeah he's absolutely right people living in medium trust societies don't get it uh I really want to find the one where some somebody's talking about oh yes uh George is very smart with code but stupid with politics and people in general okay so I mean I wrot a company I run two companies I mean I don't really run comma anymore I'm mostly uh I do run tiny Corp uh I did found comma and uh I employ like 30 people uh so you know just just just how many people do you employ right that's that's what you always got to remember this about about haters right if you want to hate see the best thing to do is get a lot of skill and then then when you hate on someone it actually matters um but no the real one that I wanted to find uh you believe the ideal Society is one where everyone is Rich and that's not the worst thing you genuinely believe it is possible for everyone to be rich hell yeah it's possible let's just all be rich when to be rich you need to EXP exploit the working class and steal their Surplus value like stop reading Marxist shit what that's not how it works you you you being rich see and and this is the distinction I Define being rich in absolutes if you define being rich in relatives I if for you in order to be rich somebody else has to be poor somebody else has to be below you because otherwise how would you know you're rich you're a terrible person and it's this gerian resentment that uh you know is leading to the downfall of Western Society because that's not how being rich Works being rich means you got a nice place to live you got good food you got nice electronics life's chill that's what being rich is so yeah everyone can be rich we can all have nice stuff we can all have a comfortable life we can all uh yeah and and we don't we don't exploit the working class and steal their Surplus value we exploit technology because technology is the best thing ever right think about it you used to have to like plant shit in the ground by hand then we built tractors is that exploiting the working class is having a tractor exploiting the working class or is it exploiting nature we need to exploit the shit out of Nature and make everybody super fucking rich you know that's a nice world right but this it's this it's this it's this resentment it's the same thing it's the same thing the guy's talking about about the animosity and that is why the demoralization of Western Society is just beginning because you got to get over that and it's going to take a long time and a lot of grounding people into the dirt in order for them to get over it we got to let some people starve in the streets until there's people starving in the streets no one's going to be ready to get over their resentment but he's a billionaire you can't have billionaires being our government okay okay you deserve no food um George you can't just say that I can and I just did uh full white supremacist based on lik Freeman podcast yikes yikes um no but where's the other one that I wanted to talk about um yes George is the type of person who has never lived under true oppression or real poverty uh and I replied to this guy I replied to this guy because user TD5 gy2 fh3 P where do you live I'm curious I'm curious because I would say that there are a few places in the world where there is true oppression maybe say North Korea uh or real poverty maybe say South Sudan um so there are definitely places where there is true oppression or real poverty but if I find out you live in fucking America and you're talking to me about true oppression and real poverty yeah you want to talk about the type of person who's never lived under true oppression or real poverty if you're truly from one of those countries where they have serious problems in their government um sure but yeah how old are you and what country are you from it also it definitely existed in the past right if you go back to like 1950s China yeah we could talk about oppression and poverty you fast forward to 2025 China yeah there's no oppression or poverty um I mean you know there's things to talk about in the west of China but if you're like [Music] uh but I don't think this guy lives in China I mean this guy guy speaks English so that's what that's what you know where's your where's your true oppression or real poverty are you claiming that you live under that or are you claiming that you you read about it in some book you watched some YouTube video about people who are oppressed yeah watch that YouTube video are you still in Hong Kong insult she why would I insult she he's bu a nice Society that's what I'm saying I'm not here to agitate because and I'll tell you about this here's here's here's this is like I don't believe in voice I don't believe that I should come here to Hong Kong and be able to have a voice to change a system I have no right at all to change a system what I do have a right to is exit if I decide that I don't like the system here I can get on a plane and I can go somewhere else and that's your right in the world I think that's the universal human right the universal human right is exit if you if you are and you can tell this you can tell this by th those are you want to talk about what countries are oppressive the countries that do not let you leave those countries are oppressive oppression doesn't mean that you're allowed to you come into my house and you insult me you're going to get you know thrown out of my house but you're not going to get held prisoner in my house and there's a big distinction there you're going to be shown the door I show you my nice hospitality and you come in here and you insult me I will show you the door and uh yeah I think that's I mean I think it's happening right now in America if you want to come into America and then start going off and start insulting America well I think I think there's the door I think I think all the people who are who are who are doing that for uh America all the people who are like I want to live in America and I want to insult it that's the door um George but you insult America in the Stream all the time yeah and I'm in Hong Kong no I'm not saying you shouldn't be able to to to criticize but the kind of people who who live in America and talk about how America fundamentally is a country founded on slavery or racism or whatever bullshit come on come on all right what what country would you like to go to uh oh yeah I mean yeah no the Ukraine situation is incredibly sad um if if this person is if this person is posting from Ukraine uh credit but I doubt it uh special economic zone in the US yeah but only if it has real power I think you could I think you could fix a lot with special economic zones in the US uh but again you need you need real power behind them like the ones in China uh yeah like I mean I don't know it's it's it's if if that's if that's what it comes to there's a question of how much how many like what percent of the people in the US are too far gone down a resentment uh like rabbit hole what the demoralization is just beginning um you can't exit the US income tax even if you live on Mars true true Okay so let's not spend uh I'm a bitter and resentful person who has no valid argument I eating pills ma'am oh how do people eat so many pills every day well I think what'll happen as the US Retreats from being a global power is that the US will no longer be able to enforce its Global taxation system so they'll be able to enforce it with friendly countries but I think countries like Dubai will say hey look uh move to Dubai uh if the US comes looking for taxes will be like I don't know is there even enough gold in the world to support the amount of money in circulation that's a good question of course there is it's just what we're going to price the gold at Dubai isn't a country I think it is it's an Emirate the UAE Dubai is a city that's true it's an Emirate it's in the same country as Abu Dhabi how can a small country resist oppression and maintain it sovereignty uh well people forget this about sovereignty sovereignty is literally just what you have how can a small country resist oppression and maintain its sovereignty again it depends on what you mean by oppression Hong Kong used to be a British colony I don't think these people were oppressed by the British not even sure what that would mean um I think about decentralized science does it have a token if it has a token it's fucking stupid okay that's that's a simple rule that's a simple rule of here's a project does it have a token yes stupid that's that's simple all right let's do things let's not just rant today that was the opening rant um we have a beautiful uh 9070 in this computer I think technically Dubai is not a country I think the UA is a country because like you can drive to Abu Dhabi and I don't think you have to go through like a border so but good you're starting to figure out that there is know like oh this is what makes stuff a country and this is what makes stuff not a country uh it's just kind of all whatever kind of accepts you blah blah blah blah [Music] blah if you're replacement AMD driver is good why does the Tiny Box web shop still lift the driver is mediocre well maybe it's time we change that should we edit it edit the tiny you want to edit the the website let's edit the [Music] website donkey now what should we change driver quality [Music] to uh developing right like a developing country does it sound good um but no really how about this what if we just went here and read comments that said good things maybe we'll do that next week maybe we'll only read positive comments look at look at how tiny my screen is now this is unbelievable I don't know how anyone works on a 1080 screen I mean it doesn't help that the Mac doesn't have uh sub pixel rendering [Music] [Music] developing now move me I think the website is uh I think I made this uh open now to so anyone can submit P quests to it beautiful web dev skills right thank you I used I used react and compiled it with Babel and I love that level IOD troll we like list this like super complex stack and he's like bro no I used PHP and my squel donkey okay so I've put the GPU in the hole and we can see it's in the hole there but when I do rock MMI well after taking a long time it tells me this which who's is someone using these gpus right now why is there a percent used just my vs code server okay I don't know all right let's upgrade Rock M who's ready for this experience or is this even supposed to work for what rock M do I have 31 is this going to support I don't know all right should we just YOLO it let's YOLO it who thinks this is going to just work come on AMD I have faith in you now I'm an AMD shell uh wait oh did I what what releas is this 2 yeah we could use dialogue we need more dialogue need more open and friendly dialogue uh install the package maintainers version all right all right sure download 2.7 gigabytes that sounds like a good number of gigabytes what's the goal for today well we're just going to play with uh this this this GPU and we're going to see maybe we can get the driver working maybe we can get the new Matrix cores working we'll just see what we can get working okay so AMD released this GPU um now there is a question if we all got majorly rugged on the price so they announced the price this is like the new scam they seem to pull announced the price at $599 but you can't actually buy them anywhere for $599 uh we'll see I mean maybe this will actually go down to 599 and it's because retailers have inflated prices wait egregious what I paid 950 for this GPU I mean I haven't even gotten a 590 yet I have no 59s I have some on order with a large company that promised me they were going to get me some and then I think Nvidia said uh you know we're not putting the tiny boxes in data centers the tiny boxes are not for data centers it's not the kind of thing you're going to put in a data center it doesn't make any sense it's a why would you use knock to a fans if you're going to like even even the Tiny Box Pro is not really I mean you could maybe put them in data centers but like this is the only one we're building with 90s it's not a data center computer it is an AI computer Nvidia basically told my understanding is they told all the board makers that you better not sell to AI fucks and you better only sell to Gamers uh so considering where AI fucks we don't get any [Music] I'm pretty sure you've talked about it but what do you think about cabus their tokens per second is insane and I really want to use them for work well then use them bro what's stopping you what's holding you back is it the don don Donkey song Because don don don [Music] donkey that's going to take a while to install [Music] we can start with the driver if we want to go crazy do want to see how far we can get with the driver we can like RM mod amdgpu let's just let this install see if it's going to work first uh yes we're playing with this GPU it's really cool AMD releases the uh instruction set so this is the instruction set for that GPU you can learn about the [Music] wames they have new fancy wames oh I didn't show you guys too we also have these computers you know what's in [Music] here a mi 300 X's so we have two of these computers they were gifts from AMD uh we're working on this stuff too well this one this one's a bounty uvn is working on it um getting the run time supported with this but we're struggling right now because the this GPU is not actually each one of these is actually uh eight gpus show you kind of how it works so there gu kind of shows you um this is what one of the mi30 X's looks like uh it's it's like wild has is a better picture so each one of these is pretty much a GPU so you can see one 2 3 4 5 6 78 uh they're all connected with a very highs speeed interconnect though and the interconnect speed is faster than memory and if the interconnect speed is faster than the memory in a lot of ways you can just treat the whole thing as one GPU but if I go in here so there's a tool called UMR um it's pretty cool UMR is uh would like you look into the registers of uh didn't actually like install where is this shit Source app UMR you got to run that shit with pseudo um let's see what we can do here let's list the blocks so the these are the blocks inside this GPU and you can see that there's uh eight gfx course so let's go back to this computer let's download UMR and let's see what blocks are in the new GPU [Music] do you like how big my monitor is now can everybody read the text roam is not the firmware on the card so roam means many different things um you guys want a little lesson on how this shit Works let's let's read the tiny gr docs I think we kind of talk about in the docs a little uh so you start at the driver uh the driver is what sits between the hardware and then provides like an abstraction to be able to do things like uh allocate memory create cues like you can create these like different cues these different cues are like compute cues sdma cues just like it brings up the physical Hardware to a point that you can start to send commands to it and then you have this thing called the runtime um I can show you where these things are in tiny grat so tiny grat has a complete Sovereign stack for AMD the only non- tiny grad code that's used at all in the AMD workflow is llvm I think I've talked about this before but so this is the runtime so the runtime includes things like a compute well so the compute queue the queue itself is created by the driver uh but this allows you to actually put things in the queue so this is like putting an exact command in the queue you have to set up some registers so like these registers include here we can look here at like uh so this is where the local size is is uh where's the global size so yeah we put the the global size here in the dispatch packet um the local size is just like the width of the of the simd within the work group processor but then this Global size is like how many of them to do and then you send that and then you can actually run these uh commands on the GPU and gpus are really very simple machines uh so this is using what's called pm4 so AMD has two lowl ways uh there's this thing called the mech or the ace I know they use lots of stupid acronyms but um so here we can look at it for the they have this little document for they might 300 so this is actually one of the what's called an xcd so those four things are called xcds I don't know where that picture went but um there's there sorry those eight things so there's eight of these in the GPU um these are the compute units where the GPU compute actually happens and then these compute accelerators are what reads off of these uh hqd stands for Hardware Q descriptor so it reads off these Hardware cues sends them to the hardware scheduler which actually puts the jobs on the CU of the GPU um wow you know you really this is why these companies use acronyms like this AMD and Qualcomm love like tons of ACS Uh Oh you mean the ace you mean the mech which is part of the me which is overall part of the CP yeah that's right uh so yeah um yeah there's there's eight vertically stacked uh xcds uh yeah so these packets are parsed by these compute accelerators and then these this these are just registers on the GPU in the hardware scheduler so that'll get the hardware scheduler to actually uh put jobs on the compute units you guys know basically what a GPU looks like right I shouldn't I shouldn't if you don't know what a GPU looks like go go like go to like Noob college or wherever noobs go uh so this is this is the 7800 XTX you can see each of these things is dual compute unit um H these is the Shader engine but not a separate GPU I know it looks like a separate GPU but notice how the l2s are here so it's all it's all just levels it's all just like different hierarchy what what is it levels Jerry ah okay so we set it up I don't know if we reboot the computer who thinks it won't ever come back oh please come back yes come back good go to computer um donkey and it has 16 sdmas so we can bring up that m300x picture again jeez what was that picture this picture so it has 16 sdmas I I haven't looked into this totally yet but the I believe that it's two per one of these so you can see this is 128 gbt per second on the pcie but it also has Infinity fabric uh linking it to the other gpus so there's sometimes a picture of that but it's just it's a full mesh so you have eight gpus so there's seven link out to the other gpus and then one link back to the host with the PCI Express so yeah we're having lots of fun with these um and then we also have the rdna 4 I picked rdna 4 because no one's really working on it externally and uh I think it'll be easier there's stuff about this chip that's complex um rdna 4 might just be literally simple stuff yeah so this shows the there's the ram which one of these is an xcd they have 38 compute units each you back yet I'm back I'm back I'm back uh but yeah know the GPU Market's actually super Bleak like I said I've got my hands on no 590s yet I'm I going to pay some eBay scalp or $3500 for but the gpus aren't even ever going to be sold for $22,000 like they're all doing a scam now you try Vibe coding you get you get by the way um I hope I hope uh uh Mr Peter levels doesn't take insult to when I describe his Vibe coded flight simulator as mediocre because it is a very mediocre flight simulator but the fact that he's making money off it love like the Indie Game Dev we like I spent five years on this game and I worked on every pixel and it doesn't make any money and then this guy makes this bullshit in two weeks yeah based um you know you you you don't like like you got to think about how to hook people and entertain them uh well you know what I got a lot of gpus does anyone want to buy a tiny box red I have a ton of Tiny Box Red's in stock can I interest you in a red the driver is developing now maybe someone will click that buy button oh no it hasn't come back yet boys it hasn't come back yet if it's just gone it's gone no I can do some stuff over on this monitor that might try to fix it um so this is a prototype of a tiny box two so it has pci5 amd's driver bricked my computer no I can connect to the BMC hopefully over here uh no sales is not above Tech you guys are idiots you guys are missing the whole point it's it's not like that that actually entirely misses the point it's not about sales it's the fact that his game is a tweet and I can click a link and I can engage with it it's that simple versus well I'm not going to sell on Steam because because steam is exploited to to to minorities I'm only going to sell it on this indie game store and you can't use credit cards like visa and MasterCard because because they they don't allow they don't allow the Wikileaks donations to the sex workers or whatever whatever whatever you got to you got to use the Halla Network and and buy it through this and then no no you you you made me jump through too many Hoops that's why tiny guard doesn't have any dependencies there no Hoops to jump through you just got to you just got to pip andall tiny gr you can go do that right now who's pip and stall and Tiny grad right now um user experience over sales user experience over everything it's not about the tech it's about the user experience John blow's take on oh it's back oh it has returned okay who thinks it's going to work now you think it's going to be there who thinks it's going to be there oh it's there whoa whoa D we just have a good experience with AMD driver shit why are all of these like on what are they doing why you doing something all right um let's try hip equals one [Music] no I don't know which GPU it's actually targeting [Music] um it going to tell me it's not going to tell me [Music] okay the problem is this can actually just be using the uh I don't know if this is using the 7900s or it's using the how are we going to find that out oh let's also try a rock and bandwith [Music] test amd's driver situation has gotten a lot better uh just overall the fact that you can copy and paste their one thing uh you guys know I'm an AMD investor now so this has definitely changed my Shi calculus we now shill oh look look at this one whoa their PCI even works pretty well so you can see this one's faster because it's uh let's just check D message and make sure we're not getting any a what are those errors oh yeah I mean okay uh no apparently so what we think the issues with the 7900 XTX were we actually think it was a hardware bug uh that was causing a lot of the issues so we have a fix niml genen put a fix [Music] in no more recent than that um I think you can see it in the dri so hard on this tiny monitor can't see [Music] anything oh here we go this one uh turn off unor dispatch so there's a flag on the hqd you guys should know what that is now called unor dispatch and apparently this is what they had to turn on to fix their race condition there's probably some Hardware issue in the uh Hardware scheduler on rdna 3 where you have to turn this on if you're going to dispatch both uh it's possible it's fixable in firmware so there's there's the Mac which does the things then there's the me mac which does the compute things and then there's the uh the me which does the graphics things why is this so slow okay um um but yeah so there's this flag called unor dispatch uh on the on the PQ on sorry on the hard work you uh yeah so I think this is and it's it's not actually the mech it's not actually the mess all my complaints are not actually about the mess the mess just turns out to be what reports the error they did not have any sort of CI um as far as I could tell uh Anish is pushing for it I think I think he's he's making progress on that front he's talking about the right things he's talking about CI he's talking about continuously updated builds like nightly like just basically normal software stuff that I think AMD kind of lacked uh but the Situation's getting better it seems so you know I'm an investor uh I've invested a quar million dollars in AMD I put my money where my mouth is um because if AMD like I'm so happy to open eye looks like such clowns like they're they're posting all that stuff where um oh the US and the China and the models and the open source and the like you guys are such clowns I I hope Sam Alman understands that he destroyed his company open AI could have stayed open they could have easily attracted the money there would have been no problems at all they could have attracted the money they could have stayed open they could have commoditized this technology and samman would be a hero instead look what happened opening eyes models are like kind of on par maybe a little better than the competition but not seriously uh you now got you now got Elon on your ass Elon never would have done this uh so it's it's yeah they're really trying to make it political instead of building an amazing product um it's going to be interesting to see what happens with deep seek I hope that the Chinese government is Chill about deep seek and understands that the best thing to do is to kind of let them cook uh that's if if your if your goal is to demoralize the Americans just let deep sea cook um yeah no Elon I watched uh I like the you guys know Channel 5 News um he had a thing where he went through like what elon's radicalization was and he has kind of a big moment was the Biden EV Summit Biden held an EV Summit where they praised like Mary bar as the future of electric vehicles and she shipped three cars at this point I think Elon was just like holy shit they're complete idiots and their idiocy is going to bring me down um like he just imagine you're Elon you're watching this and you're just like oh my God these people are retarded they're they're actually retarded there's there's no like like like oh cuz I don't use union labor what enjoy poverty and then El looks around and he's like wait who am I telling to enjoy poverty I'm the richest guy in this country I'm the richest guy in the world shit that sounds like I'm going to enjoy poverty I don't want to do that all right all right all right all right we're going to have to do something about this all right how's the government how does how does that work uh well I think you gota like if you could have like hearts and minds and you could like control the media and stuff how much does a media cost I'm gonna ask Twitter wait how much is Twitter you see the you see the wheel turnning his head right yes so F where um okay so this obviously is not working uh why device node okay let's add a print statement to hip Live Jeff equals hyp toiz okay um um donkey oh do debug equals one only really need one thing okay so yeah that's [Music] and is there a smarter way to figure this out first let's see if we have five that one that same it's node three so maybe we set it to three or maybe it's two because we have to subtract one who knows oh there we go look at that we found it it's called device 2 uh let's want to try some map malls on it [Music] [Music] that's not going to work said bual one five jiga flops well that's not a lot of jiga flops it looks like oh we have to go to half if we want to use them anyway oh llvm error okay go VM ER you guys know Don donkey I need a big T okay well no luck there um let's look into making our runtime work it's probably really simple so this is an Ops AMD let's add a that same print here [Music] [Music] [Music] uh why didn't that print [Music] oh here it is okay opening zero with rhgfx okay so we can do visible devices equals 2 you don't need a hip visible I mean you can use hip visible devices but when you're using the uh okay so that just didn't open at all and let's figure out where the arrow was okay unsupported Arch gfx 202 up that to 13 wait no way uh okay did that just work I mean this might be the easiest shit ever [Music] um we have the vigers per oh it's like taking a while to save or something how come that's already there who put that there like it's like it's already meant to support this chip did someone like already had support for it and all I had to do was change this cool uh well yeah okay so I think it just kind of [Music] Works uh I mean the cdna one is different but the rdna one I thought might be like dead simple and it looks like it kind of was uh [Music] it sure looks like it works I you guys like want to run llama on it or something let's see like what's the easiest way to run llama now do I have like a text way download that's just sick if that works that means this like stack super portable I mean I guess it should be there shouldn't have really been any changes to this stuff especially once we get we'll have this stuff so it actually imports from the other uh files should we try to get the driver to work so this is still using the kernel driver from [Music] [Music] uh every two seconds run Rock Me SMI okay cool question are you running on a [Music] 7900 got to compile the first time as will slow shouldn't be that slow give it time or did something not run [Music] okay well that doesn't look right let's see if it works if I put on the other GPU ND ND ND ND ND ND ND what are you working on you should know what I'm working on by reading the title but you Chris something something I have no [Music] idea okay so it works on the it works on this GPU tell me a long boring story okay so how come this doesn't work on the new GPU but it only works on the old GPU let out a massive Eep and then the one bumbling first grader in a state of high alert let out a mass mive Eep to the bumbling second grader in a state of high alert and the one bumbling first grader okay that's probably that's probably really has to do with this model is uh so okay why does it not work on this one let's run some tests and figure it out [Music] yeah no errors but it gives weird response I know I have a little bit of tea left we'll go get it maybe there's a good tea in there oh there's a squid game energy drink for all right did any of those tests fail one of them oh I see that's okay we can look into that problem did any other tests fail what do we think change between the Cards yeah we're going to come out with a tiny box to with uh it's written here I have a mailing list if you're interested in the new Tiny [Music] Box um we're going to do a tiny Buck green V2 we might be out of stock of the greens forever um but yeah tiny Buck green V2 with 4X 590 it's going to cost the same as the green V1 um it's going to have a Genoa processor in it to support the pci5 I mean okay so there's a bunch of ways we can start diving into this delving so to say how does all those tests pass but then I try it with llama and it fails this is really the worst probably like one bit of subtle bullshit yeah clearly broke broken and the map malls are broken do you guys want to look at the map models or do you guys want to look at the uh why the other stuff doesn't work who's more interested in w we have a guy coming to work at tiny who's the expert on uh the memory layout for sdma changed oh you think like the offsets to these registers changed I mean to be fair I am importing this which could be different now they could have changed that they could have changed a lot of stuff yeah this isn't really the right idea this should really just wait for the refactor uh uvn is working on a refactor that's going to support the uh the uh m300x might just be better to wait for that AMD GPU Navi 10 doorbell Mech ring zero it could be something like that no I mean the fact that only some of the kernels don't work probably has more to do with like this kind of garbage it sure has local memory set default scratch size to 128 bytes per thread okay it could just be like all this stuff that's different I wish we knew where this came from it's it's got to be something like this something wrong with the local memory I don't know want just YOLO that gfx wave sizes per [Music] se all right first we got to find a failing test if nothing in Ops is going to fail um well okay first I have another way to fix this let's just disable that for now uh and how do we do that we want to go into the oh this actually this functionality was just added so if I go into the C style render into the AMD renderer [Music] [Music] well at least that doesn't error anymore but it's not using tens of course uh let's go a little let's put a little gas on them tests my test DN Auto go big fail does it pass with the other one does it work in parallel or is like 32 like just too crazy no it fils with that too okay never mind I don't know why it doesn't work but oh there's probably like some like yeah it works with four I think there just some resource limitation okay so it passes on this is the uh other GPO the 7900 oh yeah it runs out of sdma Q's to allocate that's interesting we could probably late allocate the sdma q's to not have a problem there got to work back from the kernels that aren't working I know I guess like none of these things are really stressing this shit out of locals you got to get to beam if you want to do that someone want to improve the tests I always check my get diff to see what I'm up to so you see we we uh disabled the tensor cores change that we added some debug prints okay so all those tests pass let's just triple check to make sure that that's actually hitting the right GPU let's also try to understand it'll be fun we'll we'll try to understand actually what all those weird numbers mean I didn't write them someone else did so they probably understand but I don't it's always good to understand understandings of virtue okay um I don't know you want just yell all the tests and see what we get got be some tests that fail okay good we got some failing tests let's see what's failing think it's anything interesting I think it's bullshit no device found for one okay it's bullshit compile failed [Music] initialization is not supported for shared variables huh this seems unrelated to the GPU this just might be like a different thing that I broke okay let's let all the tests run I should have let all the tests run in the first place but I got impatient so it's kind of annoying because in CI we can't run uh all the tests we only run a limited set on well it testing on physical Hardware is annoying uh because it's not like like it can be in GitHub actions so these tests might just be failing on Master if you run it on AMD Harbor unrelated to the fact that it's a new GPU let's read that code that looks suspicious and it's probably the problem okay um good we got some failures that's good uh so where would that stuff [Music] be for I'm sure that many tests aren't failing some of those are probably real weak object has gone away oh no where did the weak object go come on print out the test failed why don't you print out the name of the test that fail in the meantime these are all garbage no none of these are real no one tests this oh this is annoying we should be like more aggressive about testing this so then when we have the these kind of problems we can like look at them okay uh how am I going to find out which kernel is bad well okay so first let's get a simpler Benchmark failing we have some tests here that uh actually test to make sure if I do benchmark I hate that that loads so slowly there into that all right well we can clearly see the junk there um what if I say no opt equals [Music] one it's probably something around locals it's probably pretty subtle all it's unlikely to have Chang all that stuff's unlikely to have changed oh okay that seems [Music] real I woke up in a [Music] car it's just some optimization that's bad probably involving locals does someone want to write a test that like goes through and actually tests the all right so here's what I found about asking anybody at AMD and that was my uh my mistake let's see if we get the same answer on this this [Music] GPU uh don't ask AMD they don't know or like someone in that company knows but good luck trying to figure it out uh asking companies they have a questions policy they're just not as direct about it they'll just kind of jerk you around if you ask questions but hey they sent us to computers which is a big step for them um okay is that consistent why is it different on the two gpus that's [Music] concerning uh let's try gbt two and see if that one fails too we definitely have tests for [Music] this ask the guys at AMD well okay they do have all this like Dynamic register allocation stuff let's just look through the presentation of rdna 4 and figure out what they might have changed useless okay they Chang a number of compute units Ray tracing shouldn't affect anything the tensor course Chang we just disabled them that should be fine is it actually a 4 NM [Music] chip 4 nmet in some ways I'm very excited about this GPU imagine putting huge grids of them in a computer imagine having like 56 of these okay so what could have they is any of this real it's groundbreaking that's great thank you this is real okay I also would not put it past AMD to have bugs in the compiler so this well we can check it with hip we'll check it with hip and that'll that'll let us know if there's okay so gpg2 works okay it's just llama that doesn't work let's check it with [Music] hip if it's broken with hip then uh oh that's wrong got to say hip visible devices equals to actually let's throw some debug one on there to make sure we're getting the right one just find the locals guy um yeah guys it's broken with hip uh I didn't do it like there's probably a bug in their compiler well let's check let's make sure let's check hip on the other one yeah that one works fine uh uh yeah the compiler probably has [Music] bugs and it probably has to do with so this one is using like small D type stuff yeah is the compiler targeting the right architecture uh yeah [Music] I'm okay possible possible let's let's check all of that I mean that's okay okay okay hang on we'll give them the benefit of the doubt give them the benefit of the doubt um I'm so happy we have hips so we can check this stuff it's possible that I'm not piping the thing through right let's check the arch [Music] here uh and then let's also disable the compiler cache [Music] well putting in the arch correctly all right let's disable let's try it with o Zer it's really slow because we're disabl to compile a crash no broken okay so we're putting in the we're putting in that correctly uh let's go change those to o0 who thinks it's going to work with o zero okay now before we remember we're not going to blindly criticize AMD anymore uh because we are now investors in AMD so we now shill for AMD so I'm sure that their compiler works great and I'm just doing something wrong right do we believe I okay well um all right let's let's sanity check let's go to let's check it on CPU so with tiny gr it's very easy to run like the same thing [Music] on CPU it's just Clank this is your normal CL is it targeting the right architecture uh yeah I mean probably it's possible something else is weird here okay we have a pull request to use llvm just use lvm directly so we can try that but remember it's a new architecture so it's very plausible that there's bugs in their lvm inflation um so this is running it on the CPU and we'll see if it gives something good CPU is always a good place to sanity check hello how is that the same as it was in the other one yeah okay so notice this is the one on the on the rdna 3 GPU and this is the one running locally on the CPU so we get the exact same thing out of both of them so that's correct uh CPU is faster than I thought which chip I think it's a 91 two4 yeah yeah tiny Grand guys tiny Grand's really good it's not that fast I mean there's some places where it's okay so sanity check so it's good on CPU it's good on uh one GPU but it's bad on I have another reason to think the art stuff is wired up correctly because Bose parrot has tested this stuff extensively on rdna 2 gpus uh so I think that if there was like a problem passing the arch in uh if I was loading the wrong Co manager it should throw an error I don't think it's this like hip okay let's let try there's a pull request which mostly works the guy's got to clean it up though it's just it's just ugly code um to use uh llvm directly I don't know I mean do we even want to waste time with this the back end works it's just really as far as I can tell there's just a bug in the compiler um where is is that's did this get improved more oh good this guy improve okay he's improving it he's improving it that's good uh yeah this one notice there's been 63 comments on it um I'm just going to go over here and rebase that shit on master and then we can check it out here and we can see if it works now there's a conflict we got to resolve oh Jesus [Music] render who can read this codeine that's fine guys know the Sweeney Todd songy Todd his voice were C and his eyes some 60 common PR that has a conflict yeah I know I know um I mean know it's good look okay uh oh we have my own failures here man what do I want to do about that I kind want to commit that [Music] there a pretty good changes right all right let's check out uh [Music] not let me push rna4 support good that's enough Branch name to remember let's [Music] delete that check [Music] out that pull that great uh they said Rock and won't be supported for launch well yeah I mean the compiler doesn't work all right so now this should compile with llvm if I do AMD llvm equals 1 I think that's right oh no no I did the rebase wrong [Music] superet what's missing Arch oh AMD render is missing Arch that's what I did in a rebase wrong okay that's fine no there's no way that just worked oh it's amdor lvm of course there's going to be like six things that's it's going to bitch about yeah there we go there's one that it bitches about okay requires llvm greater than 18 all right uh okay now this is fine exception but you're supposed to tell me how to fix shit uh right you want do 224 install lvm8 that's like a it's like this here we go there an automatic convenience script now we're talking I love convenience and I oh must be run his roote whatever YOLO at least they informed him about it no this is the stupid AMD attitude no you can't just be like well technically technically technically I'm going to punch you in the fucking face like no that's that's not that's not okay like you know what would be okay here's what would be okay if instead of silently being wrong they had an assert in lvm that said hey we don't support that yet support coming here here's how you can help great that's okay silent wrong output from your compiler yeses bro yeses I thought you were an adult okay good I think that works now okay now we're using llvm oh oh we going to fix bugs and llvm y should we should we contribute to amd's bullshit maybe we'll think about it we will think about [Music] it iend the tale of Sweeney Todd llama where's my no no no no no where' my llama go oh my llama's over here is it all right disable compiler cach don't use hip because hip's not going to [Music] work so this should be closer to upstream and not using the uh why is it not calling llvm inet I feel like that did actually run with lvm I don't really know why about I call lvm inet amdor lvm did I spell it right [Music] how come that didn't open L yeah I mean okay a real simple way to check is we can just do buug equals 4 and see okay it's lvm I don't know why it's not printing that or did I just miss it I just missed that like you can see that that code is [Music] lvm oh okay now it's working I don't know whatever I just supped to missed up oh because I probably it probably only prints with thebug equals uh two yeah okay that's fine no I know they say they won't support it but again having your compiler be silently wrong is like so deeply okay we're getting eyes uh even with lvm so there's probably a bug in lvm what else could it be I I am not interested in tracking this down it probably has to do with like yeah like strange D types and lvm right should we build lvm from source oh I don't like that I just ran the YOLO script because now I don't know how to uninstall it Jes install everything um really I mean I wish we would just print which lvm got opened I think I have that here what debug level do I have to have to have that oh okay [Music] three okay wow so that installed lvm 19 and lvm 19 has a problem okay again before we before we go crazy and say it's the compiler uh this just did I do anything wrong we got to get the LM does AMD have their own Fork somewhere okay good point uh let's [Music] see Rock M AMD llvm project uh has not been updated in four years I mean if we really want to prove it's a compiler bug and then we can really be like yo you guys fucked up we have to find the compiler bug and fix it right you can't if you want to criticize somebody you have to show that you're better than them and then you can criticize them but if you're just going to criticize you don't know because you could not be better so I think we're gonna have to find it all right who's excited yay cat show me the car facts love gang I don't want to die here nothing to do so baby just lie here guys know that [Music] song need to get that AMD uh all right so might even be in like loading the weights oh okay good good let's read this guy you mean the mo one of the most famous chip reviewers of all time oh oh yeah this definitely looks like bugs all right so it's just I mean it's just Mainline lvm all right let's start by building Master because that's what you do should build lvm pretty fast [Music] [Music] [Music] so I will also note that if I do no opt I get a different answer but still not the correct answer but these problem programs are probably going to be a lot easier to debug uh oh I do have some fancy beam search stuff that might just automatically find the issue I wish I had a simple way to just check every kernel with the CPU yeah so it's a different answer but it's also wrong I mean how sick would that be if it was just a simple way to check all the kernels with the CPU and then like tiny grad kind of is the regression test for everybody how's my cloning of lvm going not well I mean it's it's unlikely to be a different bug right uh we should also try I have a different llama I can use um this is this is this should work I'd rather debug it in clang than LM but that still doesn't tell me much just because I can read clang easier I don't know is that wrong I one of these Colonels is just wrong um we can dump the assembly with [Music] six okay look similar to no I know we have the arch right what I'm setting it right am I not [Music] the arch okay I think um let's disable the jit and let's find which Kernel's wrong and let's do this at the tiny gr layer because it's going to make my