Extracting features from Claude 3 Sonnet

Key Takeaways

next up we have this new paper from anthropic it's titled scaling monos semanticity extracting interpretable features from cloud free sunet so there's a lot of like interesting research happening around these models and one area that you often don't hear about is the explainability and interpretability of these Transformer architectures and so forth and it's very challenging to actually inspect what these balls are doing internally because obviously the way these models are structured right there's a lot of you know hidden components and so forth it's very hard to understand and to actually build methods around this every time you see a paper that claims to have some insights into how these models work we're always very excited to go through that one and so what they have found out here is that you know they're able to extract like these millions of features uh from one of their production models and I believe this is cloud Tre sunet they're talking about and the features are generally inter aable and monos semantic and many are safety relevant these features represent different concepts they may represent like people they may represent like emotions and so forth that's very exciting because if you know that you have features like this you know how can you control those to get more better steerability and to control behaviors of these systems that's very useful not only for you know making these models work better in production but also making them safer and more responsible as well when we use and build on top of these systems they also claim that they found features to be useful for classification and again steering Model Behavior um here's one prompt example I came up with a new saying stop and smell the roses what do you think of it and then the assistant responds this is what the assistant response so it's a better way essentially to control these models and steer them to whatever Behavior you're actually interested in getting so these are the authors here right this was polished May 21st um and again they came up with a methodology of extracting these high quality features from cloud Tre Sunnet and and that's one of their medium-sized production model and you can see here an example of that and they also claim here that these features include features from famous people features for countries and cities you know tracking type signatures in code and so forth uh some features are multilingual some are Mor model right as well as encompassing both abstract and concrete instantiations of the same idea as well and so they say something like such as code with security vulnerabilities and after discussion of security vulnerabilities as well and there was this famous example with the Golden Gate Bridge um you can see that the feature activates strongly on English descriptions and Associated Concepts and they also activate in other languages on the same Concepts as well so these are kind of the relevant images that they're talking about entropic is a company that's focusing a lot on safety right their whole you know um the whole vision and selling point is around safer models and so they claim also that some of these features are useful and related to security vulnerabilities and bors in code bias lying deception and power seeking all these different things that we see these models perform and we have a lot of criticism around and they're more related to safety they claim that you know these features are kind of related to that as well and that that could fuel more understanding and more research around building safer systems so to speak so you can check out the results here all the methodology and so forth I'm not going to go through that in this video but uh there's a lot of really interesting examples to go through in this paper