The Penguin Barcode - Computerphile

Key Takeaways

it's a kind of what you might call a visual code recognition technology that sounds a bit Posh doesn't it but it really means you point a camera at some image and the computer recognizes something about the image and it can trigger some information and if that still doesn't make sense then um you actually will be really familiar with these things because they've been around since the 1950s here is the barcode which you'll find on all sorts of products in the supermarket at home turn over anything think it has a barcode and this encodes information a series of numbers in this case into thinner or thicker lines pointer camera or camera like device at it uh and and it triggers some information um here's another kind of code that you're probably familiar with which is the QR code which is an evolution of the barcode to encode things like whole web addresses uniquely so you can go from some kind of poster on a wall to actually kind of visit a website by pointing your phone at it they've got a really interesting property and then when you look at them visually you're pretty aware they are you can see immediately that's a barcode and you can see immediately that's a QR code and that's kind of good you know what it is and you know where to point your camera and you know what you're looking for and that makes things good but in other circumstances it's not so good because they are well how can I put it they are pig ugly really aren't they let's be clear so there have been various attempts by companies to kind of improve the visual appeal and you can actually see them a bit here there are some really clever designers who are taking things like barcodes and adding extra bits of logo to at least kind of prettify them or do various other B it's if you did that really cleverly you might want to use one on your product a bit more visibly um but there's no getting away from it that at the end of the day unless you want the world around you plastered with barcodes and QR codes um the aesthetic of these limits some of the situations in which you'd want to use them so we were kind of interested in uh how could we create visual markers like these but but that had a kind of a nicer aesthetic and also how could we give you creative control over actually making that aesthetic in the first place and we picked on a piece of work that was um done by a guy called enrio Costanza while he was at eth in Zurich and he came up with this um idea of using the topology of the image as the wave encoding information so what does that mean that means that I need to draw a a certain number of regions that contain a certain number of blobs within them and if I get that that number right then the shape of them doesn't actually matter so let me have a go this could be interesting I'm afraid I'm not a great visual Artist as you're about to see so this is me drawing what of I drawing here I'm drawing topologically five regions that are all connected together there you can see them and I'm now going to put some solid and the rule of the game is these things have to be solid that one's a DOT and that one's a line that one's a cross another couple of dots because I'm so boring I can't think of anything else to put so I've now created a pretty simple topological shape it consists of five regions each region contains a certain number of dots uh and in this case there is one one one one2 that could be the Code 11 one two maybe let's have a go I now have my aesthetic codes app running on my mobile phone here so um I'm going to just have a bit of a scan of this and lo and behold that uh red marker says it's found what it is and it's now eventually um going to kind of take me to what whatever I want it to go to in this case I programmed it to go to a website which happens to advertise aesthetic OES of course but you you know that you could program it to to point to pretty much anything you want because it's it's only concerned with the number of regions and the number of things within those regions the key thing to remember is that the shape doesn't matter so let me have a another pop at this so that looks suspiciously like it's the same code oh there we go we've got it again and it's going to take to the same website the order in which these appear it doesn't matter at all yet it's just the number of regions the number of subbits they contain the thing that we did if you like to move this on was that we got it out of the hands of uh computer scientists like myself who don't draw particularly well into the hands of people whose's you know whose skill it is to draw beautiful patterns so we took it to designers and we started to teach them how to use it to see what they would come up with and that's where things got really interesting so um we worked with some folks at Central St Martin's College and camic design department and we gave them some training and we said come back to us with some patterns and here's what they created we gave each designer three codes so that's code one that's code two and that's code three embedded within a pattern What's Happening Here is that the code is actually appearing several times within this pattern and that makes it more robust in computer science terms that's redundancy built in and we'll come to why that's really important in in just a moment so this is the same design designer who's created three different patterns with the Co in so this is the same information and this is what's so beautiful about this technique you can kind of create these different designs that embody the same information or very similar looking designs can embody different information and it's all down to the skill of the designer and how clever they are at hiding these codes within the images as to which of those effects they might want to create so it becomes a really playful and creative kind of way of thinking about design and you know we can flick through a few more there's some lovely designs here you can see this is Annabelle D's designs she's gone for a range of styles from the kind of literal to the abstract uh and so on the penguins which are proved from Nicole extremely popular so we have this kind of pattern book of designs I guess the interesting question you might ask you know kind of what is it for I suppose why why would you want to do this and what we're interested in is opening up a conversation about how we might decorate the world around us with interesting interactive surfaces of what you might start to do with those and this is an exploratory process so to kick it off we um we're fortunate enough to work with a restaurant chain called busaba based in London and they're a kind of Thai restaurant they take a great deal of time to kind of if you like Source their recipes and do kind of field research and and they kind of put them together really carefully so there's a lot of background and history to the the recipes and they also tend to kind of serve them in almost individual dishes so worked with Baba to start to create create a series of restaurant objects that might tell you stories so we put some of the patterns you've just seen onto plates this brings us back to the earlier mention I made of redundancy so one of the reasons we wanted redundancy in the patterns is because uh if you take a close look at this plate you'll notice there's lots of specular Reflections on that are happening here uh and these specular Reflections are highly visible to the camera when you point it that so these will actually blot out wherever they happen I'm finding that so they're actually going to blot out the pattern so in order to avoid this kind of things you really need to decorate the object with multiple versions of the code and of course plates get dirty or Fabrics get covered or marked or scuffed or whatever so redundancy which of course is a well-known Concept in computer science comes back in another another way here so basaba worked with us and we kind of uh imagined a number of uses for these we had plates that for for example would give you the particular recipe that went with them so you'd Point your phone at the plate maybe you'd even take the plate home as a souvenir or buy it and then in your kitchen you could get the latest basaba recipe place mats so when you go into the restaurant you get your place mat setting and this might give you information about uh your particular order so um you know where where was it in the process at one point we even you know played with the idea of a video view into the kitchen to see your actual food being kind of cooked a sort of menu and this might give you the dish of the day if you looked at that because that dynamically changes every day so the idea was yeah you'd have the standard menu items inside pointing at the aesthetic code would would kind of give you the current dish of the day and you know this is kind of very open conversation so we're kind of currently thinking about uh yeah other future applications of this I think one of the most exciting things for me is to scale it up to much bigger surfaces so while I'm really interested in things like interactive wallpapers uh interactive kind of furniture patterns possibly clo clothing where you've got multiple codes in a big area perhaps different codes that as you look at it in different ways kind of tell you a story or will make it kind of more interactive I mean one of the big open questions at the moment is is one about um human computer interaction is that how will people interact with these things um because you know as we said earlier the great thing about these codes is you can see where they are and you know where to point your phone the more you hide the codes within wider patterns the harder it becomes for the the end user to actually know how they're meant to interact so that's a kind of key problem to solve or if they can at all yeah or whether a surface is interactive people wandering around with their phones yeah just trying to figure out well is that a standard texture or is that something I can interact with so there's quite a big interaction design challenge there now I think there are various ways around it um I mean a simple one is you can just put more or less cues on the phone you can put uh you can put the outline shape of the Penguins on the app and that says you're looking for these and wherever you see things like penguins or whatever then that might be an interactive code or perhaps you can more subtly kind of communicate it in the design itself but in a way that doesn't destroy the aesthetic with any technique there are some kind of limitations as well um so at the moment uh we're fairly limited to the size of the code space although in theory you could have an arbitrary number of regions and lots and lots of blobs within those regions it soon gets quite unreliable for a standard mobile phone camera to be able to kind of tell exactly where the lines join up so in practice we're probably working at the moment with a code space of just a few hundred codes that you could distinguish perhaps even at best although you know the research is looking to push that further and further but you really have to contrast it with things like the barcode and the QR code which you know for their if you like Simplicity of design are really well engineered to be scalable and robust so I guess as with so many things you know in computer science there's a kind of series of tradeoffs at place it's not to say well this is the right way for the whole world to be and this is the wrong way uh QR codes and barcodes are scalable robust you know what they are the minute you see them you know when and how to interact these aesthetico as we'd call them are creative and funky you can make them yourself they can be beautiful but they pose a bunch of questions about how do you know where they are and at the moment we're challenged with how do you make them scalable and you know what happens when different codes map to the same information rather than a unique thing so important as always to see it as a kind of a design space of trade offs more than there one thing being right and the other wrong we released the app if you like is something that you can test out and play with for yourself so um just as I did once you've kind of read through the website you may work out what the rules are and at that point you can start to kind of create your own designs and test them out and we're keing to hear from anyone who has a go at that the order in which the kind of the regions if you like are found that doesn't matter but if you have different codes within a bigger pattern that might matter the order in which you expected to search them out