Contributory Audio AR: Practice and Technology

Key Takeaways

yeah so thank you it's i was extremely excited when uh i learned that this class was happening at all just because i've been doing things that i would consider audio ar for uh you know 12 years now probably um i'm very old so don't don't ask me any particulars but and uh it was just it's so exciting to be at a time now where things are becoming um i don't know if i'd call it mainstream yet but at least getting to the point where people are talking about it and teaching it and uh and uh you know so i was excited to learn from what you guys are uh going through uh in learning in this class and uh to contribute to the extent that i can so i don't know what just happened there but i'm gonna press play again on my my back um i have uh i'm a sound artist um and technologist and i've been um doing work with uh uh what i would call audio ar for quite a while as i was saying i work a lot with spoken human voices and on the contributory part of my work is that i i generally do work that um that feeds off of responses and commentary that i get from people who are experiencing the work so i ask people questions they i record their answers or they record themselves and then those those uh sort of feed back into these evolving works that are often location based and uh and sort of change over time and and often have a musical underpinning as well so i'll i'm going to talk about a whole bunch of examples of my work and then talk a little bit about the technology that i've developed to um to support some of that work which is what i would call a contributory audio ar framework platform um called roundware and then i'm hoping there'll be plenty of time to to ask questions because that's probably going to be the most useful and interesting part perhaps so feel free to jump in anytime if you have questions but there will be time at the end either way so let me just leap in i don't know what order things are gonna happen here but i'm gonna go for it whatever it is okay so so what is roundware i just uh mentioned a little bit about what ramware is but essentially roundware is a a framework um or a platform depending on your preference there that allows for the creation of basically it allows to overlay any kind of physical landscape with a layer of audio and it's actually multiple layers of audio that can interact with each other and i have this quick sort of example here of of the basic elements of what a roundware installation covers um so let's say you wanted to do something on the uh the the mall in dc and uh this is just this is the mall and the first thing roundware allows you to do is create sort of regions of what i call ambient music uh well i don't call it i mean lots of people call things ambient music but um it's a it's sort of a continuous layer of ambient sound and those right now they're just circles that are drawn here you could have any shapes you want and basically the way to think about it is each of these circles has a different underlying um ambient uh it doesn't have to be ambient but often in my case it's ambient soundtrack and as you perf as you walk from one to the next you know one fades out the other one fades in etc you know here you're you've got all three of them playing so this is a way to compose a piece of music or a audio experience of some sort that really it is very highly dependent on how somebody traverses an actual terrain so this is quite simplified in the sense that there are only circles here right right now roundware started out you could only do circles i've since moved on to polygons which is very exciting and so you can draw just you know a block or you could just draw right around here on a map and basically assign any any um audio to that and in addition to the the sort of underlying continuous layer of audio there are two types of sort of what i call momentary audio that that can be interspersed within that they're usually smaller bits and they're usually pieces of audio that kind of play once like you right you get into the area and then it's triggered it will play once and then it'll be done as opposed to sort of a looping ambient situation and those can be either curated by the person creating the project you know me or whomever is using ramware or they can be added using the same app they can be added by the people who are experiencing the project so these red ones are to indicate that there's another kind of type of content which is a dynamic type of content which happens over time when a particular piece is installed so if i installed this in dc on you know january 1st and you know the app was released and people could wander around listening to everything which was the ambient layer and then these yellow dots and then they added their own on top of that to respond to various questions that are asked within the app and you know depending on what what the piece is about i can ask questions any any question i want i could you know look at the clouds and tell me what you think and then they could do or you know tell me uh what this building reminds you of or you know anything at all and you know describe the person who just walked past you or something like that there are lots of you know explorations that can be done but the point is that these red ones are sort of indicative of of a co-creation sort of element to what i do a contributory element and a way in which i lose control over a lot of what the piece ends up being which is both exciting and totally nerve-wracking as as i'm sure you can imagine but um it's uh it's a a crucial part of my work and about and of what roundware allows because again roundware was developed obviously to you know scratch the own itch that i have my own itch that i had to uh to create um uh the pieces so i will uh keep on going i'm gonna kind of crank through some of these um some of the history of where roundware came from just relatively quickly although i'm gonna show one video which i think will be useful so roundware is actually named because the first project i did with it was called round so this was a project for a museum outside of new york called the aldrich contemporary art museum and they asked me to come in and do something that would be some sort of slightly resembling an audio tour but not be an audio tour because they really didn't like the general approach of audio tours to sort of tell you what you're supposed to think about the artwork et cetera et cetera and i decided that what i would do was basically create a sort of soundtrack for the museum so when people are wandering around they could hear that and then um infuse it with comments that people made about the artworks in the museum that would be heard when you were standing in front of that artwork so you could be standing in front of a painting and you could hear um you know the curatorial curator smart remarks about the painter but then about the painting but then you could also hear a six-year-old's comments about the painting as well and the idea of sort of democratizing and making um making art into something that i feel it should always be which is that anybody can appreciate it anybody can like or dislike anything and you don't need to have a phd in in art history or some kind of you know critical discourse to uh to have that sort of appreciation so that's where round game roundwork came from the the title the round round is is sort of the reason i named it that which is somewhat useful perhaps to to understand is um first of all king arthur's round table where everybody sits around a round table and nobody's nobody's opinion is sort of more important than others and then second of all the notion of um a musical round you guys know what a musical round is like row row rower boat and everybody's kind of everybody's singing together but kind of in an offset way which i think is is somewhat what roundware enables you to create so i'm going to hope that this is actually going to play a video this was a piece that i did at the de cordova sculpture garden museum out in lincoln outside of boston and it was one of the first pieces using roundware and i think it will give you a little bit of an idea of what roundmark can actually do and what my work is like so escapes piece is i think of it as a musical composition that is a spatially related musical composition [Music] and the volumes of those parts um fade up and down depending on how close you are to where the center of that particular musical part is okay let's walk the music is changing now yeah we get these bells more towards the entrance of the museum it's more of a sort of melodic part a very slow melody all the music is based on some of the topographic characteristics of the park itself [Music] i know that it is oh there's some voices hey there goes a chipmunk right where i'm standing some guy saw a chip there goes the chipmunk again um which could have been a chipmunk from two weeks ago yes it could have absolutely i think it's kind of fun to think about where that chipmunk is right now [Music] [Music] wow so that guy went and did this they stuck their head into this open glass dome that's part of the structure and these are the acoustics he was talking about and now i'm doing it too you might not have done that otherwise i'm pretty sure i wouldn't dab [Music] i am weak being recruited into the israeli army i'm going to be a soldier i'm very scared pretty excited being here is pretty peaceful i guess compared to what i'm about to go through good luck to me oh wow this rain that's right and they recorded the rain and left it here for us it's not raining though right now that's pretty cool so um so if you want to make your own recording okay i'm looking at this sculpture it looks like a piece of like hvac ductwork from a building but it kind of twists in a way that reminds me of like a lower intestine we hit stop and hit stop and then we can listen to it back okay i'm looking at this sculpture it looks like a piece of there i am like hvac there were these three elements going on simultaneously there was the music there was the rain from god knows when that was recorded not on a day like today and then there's me from just a few minutes ago [Music] it looks like a very peaceful garden the thing i think would would help it would be a few it would be a few of one of those type of fish so that should give you a little a little bit of an idea so that piece started out with just the music and none of the comments and then people made the comments over time which again that's that's the other sort of nerve-wracking part of this is when you start and you know the music is written to have comments on top of it so it's not that interesting until commentary sort of comes in but um you sort of hope for the best and uh and and hope that you get you know kids who who like catfish i don't know it's a type of fish that are catfish it's very interesting phrasing but um it was you know super cute though so um so that gives you the basic idea i think hopefully um with the retro iphone and everything uh so i'll go through um actually i'm gonna skip some of these things real quick here um that's not that's not that's not okay another audio arp sounds guy this was a piece i did in christchurch new zealand um this was a p they uh a number of years ago they had a huge earthquake in christchurch which pretty much destroyed um a huge portion of the downtown so the point the sort of hope with this piece was to create a piece in the downtown area and let people who are residents go sort of wander through and um you know reminisce in part reminisce about what used to exist in certain places that were you know unfortunately destroyed but then also looked to the future and looked and and sort of talked about well this this was this but now we have an opportunity to rebuild and this could become this and the downtown could transform into this new new sort of thing so the idea of having a location-based piece that allowed people to make these comments to actually create the comments when they're in the sort of you know emotional space of being there was extremely important because i think that that really puts you in the right place to make you know heartfelt earnest authentic comments and then also obviously when other people come by to listen to those comments they exist in the place where the person made them so there's this you know there's this connection to the to the specific landscape to that specific location um on both ends of that uh experience which which i think is one of the ways that um that audio ar can be really really compelling um certainly not the only but one of the uh one of the great ways that uh that you can you know navigate through a space listening to something and you know not having your phone in front of you the whole time uh and uh you know get that added layer of content so um another project tributaries this was a project in newcastle in the uk which was um a uh uh done for the 100th anniversary of world war one the end of world war well sort of the entire sort of the they were doing a bunch of the museums over there were doing a bunch of exhibit exhibitions on world war one and they wanted to try to bring back some voices from back then and bring back some uh some of the experiences that uh people who are live then went through obviously there's not many recordings from back then so we kind of had to reproduce some of that but they did have a lot of texts a lot of diaries a lot of other you know their weather reports and all that kind of stuff so there was ways that we could you know we got a present-day weather weather person to uh to voice the weather reports from 100 years ago and sort of distribute those throughout the city and and kind of create this connection between um this historical moment and the present day and i think again i think audio and audio are in particular is a great i think can be very effective at connecting compressing time if you will connecting different points in time whether it's yesterday and today you know i experienced something in this location now and somebody will have a new experience in this location tomorrow and and there's sort of a constancy of location but uh but a a breadth of time i think can be quite interesting and you know whether it's a day a week a month or 100 years i think there's a lot of ways of kind of jumping into that and and doing some exploring so tributaries that was probably the biggest aspect of that project that um it was new and different for me um so i'll get a little into behind the scenes stuff um not too much but uh so as i as i mentioned before ram ware is a software platform rate is open source i don't know what um i'm not going to get very really hardly at all technical so i'm sure you guys are um but feel free to ask more technical questions if you have them but ramware essentially is a client server the server sits in the cloud all the audio is sort of stored up there database all that kind of stuff and then there are various clients that can talk to that server via the ramware api this is a somewhat old diagram so forgive some of the stuff that's a little outdated but essentially just what i said you know database some uh software on the server that used to do the audio mixing the audio mixing is now work moved to the clients um but the basic point is that there's a database of of of uh audio that is all geotagged and the clients access that data with the knowledge of course of where the client is at any given time how the client's moving where you know all of that information and that that enables the generation of a dynamic stream of audio that is you know unique for any individual any client will have a unique um unique stream of audio whether it's um uh you know coming from the client or being generated locally and it all depends on how they're walking around how fast they're going in some cases what direction they're looking i know you guys are fully aware of all the bose stuff uh the frames and whatnot so that's a sort of high level architecture on the uh this is you know way more than i want to actually go through as i mentioned the mixing is now done on the client side so just the component parts of the audio stream are downloaded and then the client does the mixing which allows me to do um more less latent activities such as you know when you when you look in a different direction you don't want to have to wait for the buffering time to have your audio actually change accordingly so um client-side mixing makes a lot more sense and is something that you know that we've moved to that the ios framework has uh moved to that and uh as has the web the web app framework android is a little behind um budgetary problems you know how it is open source projects when you get somebody when i you know when i get an enthusiastic roundwork developer to uh participate then that will catch up i think but um but yeah so frameworks for ios you can you know put it into any ios app you want and it basically just implements the ram where api allows for the generation of these streams and the communication with the with the server this is a diagram of of the playlist so essentially at any given time there's a dynamic playlist that roundware generates which determines what of those assets will play back for the person and the assets are if you remember the diagram before the small dots like the small you could be in a spot where there are 100 assets you don't want to hear them all at the same time obviously you want to have some kind of capacity for filtering um and whatnot so here's a here's just a sort of high level view of the the filtering that's possible you have all the assets on the top and then you know for a particular project then you can filter them by tags there's basically it's you know sort of very simple metadata kind of um tags that are grouped by categories you know if you're which question are you responding to are you an adult or a child if you might want to if you care about that um you know what's your favorite color i mean you know it could be anything um and then there are location-based filters you know based on where you're located based on how frequently whether you heard the the asset recently and you know if it's been blocked for some reason then there's ordering and prioritization all that kind of stuff so again i don't want to go into huge detail because we don't have a ton of time but um it's important obviously to be able to filter content to be able to create the kind of experiences that you want and um some of this content filtering is sort of automatic and others are you know can be exposed to the to the user based on what you want um so in some cases i might want people to be able to say i want to hear only comments that museum visitors have made versus curatorial comments in other cases i might want to enforce that those are mixed with each other to make some kind of point for example um so here's a few other types of um these are kind of different listening modes if you will that roundware supports the traditional mode is you know when you're within the range of a particular asset you can it becomes part of the playlist so right here these three are in range they're either circles or you can have a polygon for assets as well that's pretty straightforward you know what whatever you're close to you can hear the global listen just makes everything available no matter where you are that can be very effective at times um for certain types of projects um or at certain times within the experience of a project you could switch to global mode and then switch back if you wanted range listening allows for you to um create some different sort of uh approaches of listening to stuff that's at different distances from you so there's a minimum and a maximum distance and you can i've done some experimentation with you know lift your phone up and it point you know it sort of lets you listen to stuff that's farther away from you and you point it downwards it brings the stuff closer so again that's a way that you can determine what content you're listening to and that's that's useful if you create projects that don't have content everywhere and you want people to still be able to listen they can just expand their range and then directional listening again this is what bose frames do but also the imu and your phone does so if you you know if you take your phone when you're listening to the app and you just move it around you can give yourself a certain arc and and listen to the content that in this case these three guys would be available you can combine these in different ways you know you can use the distance minimum and maximum distance with the range you can you know combine these sort of approaches in different ways so um i'm not going to demo the web admin because i don't think that's very exciting but i'm happy to answer questions about that and you know we're coming up on 4 30 here so i thought um i would be it would probably be better for you guys to have me stop blabbing a lot and um perhaps dive into what is most interesting to you guys hopefully something um and uh we can go from there is that that cool that work for you guys i can certainly talk about more projects but i'd love to hear what what questions you guys have and i can i can certainly demonstrate stuff as well um if uh if you want questions comments there's got to be something it's gotta be one of our first game ideas that we haven't ended up pursuing was actually gwen's idea but uh was this like murder mystery where you have to go to different locations to get different clues to hear different clues like the the idea was like you'd go back in time and hear something relevant to whatever case you're trying to solve that sort of thing yeah that's super cool yeah that's that sounds like it would be an application totally yeah no i love that idea um i think uh that the time traveling thing is is as i was mentioning before i think that can be really helpful both you know you can create sounds you know you could have a horse and buggy driving by whatever you know you could have like period type sounds going if you wanted to sort of really make somebody feel like they've gone back in time but then you can also each user yes you would that would probably be a set of tags you would assign different tags to different stories you could have story a story b story c and people would choose which order could be assigned one of those three and then that would segment the content based on based on what they wanted but yes going going to a particular location sort of gathering more information or something like that for the murder mystery um is is definitely something that that uh that ramware could do it's actually funny one um actually that escapes piece this case piece that i showed the video of people um started making up their own sort of games with it um and one one person sort of did kind of what you're saying it wasn't a murder mystery by any stretch but they went to sort of a random place and said you know go to the corner of the parking lot to receive further instructions or something like that and then they went over there and then they proceeded to talk about the further instruction and then they you know and it it was kind of a game within within the piece which was you know not i didn't have anything to do with that it was it was uh you know somebody taking their own initiative to sort of figure that out which um which i thought was great other other whole bunch of kids talked about zombies and stuff they were like i'm hiding beneath this sculpture the zombies are coming and then this this this story of zombies actually continued over over months um and with different people like different you know like all the zombie fans kind of came together and there was this one area of the sculpture park that kind of turned into zombieland and um and then at the end you'll be very glad to know the helicopters came and landed and everybody was saved so nobody died but um i mean obviously the zombies are sort of i don't know what they are i'm not going to make a comment i'm not gonna come down one side of that argument but um but it's just it's one of those things where when you open up your um work to this kind of um potential interaction with it you sometimes get really really neat kind of takes on it that you uh you know i mean i certainly didn't think to create some kind of zombie scenario um but it's wonderful and it's it's uh you know it kind of adds to the um to me it adds to the excitement of of sort of creating these pieces i i am you know my work is very you know i guess maybe it's a fear of ever of ever having something be done but it's there's something about um knowing that um things can you know can be sent out into the world and kind of take on uh their own uh their own life if you will much like a zombie no um and uh and uh and and kind of you know get this level of creativity or level of input that i couldn't do on my own you know i i really get tons of inspiration from people who come in and uh and do stuff so yeah i think that that idea is certainly one that would be um well suited for round where um you know i have dreams of ramware becoming a a sas at some point it is not yet it is kind of requires a little more technical um uh involvement to set up the server and then get the client talking so it's not you know all the code is there it's just a matter of um kind of going through those those uh steps um there's a demo roundware app that can be used for sort of testing stuff um which is pretty handy if you just want to go out and make com you know make recordings in certain locations and then sort of see how it how it works that can be done that's uh the ios version is the uh a good place to do that but i think that idea would definitely be um you know roundwork could certainly help at least get the basics going on that question yes you know you show up a little bit of the scape's user interface the end user what you can record and just pretty much just drop a sound that has some radius but then you also showed all this other capacity of direction and like minimum maximum radius is like how much of that do you see being useful to expose to kind of like the most fundamental part of the the most difficult part of the thing yeah and how much of it is just like too complex and he is off-putting yeah i to date i've exposed very little of it um and that's a really good question it is something i think a lot about because i do want it to remain simple for the end user that's why you know there's like two buttons it's listen or speak and that's kind of obviously within speak there's a few steps that you need to do to what questions you want to respond to and whatnot but it's fairly fairly straightforward in that regard and exposing more does cause some problems sometimes so i've experimented a little bit with adding additional filtering capacity but having it sort of semi-hidden not really hidden but like it isn't it's like behind a button instead of you know you hit a button then you open up oh here are some filters i can do and the filters have a default state which is basically listen to everything and then you can you can kind of specify from there become a super user of sorts if you if you want the user choosing their listening experience based on eliminating things that you don't want to hear in this case yes yes i like to start with the broad set if you start with nothing then people are going to get confused because they won't you know so kind of starting broad and then narrowing down um as far as the other um the other sort of listening modes and everything goes that's that's more of a kind of um you know set up the project that way one way or the other and and if you're seeding the project with a certain amount of content you know some like the skates piece was seated with basically no content i allowed people to come in and do whatever they wanted the tributaries piece i created a lot of content myself and i put it out there um and uh you know the weather reports and diary entries and you know uh you know machinists shop logs i mean stuff like that that was historical material i could create that and then i could place that where i wanted and i could create the shapes and i could do anything like that um the thing that i think actually the the the the scenario in which i think exposing more to an end user that would be most appropriate would be something that i'm sort of i'm working on now which is um enabling the app to be more of uh to be a more to be a different type of creation tool so so so the person creating the experience would be able to go and say you know create a recording and then walk around a closed polygon that they want that recording to exist in for example um and and do sort of other other manipulations be able to move stuff around to be able to adjust so you're really in the actual physical location that the piece is going to be experienced when you're creating it because i think that's that's a far cry from sitting at your computer at a google map and drawing something like i think this is where it is but i'm not really sure and i don't really know if this is going to work so you do that i always find myself doing that and then going out and listening and then going back in and then going it's like this back and forth can be very frustrating so so i think the exposure of more in-depth sort of i don't know if you want to call them editing tools or just or just sort of experience modifying tools i think i think makes more sense for sort of more of the creator side of things of course end users are creators to a certain extent because they're adding their own or they have the option of having their own content but um but it's a good question because it's i think it's a constant balance and maybe when people get more familiar with the overall idea of audio ar then and then we'll be able to push a little farther in that direction but right now i my experience has been that people when explaining it to them it was wonderful when pokemon go came out for me because i could even though this really isn't pokemon go there is an element of hey it's like pokemon go except instead of finding pokemon you're going around and you're finding little pieces of audio and those are playing and it's you know and you can add right pokemon you can't you can't spawn your own um pokemon really at this point so um but that was a nice thing to sort of give me a context within which to talk about this stuff um despite the obvious differences but but perhaps when ar generally and maybe audio ar gets to the point where everybody kind of understands that it exists and there are possibilities on a base level then you know we'll be able to push things farther but yeah i think there's a ton of stuff with the directional and that's why you know that the uh you know bose imu enabled headphones or frames and you know i know that there are other other manufacturers doing stuff as well and i think it's going to be really exciting uh for sure or i mean airpods or you know i think they've got imus or some semi-imu the new airpod pros have something they certainly have accelerometer in them so i don't know well yes right because you can test them yeah cause you could tap them and i i don't know what they're gonna you know what kind of apis are going to open up on that but it seems like people are thinking about that stuff wearables hearables that i don't know if i'm too into that phrase but it's uh it seems to be what people are using somewhat now but but yeah for me the more hardware that companies put out that i can experiment with the better it's great because i certainly can't build hardware on my you know i tried strapping an imu to my you know like is a major failure um but uh i am i'm not really a hardware guy so more questions more comments more things you would like me to show i can go to a uh just a yeah take question what is the precision where you uh delineate where where the sound will play is it how exact is it well ramware is is what i would call location um determinate it's sort of it's agnostic to what where it gets its location information from so i you note i showed you mainly outdoor projects because we all know the problems with gps indoors we also know the gps is the only sort of ubiquitous location you know essentially free location technology that exists right now so so so we have gps error bars right now which inside a building are you know way too big to make it useful i have done a lot of experimentation with other you know interior you know systems whether they be beacon based or you know magnetic field based or visual positioning systems but at the end of the day roundware just takes at this point roundwear just takes two coordinates it roundmer doesn't have a z axis at this point but adding that isn't that isn't a huge huge leap but i haven't bothered to add it there's no point until i can get something that actually gives me useful z data to uh to actually base something on but it would be super cool if you know you get uwb systems which are like you know ultra wideband it's like so it's like you can get like millimeter level and then you could like raise your phone up and you could stand up and sit down and get whole new levels of of ambient sound i mean it could be totally amazing but unfortunately you know roundware depends on the systems that are built by large companies and governments that i don't have any control over your experience with implementing this museum park walk or art exhibit did it feel like it connected spatially and conceptually so um it felt very connected in the demo or he must have been talking about this particular thing uh but was that the experience when you when you implemented it as well that yeah people really can't see what object they're talking about or does it sort of pop up in the wrong place sometimes it would be unfair of me to say that it was always fine because it certainly wasn't sometimes gps gets wacky and of course the the error bars get multiplied because you you make a recording and there's an error bar associated with that and then you're listening to the recording there's another error bar associated sometimes those can destructively interact which is great other times not so certainly there are times when weird stuff happens i think generally you know i was pretty aware of what the error bars were when i was when i was creating the experience so i sort of could set things up to a certain extent um understanding uh understanding that and kind of guiding the experience such that it would work relatively well with that and i will also say that as an artist i kind of have it a little bit easier because people are a little more willing to be um you know with aesthetic experiences that are a little more willing to be uh sort of forgiving about stuff like that so if you're talking about something that is maybe over there you'd be like oh the artist wanted me to walk over there and you know i mean if you're if you're in a a generous kind of interpretive mode that can work i also do i will say that a lot of times little gps hiccups can actually be really interesting because um if you're walking by and somebody's like this happened to me in the escapes project just to use that as an example i was walking by and somebody was like i'm standing inside this ring of trees and i'm looking up towards the sky and it's really it reminds me of you know my grandfather's house up in maine where i had this similar situation i was like i'm not in the ring of trees right now i'm just walking out in this field so it made me look around and i was like oh you know 30 feet that direction was in fact something that looked like a ring of trees and it got me to go into it whereas so i think there is something really nice about sort of this audio are generally being able to kind of pull you in different directions or guide you or or encourage or whatever words you want but um to get you to explore a little more by um you know by by kind of giving you information that is pertinent to a spot when you're not quite at that spot yet um for example and i i know that might seem like an excuse for gps you know errors but sometimes they can be nice of course sometimes they can be inappropriate and bad but yeah that's you know when you when you open yourself up to you gotta open yourself up to the the goods and the bads but um but yeah i'm just super excited for the day when positioning interior positioning is accurate enough to um to really do something where you know you can place sound objects walk around them and and you know understand uh have them feel like they've been spatialized and and really kind of get a much more true to reality kind of experience but you guys probably know more about how close we are to that than i do but my uh from from where i sit exciting stuff is happening but um it's still i mean the big guys are all working on it right i mean spatial computing is like everybody wants their spatial engine and amazon is doing their thing and you know niantic of course is well into it magic leap has got the whatever i don't know all the the magic verse is that what they call it i never know what magic neither do i neither do i but it seems like they're spending a lot of money and doing they got good people doing stuff so yes they got neil stevenson doing stuff so i mean they'll be they got some content possibilities so um so yeah i mean it's it's super exciting but we're definitely dependent on the you know systems that that are out of out of my control out of most people's control but i think if you work within those constraints you can still create some pretty cool stuff so other questions yes for my project i wanted to make something that people could like potentially contribute poetry to like a location so i think this might be really helpful with that yeah yeah that's that's very cool yeah poetry um that i think poetry can have some wonderful obviously a lot of the the language can be very sort of you know physically as you could put you in a place and it would be nice to actually be in that place too yeah i did i did a piece it's no longer available because of the inexorable you know progress of new ios versions and whatnot but i i did a piece in harvard yard that used as the source material it used all of the not all but used a bunch of audio that um the woodbury poetry room at harvard had they've had a poetry series for like 100 years of poets coming in and reading their works so they had recordings of like robert frost and william carlos williams and ezra pound i mean people like really old amazing recordings of these actual poets reading their work um and then they have more contemporary poetry uh poets as well and i took those and i kind of sliced them up line by line and then distributed them around in different places so i had like all the first lines of the poem in a certain area so if you went to this area you could hear first line after first line after first line then you could go somewhere else and hear all the last lines it was very made it very abstract and the poets probably would be very upset with me if they found out about this but um it was you know it was a it was a remixing of of that um and it you know a spatial remixing which um which was pretty fun to do but i loved your idea so were you thinking of having people read their own poetry or or something along those lines so so write upon that related to a certain location and then go to that location and kind of recite it sort of thing or i could imagine if you want to share any more thoughts yeah oh like almost almost like like improv or something like like all this oh yeah that could be really that could be really great yeah so so yeah the question would be i mean the you know in a round more context that would be you know open the app press speak recite it you know make up a poem right now and then you just press record and you record it and then upload it to the cloud and then whenever somebody else came by they could uh they could um you know they could hear that and then you'd have the ability of course as an administrator in that case to listen and move the recording or make the area bigger or you know those sorts of things if you if you were to to want to do that but that could be a a wonderful way of kind of it would be a nice way to traverse the city right you kind of wander through a city and you hear residents improvised poetry at uh you know sort of you know at crosswalks or wherever like that's a that's a cool idea and you had a lot of uh you you've been working a lot on prompts yeah like get people who will not necessarily be able to compose a whole pool up on the fly to be able to stop like that's really smart to think about that because that's really hard like even if i mean poetry is like particularly hard but even for me being like to say anything is sometimes hard so i always think about the what questions i i ask and i usually have some question that's like so broad that it's you know that anybody could do it like you know just tell a story about your day i don't know look at the you know look at the ground and tell me what it reminds you i mean i don't know i'm just sorry i'm making bad examples right now but something that's very broad to uh i mean literally it could just be share something you know say something and then as i go down the list i sort of get get into more details because i find that some people freak out when they have a super broad question they're like uh what do you mean something what does that mean you always say something they are like much happier being like given instructions like look at a tree near you and describe it then they're like okay i can do that there's a tree record you know the tree is quite tall and it's got lots of branches and leaves right now are green but they're starting to turn yellow that some people are much more comfortable with that um so in a poetry context that could be really interesting like what are some prompts that you've come up with thus far that you've because that's a big challenge that's a challenge yeah like right now i've just been thinking about like sounds so um it wasn't necessarily like a poem about sound but ask like what sounds can you hear right now and like like looking really good too and for like one test i asked them to write a poem about a different subject based on the sounds they heard you mean like a poem about sound that isn't about what that sound is actually like what it reminds you is that what you're saying sorry i'm okay like the theme is not necessarily the sound itself right yeah so what it reminds you of or what it you know maybe points towards or something like that yeah yeah no that's great to use sound as a poetry is such a you know in my mind poetry is meant to be red it's meant to be you know it's got all the rhythms and everything really come out when you when it when it gets into the the audio from just the written page i mean the written page can be really interesting too but it's uh um that's uh yeah but it's hard it's hard to just make up a poem and it must rhyme uh yes don't yeah a little bit more fruitless yes that's good that's good yes free form is uh i'm always impressed with like uh like free form like you know like rap battles and stuff like that people like i mean like obviously eminem is like you know the eight mile thing it's just like oh my gosh how do they come up with this stuff like and i know a lot of it is i know there are strategies and techniques that kind of make it more doable but it's um my brain doesn't work that way i can't just although it sounds like it right now that i can just talk forever but uh i i don't always uh don't always uh not always able to you right and right away your sdks yes yeah right oh that reminds me i i should also mention that um i have uh started um a new website called audioar.org um which oh great great great so so so the site is in its infancy right now i started it with a couple colleagues a couple journalists and fran who came here a few months ago friend panetta a month ago to uh talk to you guys british probably remember because her very perfect british accent which is um uh [Music] so she's been doing audio air stuff obviously as you know for a while too so we started this site just to kind of try to collab try to collect different practitioners in the area and try to it felt like something was starting companies obviously bose and you know amazon and apple and others are getting interested and and we thought it would be a nice place to sort of um let practitioners and um and technologists and academics and other people kind of share um share their experiences things that work things that didn't work and just sort of uh give and you know let people know when projects are happening and whatnot so we're we're kind of ramping up on that and um philip has agreed to be interviewed for that at some point whenever i get myself um in gear i promise i will and i'm looking forward to that but yeah i know you're like please forget um but uh but yeah any projects you guys do and um you know we'd love to uh uh you know think about getting getting stuff up there because i do think it's uh um you know i think this class is evidence of the fact that it is becoming a real thing um which is which is really exciting so i'm super excited to hear what to hear to experience what you guys uh choose to do with the um with uh you know with the class is it is there a project there are project is a project sort of a deliverable for the class is that right so um there will be a presentation we need to discuss that before you leave when it's going to happen because there are possible changes there but there will be a presentation at the end of the class uh francesca will hopefully attend some people from both will attend if you can come that way oh cool yeah we'll spend the next two weeks thinking through how to disseminate our insights from the class yeah so there will be something online i dare say but we will try to decide that together exactly what form it will take so there will be something we will leave a trace well to the extent possible we'd love to amplify that message on on audiohour.org if it's you know possible but we can we can obviously talk about that but but yeah so if you guys have any ideas about things to include or practitioners or whatever please feel free to reach out because we are super open and want to learn from each other so i just so we're running out of time but i'm very curious about one aspect of the system [Music] is is it dynamic in any way that people who make contribution could actually change parameters or aspects of the experience for other people beyond just the sound basically so an example would be like if there's a lot of people leaving a i don't know why i call them decided to call themselves races but if a lot of people leave sound traces in the same location can the system detect detect that and decide to play sort of a more multi-layered ambient uh soundtrack in that area or something is there's some way of being aware of what's being input that's a that's a really interesting area to think about um i think the simple answer right now is is no in the sense that ram where doesn't um i mean ram where you can choose how many simultaneous voices can play and therefore if you say well three simultaneous voices can play if three are available then in that sense if there's a location where people add more then that will be a location where potentially three will most likely if there's a hundred in this spot you'll probably have three playing altogether at any given time that's not the music layer or whatever that's just the the the so whereas if you have a spot that only has one available even though three could potentially play at the same time only one will ever actually play because there's only one available so that's a very very rudimentary aspect of what you're talking about and that's the only sort of at least if i'm understanding you correctly that's that's the only thing that roundworth sort of does currently but i love this idea i hadn't thought about this i love this idea of more more activity kind of the heat map heats up and something changes with we're always going to think about this is there a way i can program this as a participant you know like because so so even there might be a way even if you don't know it uh you know injecting codes somewhere by yeah who knows uh you know uh overloading the database or something but uh yeah it's it's just i think there's some potential there somewhere and like what it could be very advanced like the words you say could actually be understood by the system and it could change things but yeah i've been then yeah then you're halfway to the matrix i've i don't know it's super i've been thinking about you know the first step to that obviously is sort of getting some kind of accurate translation you know or transcription service you know automated so that and then and then you could get to the point where you could search for only play me only play me commentary that's that uses the word love and then you could walk around and get this whole experience with just um and then you do sentiment analysis and then it's like only give me the the comments that feel love even though they don't say love but they feel it um so i think those are super exciting areas to look into i just want to know what the zombie people would do with something like that yeah where they can sort of affect the system itself or the zombies themselves yeah i mean this is this is very very very important but no i i i love that idea um and yeah so uh it's open source right time i think this was really helpful actually we have one team who are out today because of medical emergency or basically doing the kind of pokemon go go music oh cool so uh we'll definitely have them look at your system and they great i would guess will wanna use it well feel free to reach out i can you know again depending on the level of technical skill and time involved or time available it's um you know i can recommend different approaches to to things but all the code is there the server's written in python using django obviously ios is swift and um the web stuff is is uh very nascent right now but it is um it's all you know javascript sort of modern day javascript stuff so to the extent yes but do you is there a way to like procedurally generate a location for a sound fight or is it totally tight um can you describe what like what scenario you would like what what input would you be taking to so pokemon go so this is has to do with the pokemon go but like audio music um project that chuck is doing um yeah uh checking chairs are doing so um in pokemon go you walk around pokemon like spawn can audio also spawn probability based on like a probability yeah um that is i mean it that would be [Music] the kind of thing that um would be a relatively small modification for a particular project i haven't done that the other thing that i haven't done that i'm very excited about is is that they is moving after after their spawn then they move you you assign a path and a time whatever and they loop around or or so then you can just sit in one place and you can let stuff flow over you or you can move around and there's this sort of double double um interaction in that sense well it's a single interaction but you know what i mean double um motion so um that is also really um that kind of dynamism i think is very exciting but yeah i like the idea of of spawning um sort of popping up based on you know maybe your previous experience within the app or or something like that but but yeah i think the framework is there that that you can you know i've been just adding stuff as we go you know semi-organic code base in that sense which is sometimes problematic but um it's fairly clean so check it out if you know python you can jump in and have a look all right thank you guys so much thank you you