super mario spacetime organ (illucia & soundplane)

March 25th, 2013 § 0 comments

A critical inquiry into Super Mario Brothers.

In this video, I directly manipulate the RAM state of Nintendo’s Super Mario Brothers to transform it from a game into a strange instrument.

First, I play the game as it is traditionally played.. but I have access to the game’s memory, so I change Mario’s Y position using the Madrona Labs Soundplane (a surface that sends data to the computer about where it is being touched). This is how I hover Mario during the playthrough.

Also, before I start playing, I flip a switch on illucia that I assigned to trigger recording — not video, but actually recording the entire memory state of the NES for each game frame.

Think about it – Mario’s universe is held in RAM, which the NES uses to draw his world for each frame of the game. By recording the entire state of the NES memory for every frame, I’m able to go back to any moment in Mario’s life.

So then I use the X-axis of the Soundplane to sweep through the timeline of Mario’s universe.

Not only that, but the Soundplane is multitouch, so I use a second finger to specify start and endpoints in a playback loop. Technically, this is similar to the way samplers and granular synths work in audio.. but with the entire memory state of the NES. Conceptually, it is like Super Mario meets Groundhog Day. Mario’s universe computer/time machine gets caught in hellish loops.

Then I start using illucia to send alien data into various other places in Mario’s universe, which makes for all sorts of audiovisual insanity amidst the spacetime loops. This is sort of like circuit bending, but in a protected sandbox – at any point I can revert back to the clean recording of RAM states (aka moments in Mario’s universe).

I then try to go back to “playing” the game, watching Mario navigate a melting world of glitched-out ephemera. I then push things into full on glitch insanity. I use a pair of rubberband mallets on the Soundplane to jump around in Mario’s universe while leaving illucia to send a heavy stream of alien data into Mario’s RAM state. I eventually (accidentally/luckily) land at a place that triggers the game over music, and end the take.


I don’t condone piracy or the distribution of Nintendo’s IP. I’m posting this video as part of a critical-cultural project.

(Further – although the emulator tools I used to make this performance are fully legal, I’m not distributing any other materials related to making these interventions into NES games, so please don’t ask.)

The video is shown with the intention of stimulating critical thought about the role of software in our lives.

My hope is to get people to think more about the everyday software in their lives. By showing that there are other ways one could interact with this well-known cultural artifact, hopefully I can inspire others to consider: Who decides how and what we see in a computer program? What is hidden? What sorts of strange other things lurk beneath the surface of our trained expectations?
What if we lived in a culture that deeply embraced the expressive possibilities of computer programs? What if we encouraged curiosity about the interior of the systems that compose game worlds? Is that curiosity and critical inquiry itself not a form of play?
Indeed there are platforms that embrace this sort of approach, but many don’t. In fact, many close it off. What if we prioritized this sort of inquiry more? Might we find new frontiers for communication or even human knowledge and representation? What is a videogame, and how can it relate to all this?


Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

What's this?

You are currently reading super mario spacetime organ (illucia & soundplane) at paper kettle blog.