Twitch Puppetry: A Presentation

Introduction, Inspirations, and Predecessors

 As someone who plays a lot of video-games it's fair to say that I've encountered and played as hundreds of characters, however in terms of any philosophical conversation that ever crops up from video games the most common term I tend to get involved in is that of "Player Agency" in games and "Projected Autonomy". These terms refer to the idea of the level of control a player has over an in-game avatar and the level in which the player projects his/her own personality and habits via the character themselves. Now games are inherently limited, be it by game code, the size of the environment or the fact that game control is normally from buttons and keys therefore never allowing a full sense of freedom and expression.

 I mention this limitation, not as a statement to say that I can break it, but more as an accepted weakness, I say this now in hindsight of the criticism that the base actions of this piece are inherently simple, while this will be explained more later it needs to be said early on that the focus of this is not the idea of free control for the participants, but more the relationship between player and character.

 The idea of human control has been shown before in works by Stelarc, his consistent talks about the human body in relation to technology is fascinating, however with works like ping body, he often shows the ideas of humans both fully in control of technology, and technology in control of the human; it was this video here that, in particular caught my attention: 

 The idea of human control is, personally a fascinating one, however a single person controlling someone for the most part can tend to be restricted by a sense of consequence caused by identity, and morality which would cause someone to be more restrained in their decisions. The first step to eliminating this issue is by making them know a human is being controlled, but void it of its identity and responsibilities, so that this "shell" can have the participant project their wishes unto it. However for ease of use for all involved, the interface and the actions would have to be simple and understandable, which leads to the problem that simple actions with no particular goals is not perhaps something that a participant would find compelling, so an obstacle or challenge would need to be made to insight interest so that the goal that the viewer has made for themselves can become more engaging and, for the "puppet" more cathartic, this is where twitch plays pokemon comes in:

 Twitch plays pokemon was an experimental livestream that gave viewers full control of a game of pokemon by typing directions in a chat, the problem was that there were hundreds of them at once giving constantly  contradictory information to the game. It was cathartic, funny, and utterly captivating. The fact was that there were hours of random movement, the moment the chat were not unified on specific goals everything would reach a pointless standstill, but the game would just keep on playing regardless, the character would be a slave to the masses, so I decided to harness that idea with a simpler interface, but one without specific goals, and more importantly, one where the controlled subject was real, not digital...

Aim

 Twitch puppetry is an art piece that uses the simplified interface of "twitch plays pokemon" but in a small area that contains an assortment of items representing both responsibilities and distractions. This would then control a faceless human (in a morph suit) to do the actions of the chat by following a screen interface, this allows the chat that same sense of catharsis, but now representative of a real life human...

The Real Aim

 The actual aim of this piece is not necessarily for the chat, nor those watching, or contributing to the piece. It is a personal journey at the hands of a majority and, in a a way, a sense of willingness to submit. The wish of this piece is to personally lose my identity, sense of direction, and any real semblance of control, it is, by definition, masochistic. It's a curious idea that I want to personally experience due to the fact that, while one can complain about having no control in their lives, it's rather doubtful that they have actually experienced the sense of true submission, an experience of genuine masochism at the hands of an anonymous crowd. It's slightly embarrassing to admit but part of me has always been curious of that ideology and the feelings it maintains. A true sense of randomness and dictated movement that, while initially voluntary should quickly become second nature after the initial phase of getting used to the feeling. The journey is therefore not just about the journey of movement that will be creating, but more a personal psychological one, a journey into the world of sadomasochism outside of the sexuality that the term is often associated with.

The Process

Using an open source code from the "Twitch plays Pokemon" stream, it is possible to make certain comments in a twitch-stream chat correlate to keyboard presses, by using basic game software with a similar top down interface to Pokemon (RPGMAKER) a simple grid can be created and a character in the center can be controlled. This game is sent on screens surrounding the "puppet" who is on an identical grid with identical item placement and needs to replicate the on-screen characters movements exactly, a stream of him is what the audience sees and believes they are controlling. The twitch stream will be public and, while the main advertisement with links will be at the site of the performance, it is allowable (and in many ways beneficial) that anyone online can potentially tune in and join the system.

[Insert scans here]

What needs to be decided/Problems

-The room is still unknown, it needs to facilitate projectors (preferably 2), at least have a square of 7 metres (for a 7x7 grid minimum) with room for cameras, an internet connection, and hopefully some space for live viewing.

-The items need to be specifically decided, however on the work side the most likely things will be paper filing, cleaning equipments, and office supplies (such as paper shredders etc.); The play side will most likely contain musical instruments, childrens toys, and gadgets (such as stereos).

-Stream delay is inevitable, meaning actions may not seem instantaneous to the viewers, especially those at the event also using the twitch-stream as a control interface.

-The performance also needs a decided time to run, which will most likely be an afternoon time slot of a few hours.

Objective Timeline: (updated if objectives are completed)

-To have specific objects and functions decided, along with the grid layout within the next week.

-To have the room decided, a long with the corresponding safety forms dependent on the room; as well as the rpgmaker grid actually made withing 2 weeks.

-To have the chat engine fully functioning with the game for full twitch control within 3 weeks.

-The rest of the time is to iron out the kinks and prepare for the performance!

Maps and Journeys....A Journey of Epic Proportions!

when tasked to go on a journey and present the narrative of it in accordance to a map-route of your choosing, it is fair to say that one can be pretty stumped. I say this because recently that exact thing happened to me in a recent workshop.

 The ting about this year is that now the realms of interactive art can be included which immediately leads me into the realm of video games! Now I'm not here to debate whether games are art or not for a single reason: They are. Video games are an incredible, exciting, and artistic medium through interactivity and gameplay. The reason I stress this is because one of the things games can do is make the player feel in a state of "unrealism". A different state of logic and action where individual actions, no matter how absurd in isolation can make coherent sense in its own world of rules etc. But I love the joke of applying video games to real life, the effects of reenactment can help highlight and isolate the nonsensical nature of certain aspects of many games and yet also celebrate the feeling that games can give to a player because of said absurdity.  So I decided to use an odd base inspiration for this mini-segment of the unit, I used comedy troupe Mega64:

Now I actually have cosplayed as the person from journey before for an expo, and as tempting as it is to get my costume back out and basically just copy them it is a) pretty lame to just steal ideas and b) doesn't incorporate a map at all. However I recently managed to get the classic game "shadow of the colossus" for my housemates Playstation 3, and while looking at a map for collectibles I noticed something rather interesting (or extremely loosely coincidental and still a bit of a stretch when you think about it for 2 seconds!). However I figured it'd be a fun way to look at the project from a slightly different angle, so I made this as a chronicle of my journey!:

Reflections: Photography and Such

So when it came to the themes of reflections I looked at the ideas of memories in digital spaces, in recent times the idea of virtual reality has really taken off, but it's not quite prolific enough nor publicly available to make full works in a 360 degree virtual space. However panoramic camera technologies are something that interested me because it allows some of that wonder of a virtual space by almost looking through a movable window, it's not full immersion but it allows some of the ability to look around a fixed point in space. This does however mean I could focus on making explorable pictures as opposed to fully fledged 3d spaces, thus when looking around I found a piece of consumer technology that I could export panoramic pictures from and photoshop them to create these odd spaces; that being the Playstation Vita.


 The Playstation vita has a mode that allows basic panoramic photographs to be taken at 1280x960 pixels, by taking several photographs from a program in the vita it automatically creates the panoramic space and uses gyro-meters to allow the person to track around with these, and thus I simply thought "How do I use this in unconventional ways to confuse it into making more obscure photographs?" Originally I simply used DSLRs and experimented with photoshop to create odd and obscure landscapes and mazes to represent how memory can be obscured and almost maze-like.

I will quickly say that in a 3d space this looks rather different, but what I did find was that in the first photograph people became confused and lost in the space due to the odd sunset landscapes. The second picture based on the idea of monotony interested people while the third picture just confused (arguably as intended but to a point of incomprehension". However in the idea of a reflection I had hidden a cuddly toy from my youth in the 3rd picture, and the initial confusion was stopped by the satisfaction of seeing something a little bit hidden. Combining the idea of an easily explorable but obscure space with an extra point one needs to look around for became my new objective:

This first picture was created by using the standard panoramic software of my room before I moved out, by obscuring and blurring it i attempted to create the idea of a memory being slowly lost. The Computer screen asks "Is it how you remember?", the hidden secret in this one.

I always wondered whether people would remember what I looked like as much without my hat that I usually wear, so I took it to the extreme and followed the camera on it's panoramic cycle holding my hat in front of it thus confusing the software making a 3d sphere out of hats, the hidden part of this is a picture of my face wearing the hat with the face scratched out.

This one was created by confusing the camera further, this time I induced an error in the panoramic cycle  forcing it to re-calibrate it's position, while it was calibrating I ran to a different room and set up a shot, this happened 37 times meaning that this landscape is an obscured version of every part of my old house to represent memory blurring together. all screens and technology contains text on it so as the person looks around they see that the piece is every part of the house on a certain date.

This one was based on my internet history and that classic idea that someone is always watching you, the irony being that it's the computer, not a person, thus I took all my most used sites for a week and displaced them inside an eye with a smaller eye within the pupil. once started the small picture people see is the small eye looking at you, often unaware that the whole picture is in fact a larger eye forcing people to "see the bigger picture".

Finally I modified my previous shot on monotony, in the panoramic mode the person must look up and will see a message saying "it loops and it loops and it loops and....". Using this still conveys the idea of monotony but allows that extra satisfaction of finding the secret within.

Reflections: The Making of Digital Distances

Okay, so it's fair to say I've been trying to write this thing for a few days now, but honestly, a lot of the time I've no idea what to say, when I plan I just plan, when I film I just film, and the footprints that I leave behind tend to be the evidence that I was working there, I've never been one to show my workings, be it maths, science, performing arts, it's never been my style, is it an excuse, hell no, but it's almost 3am and 10 hours ago I retrieved the wonderful news that my granddad was as dead as a fucking doorstop; so fuck it, I'm gonna write it in my style now as a trail of thought, if you don't like that, hard luck, it's how I'm gonna do it and you just have to mark me down for it, so hoo-fucking-ra marines, this is how we made a film!

okay, so starting simply, inspirations: well it's a 3 part film, was always seen as 3 parts and was always imagined as 3 parts, if you want to know what those parts are look back to an earlier post, that explains it better than i will now in this state, any-who...

 act 1, make it melancholic, so I based it on the animation aku no hana, a slow brooding drama from Japan, that despite being filmed around and outdoors, has a sense of pace and atmosphere that is as brooding and melancholic as it gets:

So challenge 1, make that....indoors, so more close-ups, more claustrophobia, and a different theme, in this case the blue flash of a pc, now I will admit that when filming with some lights and a shit-tonne of blue filters on the lights things were arguably a bit too low key for some, and part of the significance of the imagery is personal, and to some that is a problem, the whole use of coke cans everywhere is about binging during depression, and to some people this may not come across, and that is one thing I must first address:

The piece was always personal and sentimental, and I tend to feel that going into something with personal wants and feelings is frowned upon somewhat, y'know what, i get why, personal reasons and wants is no excuse for poor film-making and confusing plot, an inability to communicate a message is just poor conveyance and personal, selfish, indulgent films are of such a thing. the reason i say this is because it stops me from having to use the personal reasons excuse, if someone does not understand something in the film that is my fault as a film-maker, and not validated by the "i was being personal" card, so basically call me out on hypocrisy if you will, so that i may actually have some argument that i am unable to bullshit my way out of.

anyway part 2 and 3 is where we introduced the digital aspect, in terms of the effect my main source of inspiration came from a japanese (naturally) indie tv drama, Mirai Nikki Another World which follows similar themes of digitalism, although in a more literal sense than the metaphorical one employed in this work, however if you happen to see, for example....cubes floating around, it's fair to say that it may or may not be a slight homage to a show i adore: 

Editing and pacing however was where the most debates were had, it's fair to say that the order of a lot of things switched around during post-production as we made it a priority to secure our effects first so that the film could be easily reordered to suit the needs of the group, that and because on certain parts the idea was to break the scenes to create a sense of catharsis. A personal inspiration in which we based certain scenes on (specifically a scene involving a loop of myself running towards a shadow) was the american trailer by funimation for mnemosyne, a rather insane horror show and while we definitely held back compared to the trailer, the original DNA of the midway editing has its roots in that trailer:

So the question really is "did I achieve my ambitions for the film?"; honestly, no, because it became more than my own original ideas the moment I had a team, the fact is that interpretation is an important part of any art, and to present these inspirations to the group shows different influences and details than i could find on my own, the ambitions and ideas evolved over time and I can only be proud of that. on set it's true that lighting and contrasts occasionally suffered due to a shortsighted focus to the compositions and the timings and an initially rigid shoot schedule; the film was arguably planned too much at points creating a rigidity in the shooting schedule and an occasional attitude of "it might be salvageable in the edit". An attitude that we initially wanted to avoid but with the clock ticking became an inevitability. However I can at least say I saw people come out there shell during this film and, any experimental side to the film became an interesting hurdle for all of us but one that allowed us a sense of odd vision. The reason i cited Child of Eden and End of Evangelion as influences was because they gave me a vision of a digital paradise gone wrong and a feeling of odd dread and tension through seeming randomness together. I wanted to emulate some of that spark they created within me the first time I saw them and, in the course of making this, I'd like to think I found some of that spark again.

Also if you're reading this I'm sure you have a lot to get through today, so here's a lovely song from end of evangelion to keep you going (and yes it's my favourite film): 

Another Thing with Stuff, this Time about Time-Lapses

When confronted with Time-Art I was made to watch the film Samsara to which I realised something, Time-Lapses are absolutely incredible, and so pretty much instantly I needed a reason to do one. A few days later I was reminded by my optician that I needed to have another check up for my eyes, to which I became rather curious about how my eyes have, and will develop over the years.

 I decided that I personally wanted to represent birth, life, and death, through 3 simple metaphors being sunrise (in a place for children to play), midday (showing monotonous work), and sunset (being in a calming field/farm. The time-lapses themselves were somewhat simple however the devil is in the detail and I will say that exposure and focus are nightmares to control as I especially learned during the sunset scene, and while I could make the excuse that the slight lack of focus is for the eyesight problems of old age, it really was a dumb mistake in filming

in terms of eyesight problems presented the retina was represented by a reverse keyed lens flare to create a small iris effect, loss of focus was made of Gaussian blurs with some masking for distance, cataracts was a reversed  light ray made to be black and blink effects were small animations. The piece was mostly about using keyframes to set up slow blur increases and so on, and while I do hope to use more complex systems in the future, finding methods to reverse light rays and key them out became easily the most complex part of the project as they weren't conventional techniques and therefore not covered by lynda.com (for reference the technique is to make a green screen with light rays and then key and reverse the effects separately).

 While I am reasonably happy with the end project for each I worry that it lacks any effective use of the 3 screens and instead adopts a more simplistic approach, the blinking arguably also looks unnatural due to it being limited to a 25fps limit for the animation that is a full 7 frames long. In the end what is achieved is some interesting time lapses and ideas, but I'd argue that it lacks the complexity of idea to make it stand-out in any way.

A Load of Stuff About Sound, Hoorah (or Hooray)

When creating my sound piece, I was given this particular picture of Louise Borgeois' Maman: 
Maman is a large spider sculpture that is supposed to be representative of the sculptor's mother, who was a weaver. The large metal sculpture with small eggs inside the main compartment is based on that of a connection, a deep sentimentality and ultimately a sense of solace in the abstract. I absolutely fucking hate spiders so to say my first reaction was that of distaste and a slight amount of fear is no exaggeration. However to me from this picture what I began to see was not simply a large spider, but a collective connection between people, like thoughts bubbling into a cloud of data, yet that spider was there, it bridged the connections between people and yet it was scary to me, and no one seemed to be noticing in the picture; personally I wanted my sound project to reflect that feeling.

 The first inspiration for the spider itself it reminded me of an image from a personal favourite manga/anime series "aku no hana", in particular the flower of evil that appears at the end of the first episode has a distinctive crackling sound, it's distorted, has an odd bass backing, and it's almost static-like input both gives the feeling of this electronic frontier and an added sense of fear from myself, however that fear most likely stems from my association of radio interference with monsters in the "silent Hill" series of video games!


 For the rest of the sound I took some standard recordings in similar environments to the picture for backing, and used several royalty-free ringtones and crowd and recorded conversation noises from a sound studio recording and started to distort them, each with reverberation to reflect the large open space, but also with slow build-ups, reversed sounds of the people, and increasing volumes to gain this sense of overwhelming the listener. This was based partially on extracts from Einstein on the Beach by Philip Glass in terms of it's method of starting comprehensible and building up to the point of pure incomprehensible babble. It was from that sense of overwhelming that I attempted to make the sound feel almost fearful before cutting out to show no one noticing it at all.

 I feel what I should criticise in this piece is perhaps in the sense of pacing and that the end doesn't overwhelm as much as it perhaps is aiming to do, perhaps by adding more people and extra more digital effects such as white noise I could have furthered that sense to its logical conclusion, however the one minute restriction does lead to a sense of wanting a longer build up, that and the crowd sounds are arguably a bit quiet and that i potentially overstepped the dynamic range of common audio equipment; however personally I'm rather happy with it, I set out to make a less literal and more abstract interpretation of the picture, and I'd like to think I have completed that goal!

Osmosis - Treatment

Osmosis is a 3 screened (each screen at resolution 1080 x 1920 Portrait) display based on the nature of eyesight over one's life. This is achieved by the left screen representing birth/infancy, the central screen representing the main part of one's life, and the last representing the latter stages of one's life. This is done over 3 Time-lapses in fields of 15 seconds each:

• The Birth Time-lapse: This is a time-lapse of sunrise (sun coming from the left of the frame) with a baby-milk bottle or rattle at the bottom-left third of the frame. The Screen fades in from black to reveal all in geometric, colourless shapes (most likely using rotoscoping to give this effect); this quickly fades into primary colours and then slowly fades into an out of focus, blurred version of the video with correct colouring; by the end of the 15 second loop this blur should recede entirely to reveal the image in it's original state.
• The Life Time-Lapse: This is a time-lapse of daytime (sun going over top center frame) with a computer monitor in center frame, every couple of seconds it will begin to blur which will be followed by quick frame cuts to opticians and glasses, once it's cut back to the time-lapse the frame is back to being in focus and this process repeats itself as the time-lapse continues.
• The Old-Age Time-Lapse: This is a time-lapse of sundown (sun leaving across the right of the frame) with a newspaper on the crossword section and a pencil at the bottom-right of the frame. as the time-lapse goes on there'll be loss of foreground focus, followed light-tearing on the screen, and cloudy spots developing and encroaching the screen  like cataracts. the problems worsen as the sun goes down as the frame goes into pure darkness where it remains for the rest of its loop, symbolising death.

Shooting can be done over the course of a single weekend (weather allowing) with permissin from either a local park are or done in the privacy of garden grounds. This can be done simply using a High Definition DSLR with a Large memory storage space and automatic timers for taking photographs over set periods of time. for a 3 hour space of time to be captured on screen at 25 fps format for the image sequence then for a 15 second loop the pictures will be automatically needed every 28 seconds approximately with careful need to look after the camera and check the framing, luckily as the camer aill be facing more towards the sun there is little risk of self shadow getting in the way, however it is rather likely that multiple shoots may be necessary as it is more a game of patience and thus having multiple takes of the same time scene allows optimal choice for the best one, therefore filming should be seen as perhaps a 3-4 free day based affair.

Once this is done the chosen sequences will be imported into Adobe After effects where the correct filters can be customised and applied to each sequence.

During showings of one minute long each scene will run fully from left-to-right with the next one being triggered by the last sequence finishing. after te 3rd sequence all will play simulaneously before starting this loop again. If the screens are labelled in numbers in order from left to right then the loop will precede as: 

1 then 2 then 3 then 1+2+3.

Childhood problems based on: http://www.babycentre.co.uk/a6508/developmental-milestones-sight

Life-Problems based on: My Awful Eyes

Old-Age Problems Based on: http://www.webmd.com/eye-health/vision-problems-aging-adults