Site Navigation


Gaming and immersive art intersect through the Lightpad Block

TJ Hughes uses MIDI technology to create immersive interactive artworks

Date Posted
Share on Social Media
Play video
Gameplay footage from greg • gar • i • ous by TJ Hughes, an interactive artwork on display at the Art Hack Day, St Louis.

Are video games art? The question has been asked for almost as long as video games themselves have existed. But as more and more creative titles enter into the mainstream, it seems that the answer is emphatically yes. For some players, beautiful environments, immersive storylines and luscious soundtracks are now just as important as gameplay. And as gaming moves ever closer towards the arts, so too does the potential for visual art to become a more interactive, game-like experience.

As these lines between games and art blur, the type of controllers used by creators become all the more important. Controllers are an essential bridge between an audience and an interactive artwork — and new technology means that the possibilities for more immersive pieces are expanding too.

One community dedicated to exploring this potential is the Art Hack Day community, a global project founded to showcase the intersection between art and the latest technology. In St Louis, Missouri on August 17-19th, the latest Art Hack Day focused on the theme of “Eclipse”, to coincide with the solar eclipse overshadowing the US. And one artwork, created by self-taught developer TJ Hughes, used a cutting-edge controller — the Lightpad Block — to create a haunting experience centered around this theme.

Just like a video game, greg • gar • i • ous, puts you in control. By moving your hand along the continuous surface of the Lightpad Block, you move a spotlight across a shadowy plain — and all the while it’s followed by a group of black, red and glowing human figures. Press into the Lightpad surface and the spotlight becomes brighter. Accompanied by ambient electronic chords and voices, greg • gar • i • ous is a hypnotic and engrossing piece of art.

“The piece was inspired by a dream I had, where featureless figures fought for dominance on a hill made of ragdolls” recalls TJ. “In the dream everything was shrouded in almost complete darkness, except for one point of light. I just watched as these figures danced and fought”. Much like the mysterious dream which inspired it, TJ prefers to keep greg • gar • i • ous ambiguous so his audience can come to their own interpretations of what it might mean.

TJ, who has founded his own games company called “Terrifying Jellyfish”, created greg • gar • i • ous using a free-to-download game engine called Unity3D. The MidiJack plug-in by Keijiro Takahashi allowed the game engine to read MIDI input, so the Lightpad could be programmed to interact with what the audience sees on screen.

The Lightpad Block is used as a controller for greg • gar • i • ous — both the position and the brightness of the spotlight can be controlled with the multi-dimensional surface.

“I like to incorporate cutting-edge MIDI Controllers in my work because the crossover between the latest in music, gaming, and hacker culture has so much potential” explains TJ. Traditional gaming controllers have deployed buttons, toggles or 2D touch screens, but the extra dimensions of control available using controllers like the Lightpad Block — made possible by the ability to press into the surface — expand the creative possibilities. With so much potential for inventiveness, TJ recommends experimenting with MIDI to any aspiring game developer: “MIDI as a protocol is very flexible and really fun to hack!”

TJ’s latest project also makes use of MIDI controllers. Nour is an experimental game about food with a simple aim: to try to make players hungry. Players can interact with virtual meals using a pad controller, such as DJ Tech Tools’ Midi Fighter 3D or the Lightpad Block. Combining bright colors, visuals, textures and sounds, Nour aims to stimulate your senses — and perhaps your stomach too! By taking advantage of Midi Fighter 3D's inbuilt accelerometer, it’s just another example of how conceptual art, gaming and MIDI control can coincide to produce fascinating and delightful experiences.

With independent game developers like TJ Hughes coming up with new ideas everyday, the future of gaming looks sure to be filled with colour, creativity and artistic flair.

Check out the latest from TJ Hughes on the Terrifying Jellyfish website and Facebook. TJ is now raising funds for Nour on Kickstarter — learn more here and pledge your support before October 18th.

Watch TJ Hughes explain and preview Nour: