Mike Robbins works at Helios Design Labs in Canada and has been working on 17000 Islands for two years. He has done a great job with the technical and creative innovation as a designer and programmer. Read what Mike has to say about challenges and milestones with 17000 Islands.
The 17000 Islands project (17K) revolves around audio-visual explorations of Indonesia’s Taman Mini Theme Park.
The physical park itself is a idealized replica of Indonesia’s 17 thousand islands and their diverse cultures. The digital experience recreates these islands with short film and audio clips, and allows the viewer to remix the original footage into their own interpretation. This new film is then added back into the experience as a new “island”.
Here’s a quick snapshot view of two years of hard work, in the form of two integral design challenges the project posed for us and the milestones reached through the effort.
Challenge: Development
Build a powerful, web-based timeline-based film editing console that would allow the audience to assemble and edit video footage, sound, text, subtitles, with the added and important function of recording audio all through the plain old simple web browser.
Milestones: New Procedures
On an interface design and programming level, we pushed already useful toolsets like jQuery and WebAudio, and HTML5 video and canvas elements to places that were difficult to get to but allowed for drag and drop clip assembly, super fine-grained control of media in and out points, and real-time compositing.
On a backend functionality level (nerd alert!), 17k uses websockets, Node.js, and FFMPEG in combination with in-browser hardware access to allow for audio recording through a microphone input straight through to the web-server.
On a work flow level, the 17k editor is all about content creation online and offline. The final save functionality involves the conversion of ones completed piece into a downloadable file that can be imported into Final Cut Pro for conversion into a complete high-res version of what is now a film, thereby extending the reach of the documentary back into the theatre.
Challenge: Design
Make this all fun and engaging, and to encourage an audience to become participants and part of an informative documentary experience.
Milestones: New Visual Language.
17k has a visual and conceptual counterpoint to the linear, timeline-based experience outlined above in the form of a very non-linear, organic experience, loosely labelled the Island Builder. Powered by a simple but complex algorithm, the Voronoi equation, the Island Builder divides a space into a crystalline lattice structure, allowing us to break the video into shapes other than the rectilinear.
People can pull and push these image fragments around to form new shapes and connections. The process is abstract, but the math is such that while maintaining the sense of play and fluidity, all interaction with this Island Builder is actually creating a film edit that is reflected in the linear, timeline-based experience.
On a technical level, it was an achievement to fold the math into a programming structure that could make a connection with the super-nonlinearity of a voronoi equation with the strict linearity of a timeline editor.
On a design level it was an achievement to glue everything together: image sequencing, canvas drawing, cursor tracking, drag and drop, and above all, naturalize the experience with colour, shape and sound.
When the dust settles
17k is borne out of a set of challenges. And it will challenge its audience. The user experience it offers it deep and complex, 17K does many things above and beyond what is described here. Of interest to us and the film-makers, apart from the coolness of the final piece itself, 17k contains the seeds of many new ideas to work with and build on in the future.
Mike Robbins
mike@heliozilla.com
