The Art of Research

Research scientists often claim that they aren't creative. They say this, even though the work that they do produces something that didn't exist when they started. The truth is, they are highly creative. Their work inspires my own.

What they really mean is that they aren't "visual". Of course, that isn't true either. All they really, really mean is that they can't draw. Drawing trains a person to imagine and work towards a final aesthetic without much guidance. "I could see it in my head". Drawing lets me work through all manner of design problems in a way that lets others see and comment on the decisions I'm making. It becomes a very collaborative activity. Scientists theorize and artists visualize.

If I'm honest, I've been doing pretty much the same thing since I was just a kid. Art is the skill I have, and art is the skill I use to earn a living. Aside from a few forays into the worlds of construction and dishwashing, I've been employed as a designer of one sort or another since the 90's. From then until now, my goals haven't changed: Explain whatever my client wants explained as efficiently as I can.

In 2002 I began working with with scientists and engineers at the company operating IKONOS, an Earth observing Satellite. Initially, I was hired to help market their imaging products. As we continued to work together, it became clear that they were having trouble explaining certain concepts. It turns out, I could illustrate those concepts.

Initially, I created simple infographics showing the satellite in various situations. In order to depict the satellite well, I built a fairly realistic 3D model of it. The use of 3D models in the illustrations had a side benefit. Those same models could be repurposed for animations. As primitive as they look by modern standards, those animations were actually used in a few national news segments. Including an interview with, then President, George W. Bush.

The animations lead to an interactive CD-ROM exploring IKONOS, it's products and the science behind them. In it, Satellite imagery was combined with other data to produce detailed depictions of landscapes, use-cases and collection methods. The CD-ROM was well received. So well, that I was invited to participate in the Geobook project. That was my first real introduction to User Interface Design. I had experience working on games and CD-ROMs, but this was more complex. For it's time, Geobook was a novel way to look at pictures of a location within the context of a map.

Those experiences lead to more work visualizing everything from the the global spread of diseases to the way that a nuclear reactor works. In 2003, I answered an ad for a graphic designer with multimedia experience. FXPAL, a Silicon Valley research lab, needed someone to lead a group of designers producing artwork for an experimental multimedia platform. Even a small lab produces a fantastic number of ideas. I quickly found myself jumping from one project to another. By 2004 I was a full time member of the staff.

At that time, FXPAL was almost exclusively conducting research into software solutions for various forms of media creation and consumption. The focus was mainly on multimedia documents in the workplace. From the beginning, my own work fell neatly into two categories:

1) Make a video or illustration that explained a concept.

2) Make a research prototype more useable.

Task #1 was made much easier by my colleague at the lab, John Doherty. John had been a professional cameraman and electrician in Hollywood. I studied film in school, but all of my work experience was with video and fairly unorthodox. With his help, I've learned to incorporate all of the 3D and illustrative skills I can muster into "vision videos"—videos that describe not just a technology, but an imagined application of that technology.

The illustrative work includes everything from detailed concept renderings to icons that encapsulate the intent of the research. I've drawn so many people interacting with so many screens... 

For Task #2, I use classic graphic design principles to produce static or interactive software prototypes that distill research into usable User Interfaces. More often than not, my contributions help scientists refine their ideas. To me, it's all about communication. Does the UI tell the user what they need to know? Can the user tell the UI what it needs to know?

FXPAL, like all research labs, has evolved along with the technologies it investigates. Today we build as many devices as applications. This has lead to an unexpected evolution of task #1. In the past I might produce some concept art based on a prototype in our lab. As the illustrations become more refined, I generally build 3D models of the imagined devices.

Before the "maker revolution", my 3D models existed only to produce still and animated artwork. Now, I'm being asked to actually build some of the things that I illustrate. 

In 2014 and 2015 a group of researchers and I worked on a robotic telepresence device called Jarvis. The same 3D files that I created to make illustrations and animations of the device were repurposed to laser-cut and 3D print the pieces used in its construction. We went through nine different iterations of the design, but we only had to build three physical prototypes. The 3D renderings and animations improved the actual physical design, and vice versa. Any time we needed to build a physical prototype, we already had refined 3D files.

In 2016 I produced a series of concept renderings depicting telepresence and document sharing devices. One of these illustrations featured a floor lamp design.

This design was actually something that FXPAL could use as a platform for evaluating the technology. I was asked to build one. This involved designing and 3D printing various pieces that attach hardware to the lamp's frame. I also worked with a talented metal fabricator and a company that makes custom lampshades. This resulted in a simple, highly customizable test-bed for a collection of related technologies.

Drawing lets me work through designs very quickly. It enables me to share my work as I go. I make dozens of sketches, often while I'm meeting with the researchers involved. This way, most of the engineering problems have been resolved before I ever start working on a 3D model. I suppose this has become my Task #3.

 

Research scientists come up with all sorts of crazy ideas. Artists can visualize and build those ideas.

Artists come up with all sorts of crazy ideas. Research scientists can visualize and build those ideas.

Both statements are true.

 

 

 

My Adventures in a Chocolate Factory

In 2008, I was part of a team that built virtual representations of real factories. This project was lead by Dr. Maribeth Back and Dr. Don Kimber with contributions from many other research scientists. Tcho, an under construction chocolate factory, was the subject of our study. Our team traded insights gained for the right to lurk about and eat Tcho’s chocolate. If you’re interested in this research please read: The Virtual Chocolate Factory:Mixed Reality Industrial Collaboration and Control

Panorama of the factory as it existed in 2008-09

Panorama of the factory as it existed in 2008-09

It was my job to visualize our work. I sketched and 3D modeled all sorts of machines and their surroundings. This lead to the creation of an interesting set of artifacts. I’ve never had the opportunity to explore a space quite so thoroughly. I crawled up, over and into various machines. I measured and photographed everything I saw.

Sketch of  a  Carle Conch

Sketch of  a Carle Conch

All of the major elements of the factory were recreated in 3D. Chocolate making isn’t new. Some of the best machinery predates CAD files by decades. Luckily, I was in the factory while much of the vintage equipment was being restored. This gave me access to the inner workings of some beautiful machinery.

Inner workings of a Macintyre Conche

Inner workings of a Macintyre Conche

Detailed rendering of chocolate making conches

Detailed rendering of chocolate making conches

All of this work enabled us to build a virtual chocolate factory. I simplified the 3D models and gave them to a developer who incorporated them into a 3D game engine. Production data from the working factory was integrated into this new virtual space.

The factory as depicted in a game engine

The factory as depicted in a game engine

One of my favorite machines in the factory was the highly articulated Carle Conch. Its a fascinating device that grinds, polishes and heats dry cocoa into liquid chocolate.

With all of its exposed moving parts, the conch is an interesting machine to watch. The 3D model I built was fully animated.

We discussed creating smaller versions of the chocolate making machinery. Something a user could hold, perhaps as a way of interacting with the VR space and vicariously the real space. I modified the conch model so that it could be 3D printed. The result was an accurate small scale replica of the original.

3D printed model

Virtual, real, virtual... it all gets a bit blurry after a while. As the project developed, the notion of what a virtual factory might be drifted. One of the more successful offshoots of the project was this simple smartphone app.

Mobile application

Mobile application