Projection Augmented Model - Projection Augmented Models

Projection Augmented Models

Figure 2 The Projection Augmented model concept

A Projection Augmented model (PA model) consists of a physical three-dimensional model, onto which a computer image is projected to create a realistic looking object (Figure 2). Importantly, the physical model is the same geometric shape as the object that the PA model depicts. For example, the image projected onto the objects shown in Figure 3 provides colour and visual texture, which makes them appear to be made from different materials.

Figure 3 An example of a Projection Augmented model (inset - with the projection off).

PA models use a unique combination of physical objects and computer-generated information, and hence they inherit advantages from both. “The human interface to a physical model is the essence of ‘intuitive’. There are no widgets to manipulate, no sliders to move, and no displays to look through (or wear). Instead, we walk around objects, moving in and out to zoom, gazing and focusing on interesting components, all at very high visual, spatial, and temporal fidelity”. PA models combine the high level of intuitiveness of physical models with the flexibility and functionality of computer graphics, such as the ability to be quickly altered, animated, saved and updated (Jacucci, Oulasvirta, Psik, Salovaara & Wagner, 2005). Thus, a PA model essentially gives a physical form to a computer-generated object, which a user can touch and grasp with their bare hands. It is therefore unsurprising that user studies, which compared PA models to other Virtual and Augmented Reality displays, found PA models to be a natural and intuitive type of display (Nam & Lee, 2003; Stevens et al., 2002).

However, the PA model concept is not new. In fact, one of the first PA model type displays was created over twenty years ago when Naimark built the ‘Displacements’ art installation (Naimark, 1984) and more recently in the “Haunted Mansion” attraction in Disney World (Liljegren & Foster, 1990). At the time technology did not exist for a PA model to be much more than an artistic statement. However, given the technology available today and a little “unfettered imagination”, exploring novel projection displays is now “potentially boundless”.

The growth in PA model technology has been marked by the recent recreation of Naimark’s ‘Displacements’ installation at SIGGRAPH (Displacements, 2005). Specifically, new technology has been developed that semi-automates the process of both creating and aligning the physical model and projected image. This supports multiple projectors, which enables a PA model to be illuminated from every direction. Furthermore, powerful projectors (2000-3000 lumens) can be used to allow a PA model to be located in a well-lit room (Nam, 2005; Umemoro, Keller & Stappers, 2003). However, whilst this technology enables a PA model to be a viable and useful type of display, it does not address its main aim.

A PA model aims to create the illusion of actually being the object that it depicts. For example, when used for a product design application, it is important that a PA model provides a convincing perceptual impression of actually being the final product (Nam, 2006; Saakes, 2006; Verlinden, Horváth & Edelenbos, 2006; Keller & Stappers, 2001). Similarly, when used for a museum display application to create a replica of an artefact, a PA model aims to create the illusion of being the real artefact (Hirooka & Satio, 2006; Senckenberg Museum, 2006; Bimber, Gatesy, Witmer, Raskar & Encarnacao, 2002; Museum of London, 1999).

However, no previous research has specifically considered this illusion. Therefore, this thesis defines the ‘Projection Augmented model illusion’ as the situation in which a PA model is perceived to actually be the object that it depicts. For example, this illusion occurs when a user perceives the PA model in Figure 3 to be real bricks, flower pots, and pieces of wood, as opposed to white models with an image projected onto them. However, the essence of this illusion does not involve deceiving the user. A user can perceive a PA model to be the object that it depicts, whilst knowing that it is actually a white model and a projected image.

Technology has been developed to enhance this illusion by increasing the physical similarity between the PA model and the object that it depicts, or in other words, increasing the fidelity of the PA model. For example, the way in which the specular highlights on an object move as the viewer changes position can be dynamically simulated. This enables a PA model to appear to be made from a wide range of materials. For example, a dull clay vase can appear to be made from a shiny plastic material.

However, whether or not the PA model illusion occurs is entirely dependent on a user’s subjective perceptual impression. Therefore, increasing the fidelity of different aspects of a PA model may each have a different effect on the strength of the illusion. This is essentially the same as the way in which increasing the fidelity of different aspects of a computer-generated photorealistic image, may each have a different effect on the degree to which the image is perceived to be a real photograph (Longhurst, Ledda & Chalmers, 2003; Rademacher, Lengyel, Cutrell, & Whitted, 2001). For example, increasing the fidelity of the textures in the image may typically be more important than increasing the fidelity of the shadows. It cannot therefore be assumed that increasing the fidelity of any aspect of a PA model will automatically strengthen the PA model illusion, and similarly it cannot be assumed that decreasing the fidelity of any aspect will automatically weaken it. Therefore, given that no previous research has investigated this illusion, it is difficult to determine the success of the technology that aims to enhance it, and difficult to make informed decisions when developing new technology. The capabilities of the human perceptual system should guide the development of any advanced interface (Stanney et al., 2004), hence this issue needs to be addressed.

Note: Projection Augmented models are sometimes referred to as 'Shader Lamps' (Raskar, Welch, Low & Bandyopadhyay, 2001, p. 89).