Projection Augmented Model - Background: Uniting Physical and Virtual Objects

Background: Uniting Physical and Virtual Objects

Spatially Augmented Reality (SAR) renders virtual objects directly within or on the user's physical space. A key benefit of SAR is that the user does not need to wear a head-mounted display. Instead, with the use of spatial displays, wide field of view and possibly high-resolution images of virtual objects can be integrated directly into the environment. For example, the virtual objects can be realized by using digital light projectors to 'paint' 2D/3D imagery onto real surfaces, or by using built-in fiat panel displays.

Real objects can be physically handled and naturally manipulated to be viewed from any direction, which is essential for ergonomic evaluation and provides a strong sense of palpability (Ishii & Ullmer, 1997). Although ‘simulated’ haptic feedback devices enable some aspects of computer-generated objects to be ‘touched’, they can not match this level of functionality (Evans, Wallace, Cheshire & Sener, 2005; Baradaran & Stuerzlinger, 2005; Khoudja, Hafez & Kheddar, 2004). It is, therefore, unsurprising that physical objects are still used for many applications, such as product design (Dutson & Wood, 2005). However, computer-generated objects have a key advantage; they provide a level of flexibility that cannot be matched by physical objects. Therefore, a display is needed that somehow ‘joins’ the real physical world and computer-generated objects together, thus enabling them to be experienced simultaneously (Gibson, Gao & Campbell, 2004; Ishii & Ullmer, 1997).

Tangible User Interfaces (TUI) and Augmented Reality both aim to address this issue. TUI systems use real physical objects to both represent and interact with computer-generated information (Figure 1). However, while TUIs create a physical ‘link’ between real and computer-generated objects, they do not create the illusion that the computer-generated objects are actually in a user’s real environment. That is the aim of Augmented Reality.

Figure 1 Continuum of advanced computer interfaces, based on Milgram and Kishino (1994).

Unlike Virtual Reality (VR), which immerses a user in a computer-generated environment, Augmented Reality (AR) joins together physical and virtual ‘spaces’ by creating the illusion that computer-generated objects are actually real objects in a user’s environment (Azuma et al., 2001)(Figure 1). Furthermore, head-mounted-display based AR, and in fact VR, systems can directly incorporate physical objects. Thus, as a user reaches out to a computer-generated object that they can see, they touch an equivalent physical model that is placed at the same spatial location (Whitton, Lok, Insko & Brooks, 2005; Billingshurst, Grasset & Looser, 2005; Borst & Volz, 2005; Lee, Chen, Kim, Han & Pan, 2004; Hoffman, Garcia-Palacios, Carlin, Furness & Botella-Arbona, 2003). Such systems enable the computer-generated visual appearance of the object to be dynamically altered, while the physical model provides haptic feedback for the object’s underlying form. However, head-mounted-display based systems require users to wear equipment, which limits the number of people who can simultaneously use the display.

A variant of the AR paradigm that does not suffer from these limitations is Spatially Augmented Reality (Figure 1) (Raskar, Welch, Fuchs, 1998). Spatially Augmented Reality displays project computer-generated information directly into the user’s environment (Bimber & Raskar, 2005). Although there are several possible display configurations, the most natural type is the Projection Augmented model.