A real-time reconstructed 3D environment augmented with virtual objects rendered with correct occlusion


Publication Title
IEEE Xplore
Publication Date
October 22 2014
Creators/Contributors
Saad R. Khattak (contributor)
Brent Cowan (contributor)
Iuliia Chepurna (contributor)
Brent Cowan (creator)
Iuliia Chepurna (creator)
Andrew Hogue (creator)
Saad R. Khattak (creator)
Persuasive Intent
Academic
Description
In this work, a novel framework is presented for the real-time interaction with 3D models in augmented virtual reality. This framework incorporates view-dependent stereoscopic rendering of the reconstructed environment together with user's hands and a virtual object, and high-precision gesture recognition to manipulate it. Proposed setup consists of a Creative RGB-D camera, Oculus Rift VR head mounted display (HMD), Leap Motion hands and fingers tracker and an AR marker. The system is capable of augmenting the user's hands in relation to their point of view (POV) using the depth sensor mounted on the HMD, and allows manipulation of the environment through the Leap Motion sensor. The AR marker is used to determine the location of the Leap Motion sensor to help with consolidation of transformations between the Oculus and the Leap Motion sensor. Combined with accurate information from the Oculus HMD, the system is able to track the user's head and fingers, with 6-DOF, to provide a spatially accurate augmentation of the user's virtual hands. Such an approach allows us to achieve high level of user immersion since the augmented objects occlude the user's hands properly; something which is not possible with conventional AR. They hypothesize that users of our system will be able to perform better object manipulation tasks in this particular augmented VR setup as compared to virtual reality (VR) where user's hands are not visible, or if visible, always occlude virtual objects.
HCI Platform
Wearables
Discursive Type
Inventions
Location on Body
Finger, Hand, Eye
Keywords
Work

How to cite this entry
Brent Cowan, Iuliia Chepurna, Andrew Hogue, Saad R. Khattak. (October 22 2014). "A real-time reconstructed 3D environment augmented with virtual objects rendered with correct occlusion". IEEE Xplore. IEEE. Fabric of Digital Life. https://fabricofdigitallife.com/index.php/Detail/objects/1207