Applying Perceptual Intelligence for Virtual Interaction
via High Performance Network

Executive Summary

With the advance of computer technologies, the interaction between humans and computers becomes necessary to our daily life. The topic of improving Human-Computer Interaction (HCI) becomes an important and popular research area. However, the fundamental research in HCI has not changed significantly for the past twenty years. We notice that most users' interaction with computers is still by the traditional means, such as typing, pointing and clicking. Existing GUIs that incorporate with human action in 3D virtual environment (VE) are still limited to mechanical devices such as mouse, joystick, gloves, or head-mounted device (HMD). Since traditional input methods are no longer suitable in most scenarios, especially interacting with virtual environment, the need arises for more general and intuitive ways for interacting with virtual environment. What we expect is an innovation of interaction techniques that can be well matched with how people will use computers. Naturally, the optimal solution is to make computers understand human's behavior and the means of human-to-human communication. The host computer providing virtual reality service could possibly understand "where are the people?", "who is it?", "what are the people doing?", and "why are people doing this?".

We herein proposed perceptual intelligence as being key to interfacing with coming generations of machines, whose interface is called Perceptual user interfaces (PUIs). PUIs are multi-disciplinary research and development projects that integrate several advanced technologies in computer science. It requires the discipline of computer vision-based sensing and perception of human activities to regenerate a more human-like environment. Our proposed architecture for PUIs, application in VR consists of (i) gesture recognition, (ii) face recognition, (iii) transmitting high volume of feature vectors through Internet 2, (iv) CAVE participants represented by 3-D graphic interface. Having the support from high-speed network - Internet 2 and 3D computational visualization, we are able to build a communication channel to perform remote interaction between different virtual environments and represent 3D realistic human character animation mimic to real particants' action in a frame rate higher than 15 frames per second. The connectivity of VR spaces can be established between IUPUI Immersadesk and IUB CAVE via Internet Abilene, therefore the performance of interaction can be evaluated.

Our innovative architecture can provide a great opportunity to demonstrate IU's strong information technology capability in understanding the concept of that future human computer interaction is highly dependent on machine's perceptual intelligence rather than traditional GUIs. It also demonstrates that Internet 2 is not only capable in transmitting streaming data in very high speed (fast) but also capable in carrying information intelligently (smart).

---

• Return to the HPNAP Projects page.
  
• Return to the HPNAP page.