Gaze Control

Near-to-Eye Display (NED) offers a big screen experience to the user anywhere, anytime. It provides a way to perceive a larger image than the physical device itself is. To be able to interact with the displayed UI efficiently, we have also integrated a video-based gaze tracker into the NED. The data from the highly integrated gaze tracker reflects the user focus point in the displayed image which can be used as an input device for the NED system.

Virtual reality and augmented reality provide means to present the user with objects to interact with. When combined with a NED system, gaze tracking based interaction becomes an attractive option. However, interaction methods for gaze tracking based UIs are not well established. Experimentation is needed to understand how to best control objects in such systems.

Project goal

The project develops technology for gaze based control of a virtual or augmented reality environment. The resulting application should interface with a 3D object tracking subsystem in order to be able control tracked objects (initiate tracking, select tracked object(s), stop tracking, possibly later more fine grained controls).

This requires integration of existing separate enablers (on Windows and Linux) onto a common platform (Linux). On top of these, an application will be built enabling gaze based control of objects in the virtual/augmented environment. Several interaction methods may be developed and tested to find out the one that works best.

Depending on team size and project schedule, additional augmented reality use cases can be built on top of the initial system.

Team

• Project management
• Linux software development (virtual “mouse” driver, camera controls, gaze tracking porting and integration)
• Linux/Qt software development (gaze control application development)
• UI / UX & graphics design (gaze based control methodology, UI design prototyping, graphics)
• UX testing (test design, user testing, result analysis)

Development tools, environments and standards

Laptop/desktop Linux SW development

Deliverables

Integrated software enablers (mouse and camera drivers and gaze tracking porting)Gaze control applicationGaze based object control interaction designGaze control UX test plan and results

Main contact person

Pasi P. Liimatainen (pasi.p.liimatainen@nokia.com)

References

Compact near-to-eye display with integrated gaze tracker