Project 89: Ambiance Sharing

Background and motivation
Digital telecommunication has evolved from transmitting simple text messages to very complex systems enabling high quality audio/video conferencing, even on small mobile phones. With the advances of communication and computing technologies, we expect that the future will bring much more. It will include transmitting the full ambiance that a person is in, representing the entire 3D visual and audio scene. This type of communcation will make us fully immersed and we will feel being in another place than our current location.

Project goals
This project will try to build a prototype device that gives a hint of what is possible with these emerging technologies. In addition, novel concepts will be studied using the prototype and very exciting demos will be realized. Some applications that this device could enable are:
- Real-time communication with 3D video and binaural audio
- 360o recording of an audio-visual scene
- Free-view point navigation of a 3D audiovisual scene

First step of the project is to decide on specifications of the device, and the required components. Next step is building the prototype and realizing novel concepts utilizing the prototype.

Initial requirements
There will be a hardware prototype having multiple pairs of stereo cameras, and multiple microphones. Initially, this device will be communicating with the PC. All the compression and signal processing will be done at the PC side. The user will be able to choose any view and its respective 3D view and 3D audio will be transmitted.

Desired student skills/competencies/team size
Interfacing cameras&microphones to PC (1 person)
Hardware design (1 person)
Software development (2 persons)
Signal processing (audiovisual and computer vision) (2 persons)
Project Lead (one of the above).
Team size: 6 people

Development tools, environments and standards
C, C++ for software development at the PC. Interfacing cameras&microphones to PC.

Deliverables and time period
The prototype device will be ready in early 2010. Follow up development based on feedback: Sept. 2009-Jan. 2010 according to course schedule.

Interaction between stakeholder and project team
Weekly meetings with stakeholders.

Support (supervision & financial)
Nokia will be assisting on all aspects of this project, including 3D video/audio compression. Nokia will be covering purchasing of the equipment.

Main contact person
Mauri Väänänen, mauri.vaananen(at)nokia.com
Please also inform Demola Staff when you contact the company contact person.

References (pubs, books, links, …)
Telepresence: http://en.wikipedia.org/wiki/Telepresence
Microphone arrays: http://en.wikipedia.org/wiki/Microphone_array
Stereo imaging/video: http://en.wikipedia.org/wiki/Stereoscopy
Dodece system from Immersive Media: http://www.immersivemedia.com/