Mobile 3D Video

Mobile 3D Video

Mobile 3D 

With recent advances in capture and display technologies, 3D video communication and entertainment services are expected to become reality and redefine the communication and entertainment services with high immersion. To enable these exciting new services, Joint Video Team (JVT) of ITU-T and MPEG has devoted its recent efforts to extend the widely deployed H.264/AVC standard for multi-view video coding (MVC). MVC provides superior network robustness and compression performance for delivering 3D video by e.g. taking advantage of the inter-view dependencies of the different visual channels. In addition, its backwards compatibility with H.264/AVC codecs makes it widely interoperable in environments having both 2D and 3D capable devices.

Nokia's MVC Software

Nokia Research Center has developed a highly optimized MVC decoder running on Maemo platform, and demonstrated a natural real-time 3D movie playback on a modified Nokia N800 internet tablet with an integrated auto-stereoscopic display. The prototype was showcased at IEEE 3DTV-CON and Nokia World 2008. The source code implementing the 3D Video playback demos mentioned above is available for download now. The Nokia’s MVC software includes two modules:

1. MVC Encoder, which runs in PC and configured to generate MVC compliant bitstreams
2. MVC Decoder is a player, fully optimized for the Maemo platform, and decodes stereoscopic MVC bitstreams.

Download

Nokia Research Center welcomes any feedback on the software, in terms of bug-fixes or additional functionalities. Please send all your questions/comments to h264sw@nokia.com or to project team.

More Information on MVC

MVC Whitepaper

For more detailed information, please refer to the below standardization documents and academic papers that discuss the MVC standard and 3D Video services in more detail.
Standard Documentation and the Software

- Joint Draft 8.0 on Multiview Video Coding [http://wftp3.itu.int/av-arch/jvt-site/2008_07_Hannover/JVT-AB204.zip]
- WD 4 Reference Software for Multiview Video Coding (MVC) [http://wftp3.itu.int/av-arch/jvt-site/2009_01_Geneva/JVT-AD207.zip]
- RTP Payload Format for MVC Video (draft) [http://tools.ietf.org/html/draft-wang-avt-rtp-mvc-03]
- MVC File Format (draft) [http://wg11.sc29.org/meetings/87_Lausanne/w10444/w10444.zip] (available to MPEG members only)

Detailed Information on the MVC Standard

- Merkle, P.; Muller, K.; Smolic, A.; Wiegand, T., "Efficient Compression of Multi-View Video Exploiting Inter-View Dependencies Based on H.264/MPEG4-AVC,"; Multimedia and Expo, 2006 IEEE International Conference on , vol., no., pp.1717-1720, 9-12 July 2006
- Ying Chen;Ye-Kui Wang; Ugur, K.; Hannuksela, M.M., Lainema J.; Gabbouj M., "3D video services with the emerging MVC standard," EURASIP Journal on Advances in Signal Processing, Volume 2009, Article ID 786015, 13 pages, doi:10.1155/2009/786015.
- Ugur, K.; Hui Liu; Lainema, J.; Gabbouj, M.; Houqiang Li, "Parallel Encoding - Decoding Operation for Multiview Video Coding with High Coding Efficiency," 3DTV Conference, 2007 , vol., no., pp.1-4, 7-9 May 2007
- Ying Chen; Ye-Kui Wang; and Gabbouj, M.; "Buffer requirement analyses for multiview video coding," 26th Picture Coding Symposium (PCS 2007), Nov. 2007, Lisbon, Portugal.
- Smolic, A.; Kauff, P., "Interactive 3-D Video Representation and Coding Technologies," Proceedings of the IEEE , vol.93, no.1, pp.98-110, Jan. 2005
- Vetro, A.; Matusik,W.; Pfister, H; Jun Xin; "Coding Approaches for End-to-End 3D TV Systems," Picture Coding Symposium 2004 (PCS-2004), San Francisco, California, USA, December 2004.

Nokia’s Mobile MVC Prototype

- Willner, K.; Ugur, K.; Salmimaa, M.; Hallapuro, A.; Lainema, J., "Mobile 3D Video Using MVC and N800 Internet Tablet," 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video, 2008 , vol., no., pp.69-72, 28-30 May 2008

Improvements on top of MVC
Below are some improvements Nokia Research Center has been working on to improve the performance of MVC even further.

- Ying Chen; Ye-Kui Wang; Hannuksela, M.M.;  Gabbouj, M., "Picture-level adaptive filter for asymmetric stereoscopic video," 2008 IEEE International Conference on Image Processing (ICIP 2008), Oct. 2008, San Diego, CA, USA.
- Ying Chen; Shujie Liu; Ye-Kui Wang; Hannuksela, M.M.; Houqiang Li; Gabbouj, M., "Low-complexity asymmetric multiview video coding," Multimedia and Expo, 2008 IEEE International Conference on , vol., no., pp.773-776, June 23 2008-April 26 2008
- Shujie Liu; Ying Chen; Ye-Kui Wang; Gabbouj, M.; Hannuksela, M.M.; Houqiang Li, "Frame loss error concealment for multiview video coding," Circuits and Systems, 2008. ISCAS 2008. IEEE International Symposium on , vol., no., pp.3470-3473, 18-21 May 2008
- Ying Chen; Ye-Kui Wang; Hannuksela, M.M.; Gabbouj, M., "Single-loop decoding for multiview video coding," Multimedia and Expo, 2008 IEEE International Conference on , vol., no., pp.605-608, June 23 2008-April 26 2008


Project Team:

Antti Hallapuro, Miska Hannuksela, Jani Lainema, Marja Salmimaa, Kemal Ugur, Kai Willner, Ye-Kui Wang, Ying Chen