MARA

Overview

The Mobile Augmented Reality Applications project explores utilizing camera equipped mobile devices as platforms for sensor-based, video see-through mobile augmented reality. The project also investigates new and exciting applications enabled by this technology, and UI solutions and paradigms motivated by the restrictions of the mobile devices.

• AR overview
• ISMAR06

How it works

If the absolute location and orientation of a camera is known, along with the properties of the lens, it is possible to determine exactly what parts of the scene are viewed by the camera.

The MARA prototype application uses accelerometers in all three axes to determine orientation, a tilt compensated compass for heading, and GPS for positioning.

Since these sensors give the (approximate) location and orientation, it is possible to annotate the viewfinder screen with information about real world objects, once the device is focusing on those objects.

Hardware

the Hardware

This is the hardware the demonstration videos were recorded with - a Nokia 6680 mobile device with an add-on box containing GPS and sensors.

Implementation and Prototyping

Our prototype application overlays the continuous viewfinder image stream captured by the camera with graphics and text in real time, annotating the user's surroundings. The prototype consists of Nokia S60 platform phone and attached external sensor box providing position and orientation information to the phone via a Bluetooth connection.

Notable features:

• The prototype enables users to find locations more easily by highlighting them on-screen with Virtual Object.
• URLs can be associated with Virtual Objects, allowing for hyper-linking of real world objects
• Objects can be stored on the terminal, or streamed from a server. Streaming allows users to share positioning data, facilitating "Find a Friend" style applications (picture).
• The prototype has automatic switching to map-view when the user holds the phone horizontally. This displays the users' position on a map of the area, and highlights nearby virtual objects.

Video Demonstration

The following videos were created by capturing the screen of the device while the prototype application was running. The videos are compressed for easier streaming; actual video quality is equivalent to the normal viewfinder-mode quality. Click on each image to download or start the video.

Virtual Video - The Town Square

This video clip demonstrates how the basic virtual objects are seen with the current prototype application (the quality of the video is a bit poor, in real life the picture seen in the phone is naturally sharp and clear). User can easily select whichever object seen on the screen and in order to get more information about the object to access its URL automatically via browser application.

Friend Finder Video

This video demonstrates the current state of our "Friend Finder" concept. In this clip, plain GPS was used for the positioning of both devices, and we hope to improve the accuracy of this application to very high levels using relative positioning technologies in the near future.

City Center Video

This video of downtown Helsinki shows some Virtual Object - associated landmarks, and demonstrates the automatic switching between Augmented Reality mode and Map mode that happens when the user alternates between holding the phone vertically and horizontally.

Additional demonstration material

Mara poster

The Future

The MARA Technology Exploration Project has concluded. More research into this kind of application is being carried out by the Visual Computer User Interface group in NRC Palo Alto.

ISMAR 06

MARA was demonstrated at the fifth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR06) in Santa Barbara in October.

Press

MARA was featured by MIT Technology review here, and listed as one of their 10 emerging technologies for 2007