Augmented Reality Mobile Application of Balinese Hindu Temples: DewataAR

Published Online January 2015 in MECS DOI: 10.5815/ ijcnis.2015.02.07

E-tourism is the implementation of information technology to support the tourism industry, travel agents, hotels, and other industries related to tourism. The problems such as limited promotion cost and the weak marketing links can be solved by applying the progress of technology and information smartly. Strong eagerness and support from government and all stakeholders is needed to develop various e-tourism products with simultaneous innovation and creation. In addition, role of application developers is also important in supporting etourism, hence it can create information systems or applications that can support tourism promotion and can also assist travelers who want or are traveling in getting information [1].

Nowadays, augmented reality (AR) is the utilization of information technology that can attract society. AR is defined as the technology which combines twodimensional or three-dimensional virtual objects into the real three-dimensional environment and then projects those virtual objects in real time. AR is the implementation of virtual objects into reality, whether they are 2 dimensional objects or 3 dimensional objects [2]. Rapid improvement of AR has been applied in many aspects of life.

The progress in technology has made the development of some frameworks and toolkits possible, hence they became pioneers in the easy development of AR applications. Below are some of openly available frameworks.

First is DroidAR, It is for Android OS mobile devices only. It is a framework that enables the development of AR applications in Android OS. DroidAR provides location-based and marker-based AR practicalities [3].

Nowadays, Layar is one of the most well-known mobile AR platforms. It can be used in mobile devices with Android OS, iPhone OS, Symbian OS and BlackBerry 7 OS. It possess for more than 10M installs, 9,000 developers and 2,500 individual AR applications, offered as layers [4].

AR technology has recently been used in several fields, for instance medicine, education, and simulated training among others [5]. So does tourism sector, which has the goal to improve the tourist experience. AR is also used to promote tourism objects or products related to tourism.

Basel is one of cities in Switzerland. What makes it special is that it has its own AR tourist guide. Previously, it was started as part of the “Augmented Reality for Basel” project. Now, it is accessible through the LayarAR browser. Tourists are able to gain useful information about Basel city and its suburbs. It also provides information for sites, museums, restaurants, hotels, information for events and shopping arcades [6].

Next, there is The Street Museum. It is purposely created for the needs of the Museum of London. It allows users to see London based on various points in history. Users can aim the camera of their smart phones to present day street views and the device will display historical pictures of the places on the top of their real view, provided by the Museum’s collection. It also provides additional information and can be displayed by touching information buttons [7].

One of the implementations of AR in the tourism sector is promoting and presenting tourism objects to tourists in attractive three-dimensional objects by using smart phones or tablets. AR will be very useful because people who get involved in tourism business can present tourism objects virtually in 3D interactively as well as in real time about objects that they present. AR information systems help tourists to gain important information and knowledge about tourism objects, as much as giving new experience and entertaining users at the same time [8].

The application AR of Temples will help tourists to gain information about tourism objects visited in more attractive ways and more interactive, therefore the tourists will experience new sensation of visiting temples in Bali generally. By applying the technology of AR for promotion method, the actors of tourism business will be able to gain more profit by attracting more customers. The tourists themselves will be able to share information with their friends or colleagues whom are not able to visit the tourism objects directly [1]. This is one of advantages of AR which combines virtual world and reality. The application of AR into tourism object’s brochures will facilitate promotions and an introduction about the tourism objects in more attractive and interesting ways.

The aim of this study is to present the mobile application of AR for tourism objects, especially for temples in Bali which is called DewataAR. DewataAR is able to present 3Dobjects and video of Tanah Lot temple by pointing the camera of smart phone to the brochure of the temple.

The rest of the paper is organized as follows: Section 2 discusses about methods and system design of the application. In Section 3, I present the results, the discussion of the application, and also how to use the application. Section 4 is the conclusion.

• II.    Methods and System Design

• 2.1.    Platform

• 2.2.    Android

• 2.3.    Vuforia

DewataAR is created for Android-based system which works with any types of smart phones and tablets. The smartphone is an ideal mobile device for AR because it is completed with build-in camera that enables objectcapturing, GPS, accelerometer and processor that can do high computation. Recent smart phones and tablets are tools for developing the AR application. DewataAR is designed by using Vuforia SDK in its developing. Vuforia is the library software widely used to design AR applications [9]. Vuforia supports markerless technique. Markerless technique is adopted in the development of this application.

Android is an open source operating system for mobile device. It is developed by Open Handset Alliance that consists of software developers, hardware developers, and providers such as Google, HTC, Intel, Motorola,

Qualcomm, T-Mobile and NVIDIA. They aimed to make an open standard for mobile devices. In July 2005 Android was acquired by Google and in November 5 ^th 2007 it was officially released by Google. In its application developing, Android offers Android SDK that provides tools and API for application developers with Android platform. Android uses Java as its programming language [10].

Vuforia is a library or SDK (Software Development Kit) used as the supporter for AR in mobile devices such as Android and iOS. As shown in fig. 1, Vuforia analyzes images using marker detector and produces information, for instance: text, video, 3D object or virtual animation in camera from detected marker by Vuforia API [11].

Fig 1. Vuforia architecture [11].

Vuforia really helps application developers of Augmented Reality in developing applications because basic code of Augmented Reality is provided by library Vuforia which supports iOS, Android, and Unity3D. It supports the developers to make applications which enables in almost every type of smartphones and tablets. Vuforia has markerless feature which is able to use colorful marker without using black frame in the marker. Therefore, users do not need to use conventional marker shaped black and white box [11].

• 2.4.    Markerless

• 2.5.    Natural Feature and Rating

• 2.6.    System Overview

DewataAR is designed by using markerless technique with Vuforia SDK. Markerless is very different with fiduciary marker. According to Madden [12], markerless is an AR used to track objects in reality without special marker. To track objects, AR markerless system depends on natural feature tracking. AR markerless do object tracking with pre-existing image which is then used as a target or an image to be detected to display the virtual objects.

Natural-Feature and rating is a rating system about how good an image can be detected by the camera. The rating can be seen in Target Manager for every uploaded image by developer through Vuforia web. Zero (0) rating indicates that the image target cannot be detected by all AR. However, if the rating has five-star result, then it is easier for the AR system to detect the target image.

Feature is the angle, sharpness and detail, as in textured things [13].

Fig. 2. Illustration of DewataAR.

The user interacts with the DewataAR system by touching menu on the touch screen and moving the brochure. The information will be displayed as augmented scene and it can be seen through the smartphone’s screen. The displayed information can be 3D objects or video, depends on parts of the brochure which is scanned by the camera.

• 2.7.    Development Flowchart

• 2.8.    Designing

Flowchart of application designing is a whole flowchart of the making of this AR application. Started from the preparation of temple’s design, video, brochure design or marker and then integrating Vuforia library and importing 3D marker of tourism object. The flowchart for this application’s designing is shown in fig. 3.

Fig. 3. Development flowchart of DewataAR.

This part discusses about the making of DewataAR application, including steps of the making of 3D object model, video, the making of the marker and the integration of 3D objects in Unity3D application.

• a.    Design of Tanah Lot Temple

The temple is designed by using 3D modeling software. It also uses photos of the tourism object as the reference, then 3D modelling which resembles the tourism object is made so that it can have a similar view with the real object. After the creation of 3D objects, then it will continue with the texturing process as seen in fig. 4.

Fig. 4. Temples texturing process.

The goal of the texturing stage as seen in fig. 4 is to make the object looks more real. In addition, the light effect is given for 3D object with directional light features, hence the 3D object will not appear dark.

• b.    Making video of Tanah Lot Temple

Video of the tourism object is created by using video editing software. The video is created by combining some videos. The next step is rendering process to create brief video which will be displayed in the application.

• c.    Designing the temple’s brochure

The brochure of the temple is designed using image editing software. It is designed in tri-fold format, thus it is more practical and the brochure will be more attractive. Moreover, it is designed so the system in the application will detect the brochure easily. Thus, the augmented object will be displayed well.

Fig. 5 and fig. 6 are the final result of brochure design that will be made into the marker and will be uploaded on Vuforia Target Manager.

Fig. 5. Inside brochure design.

Fig. 6. Outside brochure design.

• d.    Creating Image Target (Marker)

The image target (marker) is created using the brochure design that will be the marker. Then the particular parts of the design will be edited or cropped to make as the image tracker. The result image will be uploaded to the website Vuforia Target Manager and the system will rate the quality of the image using Augmentable Rating as seen in fig. 7.

Fig. 7. Augmentable Rating on brochure.

All uploaded markers through the Vuforia Target Manager will show Augmentable Rating and result a source code as an xml file. The xml file is a configured file of the Vuforia Target Manager to the uploaded markers which then will be downloaded and integrated with Unity 3D in *.unitypackage extension.

• e.    Creating Application in Unity3D and Vuforia

DewataAR, application is created using Unity3D software integrated with Vuforia SDK. The development process of DewataAR application uses C++ programming language.

There are 3 main program codes in Vuforia used to create the AR application for temples as follows:

• •    GUI Button Script

The script functions to display the screenshot button, web search button, autofocus button, help button and exit button. The screenshot button functions to save the displayed image on the application screen. The web Search button is used to search further information about the tourism object in Google. The autofocus button is for adjusting the focus of the smartphone’s camera, while the exit button functions for closing the application.

• •    Trackable Event Handler

The function of the script is to adjust the appeared object if the target image is detected. For every detected image target, the program will display the 3D object or video, appropriate with the previous marker making. Therefore, there will not be any object’s duplication that appear in other image targets (markers) and also objects displayed in every part of the brochure will be different.

• •    Video Playback Controller

The scripts function to control if the detected image target (marker) showing video. If the play button is touched, then the video will be played and if the video currently playing is touched once again it will pause the video.

Preparing components of ARcamera, image target prefab, video prefab from Vuforia SDK and choosing data set of marker previously made with the Vuforia Target Manager.

ARcamera and Image Target prefab from Vuforia SDK are the main component to create the application of DewataAR. Then in fig. 8 shown inspector from data set marker which was previously downloaded was chosen and appeared on the scene.

V Image Target Behaviour (Script)

Script

Type

Data Set

Image Target

Width

Height

Preserve child size

0 IrnageTargetBehavio О

Predefined              1

AR_tanah_lot           *

— EMPTY--

AR_tanah_lot

AR_tanah_lot_2

AR_tanah_lot_baru

AR_ta n a h_lot_update

StonesAndChips

Tarmac

0 Turn Off Behav| Script

Ж (Mesh Filter) Mesh

0 ^ Default Track.

Script__________________

Fig. 8. Choose data set and image target.

Next step, add program codes and insert GUI buttons on application design by adding script components in the inspector window ARcamera as seen in fig. 9.

Fig. 9. GUI buttons on ARcamera.

After that, the 3D object was inserted into the scene, 3D object of the temple was dragged into image target in hierarchy window.

As seen in fig. 18, these are the final process of creating the DewataAR application in Unity3D, the integration process of Vuforia SDK and importing 3D Temple object, temple video and the brochure as marker previously made.

Fig. 10. Developing process in Unity 3D.

Some configurations related with position and size of displayed image, camera position and lighting of selected object will be organized in Unity 3D software.

• III.    Results and Discussion

Below is the result of the test and display of DewataAR, application of augmented reality for temples.

• a.    Instruction to use DewataAR Application

The use of DewataAR consists of several steps as follows:

• 1.    Open the installed DewataAR application in a smartphone.

• 2.    Wait until the splash screen disappears.

• 3.    Point the camera to the brochure so that the camera can recognize pages in the brochure.

• 4.    Interaction can be done by moving the brochure and touching the buttons on the application.

• b.    Splash Screen Display

When the application is started, it will show the splash screen page of the application named DewataAR application. The plash screen will appear for 7 seconds, after that main page of the application will appear, as seen in fig. 11.

Fig. 11. Splash screen.

• c.    Main Display

Main display page of this application is the view where users can point the camera to the marker, as seen in fig. 12.

Fig. 12. Main display of application

Fig. 12 is the main view of the DewataAR application. There are some buttons such as screenshot button, auto focus button, exit button, and help button.

• d.    The Display of Marker’s Detection

In the view, if the brochure or the marker has been detected, the 3D object of the temple will be displayed and the web search button will appear, as seen in fig. 13.

Fig. 13. Application showed 3D object temple.

Besides displaying 3D objects, this application is also able to display videos of tourism objects. If the application is pointed to other parts of the brochure, then the video will appear and can be played by touching the play button as seen in fig. 14.

Fig. 14. Application showed the video.

Fig. 15 shows when a video is played in DewataAR application.

Fig. 15. Application played the video.

• e.    Application’s Testing Based on Marker Scale

The aim of marker scale’s testing is to know the ability of DewataAR in detecting marker in any size. The marker which is used in in image form with .jpg extension. It is obtained from the cropped brochure and make it as the target image. In the trial, the marker which has been reduced from the real size will be tested. The real size of the marker in the brochure is 18 cm X 18 cm with scale 100%. Then the marker or the image target is reduced with scale 50% (9 cm X 9 cm), scale 25% (4.5 cm X 4.5 cm), scale 15% (2.7 cm X 2.7 cm) as seen in fig. 16.

Fig. 16. Various sizes of markers tested.

First test can be seen in fig. 17. The DewataAR application is pointed to marker with scale 50% smaller and the result is DewataAR can detect and display the video well.

Fig. 17. Application can detect marker 50% smaller.

Second test can be seen in fig. 18. The DewataAR application is pointed to marker with scale 25% smaller and the result is DewataAR can detect and display the video well.

Fig. 18. Application can detect marker 25% smaller.

Next in fig. 19, the application is tested with scale marker 15% smaller and the result is DewataAR can’t detect and display 3D objects.

Fig. 19. Application can not detect marker 15% smaller.

• f.    Application’s Testing Based on Distance of The Marker

The aim of marker testing based on distance is to know the ability of DewataAR in detecting marker on any distance. The test is done based on the distance of the camera with the used brochure’s marker.

• •    Distance from camera to marker is 10 cm.

As seen in fig. 20, with distance 10 cm from the camera to marker, the system can detect well and can display 3D objects.

Fig. 20. Distance 10 cm from camera

• •    Distance from camera to marker is 20 cm.

As seen in fig. 21, with distance 20 cm from the camera to marker, the system can detect well and can display 3D objects.

Fig. 21. Distance 20 cm from camera

• •    Distance from camera to marker is 30 cm.

As seen in fig. 22, with distance 30 cm from the camera to marker, the system can detect well and can display 3D objects.

Fig. 22. Distance 30 cm from camera

• •    Distance from camera to marker is 40 cm.

As seen in fig. 23, with distance 40 cm from the camera to marker, the system can detect well and can display 3D objects.

Fig. 23. Distance 40 cm from camera

• •    Distance from camera to marker is 50 cm.

As seen in fig. 24, with distance 50 cm from the camera to marker, the system still can detect well and can display 3D objects.

Fig. 24. Distance 50 cm from camera

• g.    Strength and Weakness System

• 2.    The display of tourism object is more attractive because the images are from the brochure which is made as marker and not fiduciary marker.

• 3.    New innovation for Balinese temples as tourism objects

Based on the analysis conducted, these are strengths and weaknesses of the application:

These are the strengths of the application of DewataAR: 1. User can interact directly with application system, they do not have to use fiduciary marker

These are the weaknesses of the application of DewataAR:

• 1.    The displayed 3D objects do not have moving animations.

• 2.    The displayed 3D objects has no details as much as the real Tanah Lot temple.

• IV. Conclusion

Based on analysis and testing to the study of DewataAR application obtained some conclusions. This study gives new innovation to tourism field and gives new experience for tourists to gain information about visited tourism objects especially temples. The marker’s detection in the application runs well. 3D object and video is able to be displayed on the brochure which is made as the marker. GUI button can works well as its functions. The technology utilizing of AR toward this application works appropriately with the designing, that it is able to combine virtual objects with real environment in real time. In using the application, users must pay attention to distance and focus of camera with the marker. The closer the distance is better for camera detecting the marker. Reversely, long distance between camera and the marker will bother performance of the application.