A Mobile-Based Computer Controller via Android Technology

Published Online February 2013 in MECS

Nowadays, besides using computers for working purpose, most of the users use their computers for entertainment purposes such as watching a movie, sharing photos, browsing music or playing games. Sitting on a particular spot is never a fun especially when viewing an entertaining media. It can be very impractical to be confined to the keyboard and mouse while sitting 5 or 10 feet from the computer. Therefore, remote controls or wireless mouse and keyboards are also available for computers to solve the limitation mentioned above. However, these remote controls or wireless keyboard and mouse have a fixed set of buttons attached to the device it has been attached and designed to control. A fixed distance from the computer would be another concern of using the existing remote controls or wireless keyboard and mouse. When sitting far from the computer, the user is unable to view clearly the items on the computer screen, which would limit them in controlling the computer.

The rest of the paper is organized as follows. In Section 2, some of the related Android applications are briefly reviewed. The proposed system design is discussed in Section 3. In Section 4, the implementation process of the proposed system is explained. The testing scenario and result are reported in Section 5 and finally, the conclusion is given in Section 6.

• II.    Related Issues

  Android platform has made everything so easy and achievable in its environment. In line with a research from Gartner [4], Android is poised to become the second worldwide mobile operating system by the end of year 2011. There have been lots of competitions in

the Android Market. The development and improvement of an existing Android innovation has become a usual thing now in the Android Environment. Every Android developer wants to design or create a unique application that has a lot of improvement and advancement towards the existing applications. A developer who has just built an application might end up seeing more of the application just in a couple of months. Due to the fact that Android is an open source, it has inevitably given Android developers the chance to improve existing work and as well create what is currently invoke in the market trend. As regards to this project’s aim, there have been existing works related to the proposed application which has been in the market since last and earlier this year. Below are some of the related existing Android Remote applications, with a brief outlines of their specifications.

• a.    Gmote [5]: This is an Android remote application that has been used to control a VideoLan Client (also known as VLC) media player, with basic options such as play, pause, stop, forward track and backward track. Its advanced feature is the file browser that allows it user to pick and choose what to play and also a recently added feature of being able to play some of the media files directly on the phone as opposed to watching it on the computer. However, this remote application has its limitation in controlling one program.

• b.    myRemote [6]: This application is very simple and easy to use, easy to set up, and it is an excellent control for Home Theater PC (also known as HTPC). This has been rated the best Remote control for Windows Media Center, compared to other related application of its kind. The only limitation for this application is the poor skin design.

• c.    gPad [7]: This application allows users to define custom remotes via key strokes. Its limitations are only by using key strokes and key combinations, so application functionality has been limited. The implementation of gPad does not support a two way communication, i.e. the sending of data back to a user.

• d.    RemoteDroid [8]: This application performs as a wireless mouse and keyboard. It controls the touch screen on the phone into three sections, just like a touch pad on a laptop. This is very similar to what the proposed application is trying to achieve. But the drawback of this application is the user would need to memorize the application short cut keys if they want to perform functions greater than typing and browsing through a particular media file. No custom buttons or controls are currently available.

Table 1: Comparison of Existing Remote Applications

Remote Application	Secure Network	3G/Edge	Pinch Gesture	Operating System
Gmote [4]	No	GSM	No	Windows
myRemote[5]	No	GSM	No	Android
gPad [6]	No	GSM	No	Android
RemoteDroid [7]	No	GSM	No	Android
Proposed Application	Yes	3GZEdge	Yes	Android/Windows/ Mac iOS

Table 1 shows the comparison of existing remote applications in the market. There are also other remote control applications either from Android platform or other platforms that are available for different mobile phone devices like Windows Mobile and other Apple applications. The analysis inspired the idea of designing a hand phone based computer controller which is able to play VLC Media plays, to control HTPC and so on. Moreover, the proposed prototype is able to perform click operations, multi-touching, multi-scrolling, zooming and pinch gesture. In addition, users can as well use the keypad function which works exactly as the normal computer keyboard where user can type to chat, write emails, search things and others.

• III.    Proposed System Design

The aim of this application is to address the stress of an uncomfortable moment with the use of computer. It enables the user to search for files, click on the desktop applications, play with media such as forward, rewind, pause, run, increase or reduce the volume of a media file. It also allows the user to do multi-touch scrolling and pinch gesture for zoom in and out. This could be achieved by installing the proposed application that emerges to the computer network via Wi-Fi which then connects both system together, then comes with the command from the mobile phone that remotely controls the computer. The system architecture is shown in the Fig. 1 below.

Fig. 1: System Architecture

Fig. 2: Client application flow diagram

This section presents two important factors of the prototype: the client and server applications. Furthermore, the major components for this proposed application and some of the important algorithms are elaborated.

• 3.1    The Client Application

• 3.2    The Server Application

• 3.3    The Prototype Program Flow

There are essentially two ways to deliver an application on Android: as a client-side application developed using the Android SDK and installed on user devices as an .apk or as a web application developed using web standards and accessed through a web browser [9]. Here in this project the client application was implemented. Based on the platform using Android Software Development Kit [10], this application was written using the Java Programming Language and was designed to run on a customized Linux-based virtual machine called Dalvik. The developed client application has an .apk extension the interested users can download from the developer and installed directly to their mobile phone devices. Fig. 2 shows the program flow diagram of the client application.

First, the application program starts by checking the Wi-Fi connection. The embedded web server runs in parallel once the application starts. The application pulls the xml file from the .apk file to generate the main screen and moves ahead to generate also the application Graphical User Interface (also known as GUI). The application GUI awaits an input from the user and when the action has been performed, it sends a command to the Web Server which then processes the action sent from the GUI. Then the Web Server resends back the action that has been processed to the Application GUI which then updates the input back to the user as expected.

Today’s mobile phones are having more computing power, running a server on mobile phones can do a lot more help in running the device. In the project the server works as a pair to the client. For without the server which transmits the message to the listener, the application itself would not be able to function. To create a typical web server, one has to create a server socket and listen to the desired port. Then accept the connection and finally process the request and send the response to the client.

For the server side of this application, a java server application is required for listening to client activities. In this project the server is a private java application server which has been hosted by the application’s machine. The application’s machine can also be any machine where Java Runtime Environment (JRE) is installed. Fig. 3 shows the program flow diagram of the server application.

Fig. 3: Server application flow diagram

Fig. 4 shows the overall program flow for this application. When the remote control is run, it follows the path shown on the figure below. After application starts, the embedded java application server runs in parallel. Sound notification is implemented in the proposed application so as to let users being aware that their IP address has been validated and the user can proceed with the application.

Fig. 4: Prototype program flow diagram

• 3.4    Android Software Development Kit (SDK)

• 3.5    Eclipse with ADT Plug-in

• 3.6 Important Source Codes from ClientApplication

The Android SDK is a Java based software development kit that enables the developers to build applications for the Android platform. It includes development tools, source codes for sample projects, an emulator and the necessary libraries to create Android applications. Applications are run on Dalvik which is a custom-built virtual machine that runs on top of a Linux Kernel [10]. The graphical user interface of an Android application is built from the libraries in the SDK using XML and the graphical input. This ensures that the scheme created by the developer is uniform for the application and between applications. In an Android application the user interface is created using View and ViewGroup objects. View objects are the fundamental units of user interface expression on the Android platform [9].

On Android, permission is required to grant the built application to access to the internet in performing certain tasks. The developer must define it in the configuration file of the application that it would like to use the concerned permission. If a user wants to install an Android application, he or she gets a screen where by the required permission for the application must be confirmed.

One of the requirements for developing a software project is the implementation of the Integrated Development Environment (IDE). There are different IDE available, which some of them are free, and some might need a subscription fee before accessing. For Android developers, Google has provided the ADT Plugins for eclipse so as to have an easy access in integrating Android applications developments. Eclipse also comes with free multi-platform IDE which provides easy portability and compatibility.

In this section the codes for each major activity in the client-side application will be shown. All the source codes are in Java programming language and are run by Eclipse.

public boolean onTrackballEvent(MotionEvent ev) { if (ev.getAction() == MotionEvent.ACTION_DONN) {

// send CTRL down

Log. d(TAG, "track down");

Object[] args = new Object[3];

args[0] = 0; /* key down */

| args[l] = 60;

args[2] = "CTRL";

} catch (Exception ex) {

Log. d (TAG, ex.toString());

} else if (ev.getAction() == MotionEvent .ACTION_UP) {

// send CTRL up

Log, d(TAG, "track up¹');

Object[] args = new Object[3];

args[0] = 1; /* key up */ args[l] = 60;

args[2] = "CTRL";

} catch (Exception ex) {

Log. d(TAG, ex.toString());

(a)

public boolean onKeyDown(int keycode, KeyEvent ev) {

//allow menu to show

//if (keycode == KeyEvent.KEYCODE_MENU)

И return super.onKeyDown(kgy^k> gy);

if (keycode == 58) { // right (^ this.toggleButton = true;

return false;

II Log.d(TA6, ^ksydeMny'+String.valueOftkeyc^

Object[] args = new Object[3];

args[0] = 0; /* key down */ args[l] = keycode;

args[2] = new Character(

} catch (Exception ex) { Log. d(TAG, ex.toString());

return true;

(b)

boolean scrollTag = false;

int pixelsPassed = 0; //The number of pixels required to move before int gestureDetected = Gesture.NO_GESTURE; //No gesture detected yet float lastPinchDelta; //The last user pinch to zoom gesture distance public class Gesture { public static final int NO_GESTURE = -1;

public static final int PINCH_TO_ZOOM = 0;

public static final int SCROLL = 1;

} int scrollCount = 0;

int rightClickAllowance = 1; //scroll iterations before skipping Rig private boolean onHouseMove(MotionEvent ev) { int type = 0;

float xMove = 0f;

float yMove = 0f;

int pointerCount = 1;

if (mlsMultitouchEnabled) j pointerCount = WrappedHotionEvent.getPointerCount(ev);

Fig. 5: Important source codes in the proposed application:

• (a)    TrackBall Event Activity;

• (b)    Keyboard Event Activity;

• (c)    Mouse Event Activity.

• IV. Implementation Process

• 4.1    Prototype Functionalities

The main purpose of the development of this prototype is to help the users remotely use their mobile devices as an ease to access their computers via a limited distance. This application is said to increase the level of relaxation, save cost, and increase the convenience in controlling the computers.

(c)

There are four major functions that can be performed by the installed client-side application on the device. They are;

• a.    I.P Address Login Page

• b.    Touch Pad Layout Screen Page

• c.    Keypad Controller Page

• d.    Menu Page (Preferences/Help).

And on the server-side application which will be installed on the computer. There are two major functions.

• a.    User has to install the server application on his/her computer.

• b.    The server application installed should then be run as required.

• 4.2 Prototype Graphical User Interface

• a.    IP Address Login Page

Fig. 6: IP Address Login Page

Fig. 6 shows the IP address login page, where users are required to key in the IP address as appeared on the servers interface screen. Then, the connect button for connection between both machines is clicked so as to generate the desired output. One of the extra features of this application is introducing a sound/voice notification that allows the user to know whether the connection is successfully connected and is ready to use.

• b.    Touch Pad Layout Screen Page

Fig. 7: Touch Pad Layout Screen Page

Once the client application on the Android device connects to the server application on the computer, the screen on the Android device switches to a touch-pad layout as shown above in Fig. 7. The large rectangle at the top is the mousing surface and the two smaller rectangles at the bottom are the left and right mouse buttons. The green keypad icon activates the keyboard. Here the users are required to play along as implemented. The activities to be carried out here on the remote touch-pad screen are: Multi-scrolling, Multitouching, Pinch-gesture and users can as well use the touch-pad mouse button, if the user does not prefer the multi-accelerator features. And also between the left and right mouse buttons is the keypad’s icon for typing purpose.

• c.    Keypad Controller Page

Fig. 8 shows the keypad controller. This keypad screen requires inputs from the users. It works exactly as the keyboard on our computers. To use the keypad, the users have to tap on the keypad’s icon and the keyboard whole screen will appear immediately. The keypad’s icon is just located between the right and the left touch-pad buttons.

• d.    Menu Page (Preferences/Help)

Fig. 9: Preferences Menu Page

Fig. 9 shows the preferences menu page. Here users can adjust the mouse sensitivity, tap-to-click and trackball mouse wheel by pressing the Menu Key at the iRemotePad screen before connecting. The trackball preference only applies if the user happened to have a device with a trackball. Also, multitouch scrolling is supported here. To use the multitouch function, the user needs to drag two fingers on the mousing surface to scroll. The help menu is included with the guide on how the application works and how users could install and run the client and server applications, as well as with the Frequently Asked Questions (FAQs).

• e.    iRemotePad Server Application

й|^ iRemotePad Server R2

The iRemotePad server application is now running.

Your IP address is: 192.168.1.7

Enter this IP address on the start screen of the iRemotePad application on your phone to begin.

Fig. 10: iRemotePad Server Application iRemotePad Server application screen on the computer is shown in Fig. 10. It shows that the server application is running with the IP address to be entered at the first screen of the application on the phone, so that they are connected.

• V.    Result

Several software testing are conducted to test the validity and reliability of the application. Unit test is performed on individual module to evaluate whether they fits the intended purpose. After the individual modules pass the unit test, the individual modules are combined and integration test is carried out to verify the functions, performance and reliability of the application. The test case scenarios are showed in Table 2.

Table 2: Test Case Scenario

Use Case	Functions Being Tested	Inputs	Expected Outputs	Result
Applicatio n Running	Start Client Application on Mobile Device	User inserts IP Address and clicks on the connect button	The IP is valid and is then connected to server.	Good
Applicatio n Running	Start Multi-scrolling and Pinch Gesture	User applies two fingers on the touchpad screen	The command is successful. Both multi-scrolling and gesture works well.	Good
Applicatio n Running	Start up the keypad function	User taps on the keypad icon	The keypad icon pops out a keyboard frame that contains all alphabets, numbers and symbols and is ready for use. User tries to type some letters, numbers and symbols and it works fine.	Good
Applicatio n Running	Start up the left and right mouse buttons which are located at the bottom part of the touchpad	User taps on either the left or right mouse button	The mouse buttons on both sides becomes green when been tapped indicating its functioning properly.	Good
Applicatio n Running	Start-up Menu icon	User taps on Menu icon to access the preferences menu	The preferences menu includes the tap-to-click, tap-speed, mouse sensitivity, scroll sensitivity and invert scroll direction labels. They all can be adjustable and needs to either be enabled or disabled. Every of these events are working properly as tested.	Good
Applicatio n Running	Start-up Menu icon	User taps on Menu icon to access the help menu	The help menu is a guide to how users can install, run and use this application. The help menu also includes a detail of how the applications works and also its FAQ. For users to be able to access this function, they will have to go to the applications Webpage, which is written on the help menu.	Good

Other than this, an online survey has been conducted to collect the feedback from the public regarding to the usage of the mobile phone as a controller on the computer or laptop via wireless network. This survey has received responses from 51 potential users all over the world. Most of the participants own a mobile phone and 94% of them have used the mobile application which shows that mobile application is getting popular among the mobile phone users.

In this survey, 52% of participants have heard about the Android remote control application and 63% of the participants are satisfied with the overall performance of the application. Most of the participants pose a positive attitude on the application and 89% of them feel comfortable with the idea of preferring to use the device. Besides, 35% of them are willing to try out the application even when they have no android device and 40% of them are as well willing to continue in the advancement of the project.

• VI.    Conclusion

This project explores the possibility of controlling the computer remotely using an Android phone device. The proposed prototype is able to control a lot of operations a normal computer keyboard and mouse would perform. It practically turns a mobile phone into a wireless keyboard and mouse using a wireless network via a portable mobile device running under an Android Platform Operating System. It helps mobile phone users on facilitating their work in study life, home life or working life, where the use of the prototype helps in easing the device control.

It is proven that this prototype would relieve a pain in the neck and also the normal back ache due to constantly sitting at a particular place. With the help of this prototype, these stressful moments will be minimized as users will be having a very relaxed position as intended. This is a convenient application for simple operations and for manipulating such computer without the keyboard and mouse been connected.

Symposium on Haptic Interfaces for Virtual Environment and Teleoperator (2005).

• [4]    Gartner. Gartner Newsroom: Press Release 2011 [Online]. (2011).

• [5]     http://www.gmote.org/ (Accessed 16 September 2012)

• [6]     http://sites.google.com/site/myremoteandroid/ (Accessed 16 September 2012)

• [7]     http://www.gpad.mobi/content/en/home (Accessed 16 September 2012)

• [8]     http://www.remotedroid.net/    (Accessed    16

September 2012)

• [9]    Android Developers, Android Application Framework: A Quick Guide [Online]. (2010).

• [10]    Webopedia, Introduction to Mobile Devices: Android SDK [Online] (2011).

Siew-Chin Chong: Lecturer of Faculty of Information Science & Technology in Multimedia University, majoring in biometrics and pattern recognition and is interested towards mobile applications.

Lee-Ying Chong: Lecturer of Faculty of Information Science & Technology in Multimedia University interested in biometrics and pattern recognition.