Auto-Pattern Programmable Kernel Filter (Auto-PPKF) for Suppression of Bot Generated Traffic

Published Online November 2013 in MECS DOI: 10.5815/ijcnis.2014.01.07

Bot is a malicious piece of software derived from the word “ROBOT”, which when installed in a host makes it a ZOMBIE machine. Botnets are a group of distributed bots controlled by a master computer often referred to as botmaster (attacker). Protocols such as Internet Relay Chat (IRC), Hypertext.

Transfer Protocol (HTTP), Peer to Peer (P2P) are exploited for the communication and propagation of bots. Bots usually communicate with their botmaster to receive commands from them. Based on this behavior, bots can be categorized into the following types:

IRC Bots: IRC, being a protocol designed for real time chat communication, has been exploited by attackers for achieving stealthy communication. In botnet scenario, the remote Botmaster in order to communicate with the zombie machines joins the specific IRC channel through a command and control structure. Detection of IRC bots is done at network level by observing various parameters such as High Traffic rate, large/Jumbo sized packets, long lived Communication using TCP ports, and by also looking in for commonly used IRC commands [1].

HTTP Bots: Bots have been designed to use HTTP for their communication. The advantage of using HTTP for implementing command and control is that almost all firewalls allow the HTTP traffic without blocking the port 80, since HTTP comprises a majority of Internet traffic. The detection of such kinds of bots is done by observing periodic repeatability in communication by checking the network traffic flow for long lived connections and then correlating the DNS queries for detection of abnormal communication pattern [2].

P2P Bots: Bots make use of the decentralized structure of the P2P protocol for communication. Initially infections are sent to victim as Trojan horse or through any other malicious programs. Then further secondary infections are spread to the Zombie machines through the P2P network. A case study on Peacomm bot shows that the infections are spread through the distributed hash table based on the kademlia algorithm. Detection of P2P bots is done based on the hash keys used [3]. Other existing solutions at kernel level are based upon filtering of packets using predefined signatures formed by analyzing the packet payload [4].

The rest of the paper is organized as follows. The related work carried out on bots is discussed under Section II. The Existing Detection techniques are discussed in Section III, followed by motivation for the proposed work in Section IV. Section V discusses the proposed solution along with the proposed algorithm and its working. Discussions on the Experiments conducted viz., Bot Attack Generation, outcomes of the Experiments and performance analysis have been outlined under Section VI. Section VII includes the concluding remarks on the work done.

• II.    RELATED WORK

The traditional Pattern matching algorithms such as, Boyer Moore, Knuth Morris Patt, and Naïve pattern search algorithms require the pattern to be identified available a priori [5]. Rabin Karp string matching is based on hashing. But hashing at kernel level is an expensive operation [6]. The Hamming distance and levenshtein distance [7] between two strings are also currently used for pattern matching. Hamming distance requires that the strings to be compared should be of equal length and levenshtein distance results in approximate string matching. Longest Application Signature ExtRaction (LASER) algorithm exists for automatic signature generation of patterns using packet payload. But the disadvantage is that common sequences can also exist in many genuine packets which may lead to buffering of genuine packets resulting in increased overhead [8]. Honeycomb, which generates automatic signatures from packet payload uses Longest Common Substring (LCS) algorithm. Traffic is sorted on a per connection basis depending on the protocol traffic, port used, and then the signature is generated from the packet payload. Signature pool is maintained for comparing the signatures of the new packets against the existing signatures [9].

• III.    EXISTING TECHNIQUES

Botmaster makes use of the fast flux technique, viz., playing hide and seek with the Zombie machines, inorder to avoid the traceability of the origin of the attack. This fast flux technique refers to the rapid change in the IP address/domain name of the attacker. This change in identity of the attacker is further categorized into two types: Single flux and Double flux, wherein the attacker keeps changing the IP address with respect to one domain name and both the IP addresses and the domain name respectively. Existing detection techniques for such kinds of bots include Monitoring of Domain Name System (DNS) traffic [10], analyzing the network characteristics and performing data mining on them [11]. Identification of such kind of malicious bots in a network can be done by examining the pseudo randomly generated domain names [12]. Signatures of both the genuine programs and malicious programs are maintained as two separate lists namely white list and Black List. Agents are deployed for the detection of Botnets using MultiAgent Technology [13] for monitoring traffic and the registry key changes. Based on these changes and correlation of connection details such as name of the process, protocols used, and ports used at a particular instant, the attack scenario is detected. Application specific malicious traffic identification [14] is also possible, where signatures of specific applications are obtained based on parameters such as packet payload, and packet flow information, viz., source address, destination address, protocol, port number, and inter arrival time of packets. Signatures are formed by taking a sequence of bytes from packet payload. Generation of datasets [15] and creating profiles for attack detection using machine learning techniques has gained momentum and is being used for detecting intrusions.

• IV.    MOTIVATION

The new algorithm for automatic detection of patterns and filtering of malicious packets at kernel level is an extension of our previous work on “Pattern Programmable Kernel Filter (PPKF)” for bot detection [16]. In PPKF, HTTP packets were checked for a predefined pattern which is known a priori at kernel level. In this paper an algorithm namely Auto-PPKF is proposed where the malicious pattern is deduced on the fly, by inspecting the packet payload at kernel level and then passed to PPKF for further processing. In a botnet scenario, the regular communication of similar/same data between the botmater and the Zombie machine to maintain its connectivity is considered to be malicious.

• V.    PROPOSED SOLUTION

In this paper a solution for automatic detection of patterns at kernel level has been proposed. Fig. 1, shows the block schematic of the proposed solution. In Fig. 1, notations A1, A2, … , An refer to the different applications in user space. The automatic pattern detector identifies the pattern by buffering the packets at kernel level to check for similarity among them. After deducing the pattern and confirming it to be malicious, they are updated in PPKF for comparing the forthcoming packets against them. If the incoming packets contain a pattern as in PPKF they are considered malicious and are blocked.

Figure 1: Block Schematic of the Proposed Solution for Bot Detection

• A. Auto-PPKF Algorithm

• • EXTRACT PATTERN o GET Next Packet

o Check for Genuine Packet (Admin IP address)

IF True

• ■    Genuine Packet

• ■    Allow Packet

• ■   IF End of Set COMPARE

PATTERN

Else GET NEXT PACKET

ELSE Suspected Malicious o   Extract Features o  Update Pattern

IF End of Set

• • COMPARE PATTERN

ELSE GET NEXT PACKET

• • COMPARE PATTERN

o GET Packets o Compare Packets

IF SIMILAR o Update Common Pattern o Block Packet

REPEAT:      IF NOT End of List

• ■   COMPARE PATTERN

ELSE IF NOT End of Set

• ■   COMPARE PATTERN

ELSE Allow Remaining packets

• ■    EXTRACT PATTERN ELSE REPEAT

• B.    Working of the proposed algorithm

The algorithm works on a set of N packets captured at kernel level. Initially a set of captured N packets are subjected to remote address verification. If the remote address corresponds to the Administrator IP address, then the packets are considered genuine and are allowed to the network. Else the packets are sent to the suspected malicious packet list (SM). Packet features such as Remote address, Packet length, and the Packet data are extracted from the packets in the SM list. Then the Compare Pattern module uses this set of N packets. The value of ‘N’ was set to thirty in our experiment which is a settable parameter. In Compare pattern module, the packets are checked for similar patterns. The packets are searched byte by byte in a brute force fashion to check if they contain similar pattern. If the pattern is found to be similar then, this pattern is taken as the malicious pattern generated by the bot and it is updated in the Common

Figure 2: Finite State Automata for the proposed solution

IN: Initial State

GEN: Genuine Packet

ALLOW: Allow Packet

EoS: End of Set

COMP: Compare pattern Exists

SM: Suspicious Malicious

FE: Feature Extraction

UP: Update Pattern

NC: No Common Data

Pattern

BLOCK: Block Packet

EoL: End of List patterns

CP: Common Data Pattern

UCP: Update Common

Pattern (UCP) list. If there are no packets having common pattern and the comparison is over, the remaining genuine packets are allowed into the network. Then, the processing (checking for genuinity and feature Extraction) on next set of (N) packets commences.

• VI.    WORK DONE

• A.      Bot Attack Generation

Bot attack was generated with the Spy Eye Exploit kit. The connector interval time for the bot to report to its master was set as one minute and the malicious build file was generated, which was sent to the victim as an E-mail attachment. Traffic was captured using wireshark and continous exchanges of similar reporting packets between the botmaster and the Zombie machine were observed. Simultaneously genuine web traffic was generated with five tabs (five webpages containg text as well as pictures, viz., , ,       www.youtube.       com, , open in the browser window and the traffic was captured for a period of half an hour.

• B.      Discussion on outcome of Experiments

The developed filter was loaded using a GUI namely Open System Resource (OSR). The filter can also be started and stopped using command prompt which may require making few registry changes manually. Changes being done to the existing WFP filter on loading the developed filter, were observed through event viewer. During the attack period a continuous exchange of ‘HTTP GET’ reporting packets between the bot master and zombie machine were noted. Genuine web traffic (five webpages containing text as well as pictures, viz., , , www.youtube. com, , , as generated before loading the developed filter was generated and the traffic was captured using wireshark for the same period of half an hour. From the captured results it was found that the similar malicious HTTP GET packets between the botmaster and zombie machine were completely filtered allowing the rest of the genuine packets out of the network.

• C.      Performance Analysis

The performance of the developed Auto-PPKF was monitored using wireshark. Fig. 3 and Fig. 4 show the

Communication pattern between botmaster and Zombie machine before and after loading the filter respectively. The graph in Fig. 4 when compared to that in Fig. 3 shows the steep decline in communication pattern between the botmatser and Zombie machine after loading the filter. Fig. 5 and Fig. 6 show the HTTP Load Distribution between botmaster and Zombie machine before and after loading the filter respectively. From Fig. 6, it is seen that the HTTP load distribution between the botmaster and Zombie machine is Zero after loading the filter, thus suppressing the stolen user information from being posted on to the Botmaster.

• VII.    CONCLUSION

Bots being active malwares of today are propagating on every network used including the mobile network. So it is high time to come out with a solution to suppress the malicious communication of bots. Keeping this in mind an algorithm for automatic detection of malicious patterns from packet payload for filtering out malicious packets generated by bots at kernel level has been proposed, in this paper. The proposed solution was deployed in a bot attack scenario and it was found to work effectively. The proposed pattern matching algorithm can also be extended for the detection of other malicious programs such as worms which follow a particular communication pattern. The ongoing work is the identification of malicious process at host level and killing of the malicious bot process. Since SpyEye Exploit kit was recently released and not much research solutions were found addressing it, it was chosen as the demonstration environment for testing our proposed solutions for bot detection.

ACKNOWLEDGEMENTS

The authors thank the National Technical Research Organization (NTRO), New Delhi, Government of India for sponsoring this research work under the Collaborative Directed Basic Research in Smart and Secure Environment Project.