Detection, Defense, and Tracking of Internet-Wide Illegal Access in a Distributed Manner

Detection, Defense, and Tracking of Internet-Wide Illegal Access in a Distributed Manner

Kohei OHTA <>
Cyber Solutions Inc.

Yohsuke TAKEI <>
Nei KATO <>
Yoshiaki NEMOTO <>
Tohoku University


The community and tools for illegal access are getting well organized and sophisticated. However, widely deployed defense systems like firewalls are moving toward isolating networks and sites. Attackers and intruders can be anywhere on the earth and deploy collaborating agents to attack any site on the Internet. To counter the threat posed, we need to develop technology and framework that allows collaborative defenses, and maybe even offensive measures, against the mischief makers. In this paper, we discuss the necessity and potential of a distributed system for Internet-wide illegal access detection, which employs distributed information collection, sharing, and correlation techniques. We have developed and evaluated a prototype of the distributed application.


1. Introduction

The battle against attackers is getting tougher. To avoid IDSs and other defenses, attackers are using more varied and devious techniques. The latest the latest technique is distributed attacks. Attackers may target simultaneous attacks to a single victim from various points of the Internet. A denial of service (DoS) attack on network bandwidth and/or some network service will be very effective when done by a group of collaborating agents. Toward this end, attackers may use some hosts that have been penetrated or compromised and are located in various points of the network and the globe. Distributed probes are very effective for mapping the topology of a network or sub-network. Topological information is the primary input in devising a strategy to attack a network or some host contained within the network.  The next step is probing for vulnerable services, once again in a distributed manner.

The present widely deployed defense systems like firewalls are moving towards isolating networks and sites. The focus seems to be on protecting the "internal" network.  But there will always be open networking, otherwise the basic philosophy and driving force of the Internet will collapse. There is no escaping from the issue of defending this open Internet from hackers and intruders. Attackers and intruders can be anywhere on the earth and deploy collaborating agents to attack any site on the Internet. To counter the threat posed, we need to develop technology and framework that allow collaborative defenses and maybe even offensive measures against the mischief makers,

In other words, to maintain the rule of law in the Internet, it is necessary to look beyond local defenses, to wide area attack detection and elimination systems including attacker-tracking techniques. In this work, we propose a cooperative system for Internet-wide illegal access detection, targeted toward network exchange points.  It envisages a group of sensitive sensors, the facility of rapid information sharing, and organized reaction.

For the information exchange framework, we are in sync with IDWG (Intrusion Detection Message Exchange Format [IDMEF]) working group [1] of IETF, which is working on format of information to exchange illegal access related information. The IDMEF will enable communication between IDSs in a heterogeneous environment.

In this paper, we discuss the necessity and potential of a distributed system for Internet-wide illegal access detection, which employs distributed information collection, information sharing, and information correlation techniques. We have developed a prototype of the distributed application. It does SCAN detection, attacker tracking, and cooperative defense against attacks. We have evaluated the prototype in an operational network environment and have discussed the results.

2. Recent Internet-wide illegal activities and IDS

IDSs have focused on serious security intrusions like exploit of root accounts. However, that is the line of defense in the war against attackers. For proactive defense and early response, it is important to move the front line farther into from home and closer to the attacker. In other words, the coverage of the IDS should be expanded.

The well-known network mapping tool "nmap"[2] implements various techniques to find services and vulnerabilities in a network, e.g., stealth scanning, or to bypasses packet filters and monitors. Those scanning tools are likely to be distributed both in time and space to avoid detection systems and carrying out very slow scans of randomly selected targets. As an example of a co-operated SCAN activity, distributed traceroutes may be carried out to find out the entry point (router) of an Intranet [3].

DoS attacks are a major issue these days. The signature of a DoS may not seem to be serious locally, but it may affect one or more neighboring organizations and/or ISPs. The well-known smurf[4] attack often traverses several wide-area networks and its potential impact may be felt anywhere in the worldwide Internet. And "Tribe Flood Network" and "Trenoo"[5] are well-organized distributed DoS attacks. They organize and control compromised hosts to make the effect of DoS devastating. CERT Coordination Center published a report on recent cooperated attacks.

There is a report of how smart attackers are collaborating Internet-wide in their attacks [6] [7].

Most of the current security systems are independent, local and concentrating on application/OS log or specialized packet tracing, to investigate attacks on local targets however small the footprints of attackers may be. It is difficult for such systems to have a global view of the relationship of an insignificant abnormality that is observed locally with that that is being seen elsewhere.

To raise the security level of the Internet, it is necessary to develop new security applications based on a standard framework to ensure interoperability and collaboration of IDS, firewall and other security systems.

3. Distributed security system for Internet-wide illegal access

A distributed security system would comprise some security agents and a communication system. Security agents would be IDSs, firewalls, and other management systems, which work on information collection, sharing, and correlation.

3.1. Basic distributed system architecture

Figure 1 shows the concept of our distributed system architecture. Sensors or security agents are deployed on the links of an inter-network, at an appropriate point for traffic monitoring and information exchange.

Security application on distributed framework
Fig. 1 Security application on distributed framework

3.2. Inter-system communication

The requirements [8] and data model [9] are discussed at IDWG to include details for the multi-vendor environment. In this paper, we adopted SNMP and MIB for information exchange. The further details are discussed in [10], [11].

It is important to have an idea of the communications mode(s) that will be used by intrusion detection systems when choosing an intrusion detection alert format, because some formats may support certain modes better than others.

Network management involves monitoring and manipulating information about the elements to be managed.  These bits and pieces of information are abstracted as Managed Objects (MO). A detailed introduction to the current SNMP Management Framework can be found in [12].

3.3. Distributed security applications

In this paper, we propose three types of distributed applications:

4. Distributed SCAN detection

To avoid IDSs, a scanner may send a probe packet sneakily, slowly, randomly, or distributed. In other words, all the effort to do that is how to distribute probes in time and spaces widely. For early detection of these attackers, a wide area must be covered.

4.1. SCAN detection

The basic model of SCAN is based on a one-to-many access model (Figure 2). SCAN detection involves detecting such a pattern.

Typical SCAN behavior
Fig. 2 Typical SCAN behavior

The pattern detection is based on the following categories of information:

  1. Statistical information: Inter access interval and address/port number distributions are basic statistical information. In the simplest case, the probing actions are periodic and address/port numbers are sequential.
  2. Packet type: There are various types of packets for SCAN, e.g., ICMP echo request, TCP SYN, TCP FIN, UDP Echo, etc. Among them, there are obvious protocol violations, and/or suspicious packets like those with all TCP flag set and/or source port set to 65535. SCANs which use such packets can be detected with relatively ease. But SCANs by normal packet are masked by normal activities

Thus, carefully distributed probes with normal packets don't leave enough samples in local scope. In this paper, we propose a distributed information collection and correlation system using reaction packets to detect wide-area SCAN detection.

4.2. Minimal SCAN action

Figure 3 shows a minimal communication model of SCAN. The scanners generate probe packets. They can generate arbitrary packets. For example, TCP-SYN packet, which is a part of TCP 3-way handshake, is normal but it can be used for stealth scan. Reaction packets are automatically generated by network element like routers, firewalls, or end hosts. There is information in the errors (or absence thereof) reported in the reaction packets. SCAN generally attempts exhaustive address-space probing. In such cases, most of the attempts will result in reaction packets indicating "error" fail.

Minimal action of SCAN
Fig. 3 Minimal action of SCAN

4.3. Reaction packets

In this paper, to detect behavior of SCAN, reaction packets are focused. Almost all TCP/IP equipment could generate reaction packets like ICMP destination unreachable [13]. That means those are built-in sensors already deployed. Those good signs should not be lost. However, to make use of them, a distributed security agent must intercept them and report their existence. Figure 4 shows a concept of reaction packet collection.

Reaction packet collection
Fig. 4 Reaction packet collection

4.4. Correlation of information from multiple points

To make a sensor sensitive and make its coverage wide, it is useful to correlate information from multiple security agents. Figure 5 shows a model of distributed SCAN detection, collected information of reaction packets are informed to NMS by SNMP inform message, then aggregated and correlated.

Distributed SCAN detection
Fig. 5 Distributed SCAN detection

5. Tracking the attacker

We proposed a traffic pattern-based attacker tracking mechanism at RAID'99 Conference [14]. That is, using the traffic pattern as the footprint of the attacker; the correlation coefficient is used to correlate and track the footprint.

Packet contents are examined to determine the source, destination, and protocol-related details to determine the traffic profile. However, this mechanism in traffic profiling has limited applicability, as

Moreover, if the source address is spoofed the attack may be detected by whatever means, but still the issue of tracing the attacker remains wide open.

In such cases, profiling traffic-flow characteristics is a more appropriate choice. Essentially, one looks for distinguishing features of the traffic profile. For example, if there is a DoS attack in the form of a flood of TCP-SYN packets, this can be easily detected by the entity receiving the TCP-SYNS or by an RMON type device placed at an appropriate point in the network. Then the source of the malicious packets may be traced by looking for the presence of a similar flow at all the inputs of the concerned network.

5.1. Definition of traffic pattern

Traffic pattern is defined by the sequence of number of packets at time slots at each monitoring point [15], which is modeled by size of time slot d, size of window D and metric of each slot (Figure 6).

Model of traffic pattern
Fig. 6 Model of traffic pattern

All incoming and outgoing traffic patterns are modeled as following vector data.

5.2. Correlation the traffic pattern

Traffic pattern correlation is based on correlation coefficient, calculated as follow. Here, A is incoming pattern, then B is outgoing.

In case thatr(A,B) is close to 1, the traffic A went through as traffic B. One of the advantages of correlation coefficient is that absolute value of the metric can be neglected. The difference of size is represented as amplifier rate as follows. Large value of a means the volume of traffic is amplified a times.

5.3. Path selection of traffic pattern

By correlation of the traffic pattern, we can determine the direction of traffic. In Figure 7, a network is showed as bi-directional graph, so each link can have two patterns at duration, uplink/downlink. When a pattern a observed, which flowed from A to B, the same or amplified pattern might be observed at BD or BC. In such, a has direction. In this paper, a threshold to correlate patterns is value r > 0.7. At that time, the value of a > 2, it represents the traffic was amplified at node B

Direction of traffic
Fig. 7 Direction of traffic

6. Experimental evaluation of distributed system for illegal access

Figure 8 shows our experimental environment. A FDDI loop IX is the target network, which is monitored by a monitor-PC running TCPDUMP. At the same time, all links connected to IX have their own RMON [16] like probes, which monitor the uplinks and downlinks separately. In this paper, all the evaluations are carried out by comparing results with the complete traffic log captured by TCPDUMP. Network IX has 25 networks of academic organizations, A to Y.

Environment of experiment
Fig. 8 Environment of experiment

6.1. SCAN detection

Evaluation was carried out by generating artificial SCANs with random addresses. In one case TCP-SYN probes, which is a popular way for stealth SCANs using "normal" probe packets, were generated. In the other case, UDP probe packets were generated. All probe packets are generated in the network illustrated in Figure 8.

SCAN detection with reaction packet

Table 1 shows a result of HTTP service SCANs with TCP-SYN. There are two types of reactions. One is TCP, the other is ICMP. Totally 1589 probe packets are generated.

Thirty-four TCP SYN/ACK packets are normal behavior from a HTTP server; these probes reveal the server/service existence. This type of probe/reaction pair is normal, making it difficult to monitor the behavior. TCP ACK/RST and ICMP type 3 (unreachable) packets indicate some error condition, probably resulting from some mistake somewhere.

Table 1 HTTP service SCAN with TCP-SYN probe
Scanner -> Target (probe packet)
Protocol Flag Destination Port [Hosts]
TCP SYN 80 1589
Target -> Scanner (reaction packet)
TCP SYN/ACK 51916 34
ACK/RST 51916 92
No response -- 26 [Hosts]
Number of alive hosts 152
ICMP Type Code Name [Packets]
3 1 Host unreachable 53
3 2 Protocol unreachable 1
3 3 Port unreachable 2
3 13 Prohibited by filtering 4

Table 2 shows a result in case of UDP probe. In this case, ICMP could be used to detect trails of SCAN. Probes were generated for 1550 different hosts, and 149 were living hosts. Among other 1401, only 50 ICMP "host unreachable" were observed.

Table 2 DNS service SCAN with UDP probe
Scanner -> Target(probe packet)
Protocol   Destination Port [Hosts]
UDP   53 1550
Target -> Scanner(reaction packet)
ICMP Type Code Name [Packets]
3 2 Protocol unreachable 1
3 3 Port unreachable 108
No response 36 [Hosts]
3 13 Prohibited by filtering 4
Number of alive hosts 149
3 1 Host unreachable 50

In both cases, the number of reaction packets is much smaller than the number of probe packets. This is a result of filtering by firewall. Many routers and firewalls are configured to keep silent to the network about prohibited actions. If they generated a proper message, like "Prohibited by filtering" [16], and a security agent intercepted and collected them, information on scan detection would be enriched.

Information correlation for distributed SCAN detection

To evaluate the ability of distributed cooperated SCAN detection, we collected information on reaction packets from each subnet. In this experiment, about 1500 hosts were scanned with random address, distributed over 118 subnets.

Figure 9 shows an effect of distributed cooperated SCAN detection. Each mark indicates the detection of some reaction packets on time-axis. The upper orange marks represent observed reaction packets on a subnet segment. The lower blue marks represent all packets detected and collected from all subnet segments. It is clear that the orange marks are too few to detect SCAN characteristics; but the aggregated blue marks provide a rich source of information. The result clearly shows that widely distributed SCAN behavior is nicely and effectively aggregated by distributed sensors.

Co-operated SCAN detection
Fig. 9 Co-operated SCAN detection

6.2. Tracking smurf attack

We evaluated our proposed pattern-based attacker tracking in an operational network for a period of six months. A typical smurf attack consists of three components (Figure 10):

In this experiment, we tried to correlate and track that traffic.

Typical smurf attack
Fig. 10 Typical smurf attack

Totally 85 smurf attacks were detected, and all of them could be tracked. But two attacks resulted in ambiguous tracking results. (Table 3).

Table 3 Tracking result
Number of smurf attacks 85 certain 83
uncertain 2

Table 4 shows a sample result of culculated correlation coefficient/amplifier rate. The highlighted line represents correlated patterns, which means traffic patterns from Network-M to Network-O are correlated, the similarity r = 0.99 and amplifier rate a = 0.96.

Table 4. Propagation of ICMP echo requests

Propagation of ICMP echo requests

Table 5 shows similarity and the rate of amplifier between inbound and outbound of Network-O. The traffic to Network-O amplified about 24.6 times and went out from Network-O.

Table 5 Traffic amplifier

Traffic amplifier

Table 6 shows the same values as Table 4, which represents from Network-O to Network-M.

Table 6. Propagation of ICMP echo reply (amplified)

Propagation of ICMP echo reply


Totally, these above examples represent a successful traffic pattern tracking, which is a result of traffic pattern correlation obtained at four independent monitoring points illustrated in Figure 11.

Information correlation example
Fig. 11 Information correlation example

6.3. Cooperated defense

Table 7 shows a summary and estimation of influence of above 85 smurfs. ICMP echo reply packet of smurf attack is generally high volume, and goes across many networks all over the world. The table shows a large number of bursty and persistent traffic was generated by smurf attacks and many networks and hosts are related. Especially, the last row is an estimation of number of AS-hops between the site used for packet amplification and the victim. The estimation is based on Internet Routing Registry (IRR)[17]. If related sites did share the information, they could certainly defend their resources effectively.

Table 7 Influence of smurf attack
  Minimum Mean Maximum
Number of packets used for attack 2469 1178737.8 7650056
Elapsed time (min.) 1 32.2 637
Number of packets/minutes 2469 47985.9 94244.7
Rate of packet amplification 11.8 23.6 26.5
Number of networks used for amplify 1 3.4 30
Number of hosts replied 25 255 56.9
Number of related ASs 2 4.5 7

6.4. Combination with Network MAP

Attacker tracking across several ISPs or ASs requires connectivity information; we used the information from IRR. Figure 12 shows a demonstration of attacker tracking with Internet MAP. A mechanism for sharing connectivity information and rapid intrusion information results in an effective proactive defense system.

Tracking attackers on the MAPTracking attackers on the MAP
Fig. 12 Tracking attackers on the MAP

7. Conclusion

Currently, administrators and mangers of network are coordinating, if at all, at the human level.  Thus, the propagation of security information is slow and constrained. Furthermore, if the administrator/manager does not have enough knowledge and skills, appropriate reaction may not happen. An automatic and standard framework for rapid information sharing and coordinated actions is necessary to supplement that of the manual activities. In this paper, we proposed a distributed system for Internet-wide illegal access detection. We have discussed its effectiveness in SCAN detection, attacker tracking, and cooperative defense. We have carried out experiments with a prototype in an operational network environment. The results establish the effectiveness and potential of distributed systems.


  1. "Intrusion Detection Exchange Format," work in progress,
  2. "NMAP -- The Network Mapper,"
  3. Stephen Northcutt, Network Intrusion Detection: An Analyst's Handbook, 1999, New Rider.
  4. CERT Advisory CA-96.21: "TCP SYN Flooding and IP Spoofing Attacks," 24 September 1996.
  5. CERT Advisory CA-2000-01: "Denial-of-Service Developments," 3 January 2000
  6. CERT Coordination Center, Result of the Distributed-Systems Intruder Tools Workshop, Pittsburgh, Pennsylvania USA, 2-4 November 1999.
  7. Torsten Busse, "Hackers team up for large-scale attacks," Computerworld, September 1998
  8. M. Wood,  "Intrusion Detection Message Exchange Requirements,", work in progress, October 1999
  9. Herve Debar, Ming-Yuh Huang, David J. Donahoo, "Intrusion Detection Exchange Format Data Model," work in progress,, October 1999.
  10. Glenn Mansfield, Dipankar Gupta, "Intrusion Detection Message MIB," work in progress,, 23 October 1999.
  11. G. Mansfield, D. Curry, "Intrusion Detection Message Exchange Format Comparison of SMI and XML Implementations," work in progress,
  12. J. Case,  R. Mundy, D. Partain, and B. Stewart, "Introduction to Version 3 of the Internet-standard Network Management," RFC 2570, April 1999
  13. J. Postel, "Internet Control Message Protocol," RFC 792, September 1981.
  14. Glenn Mansfield, Kohei Ohta, Y. Takei, N. Kato, Y. Nemoto, "Towards trapping wily intruders in the large," Recent Advances in Intrusion Detection (1999).
  15. Kohei Ohta, Yohsuke Takei, Nei Kato, Glenn Mansfield, and Yoshiaki Nemoto, "Synchronizing Management Information Using Traffic Pattern Matching Technique," Proceedings of 1999 Symposium on Performance Evaluation of Computer and Telecommunication Systems, pp. 349-354 (1999).
  16. F. Baker, "Requirements for IP Version 4 Routers," RFC 1812, June 1995.
  17. Glenn Mansfield, et al., "Network Maps: Synthesis and Applications," Proceedings of 1999 Asia Pacific Symposium on Information and Telecommunication Technologies, August 1999.