Monitoring Suspicious Network Events on a FreeBSD Server

Abstract

This paper looks at daemon programs and kernel modifications to a FreeBSD server which monitor and log these suspicious network activities. A summary and review of the last seven months of log information will be given for the FTP and WWW server - minnie.cs.adfa.oz.au.

1 Introduction and Motivation

There are many tools available for network administrators to discover possible weaknesses in a server's network security - SATAN and Strobe are two examples. These tools are just as useful to would-be intruders. `Underground' FTP and WWW sites also have tools to perform attacks such as IP spoofing, shared-library attacks, and to exploit known holes in such programs as sendmail and telnetd. Existing operating systems do not detect the use of these tools and methods, nor do the attacked systems let the network administrator know of their use. CERT advisories describing these attacks can be found on the CERT and AUSCERT FTP servers at ftp://ftp.cert.org/ and ftp://ftp.auscert.org.au/.

Over the past few years I have been running a small FTP and WWW site to provide information about FreeBSD and NetBSD. I've also been involved in setting up an Internet firewall at ADFA. After reading several papers and books about network security and firewalls, I quickly realised how vulnerable most Internet systems are. As part of the security-tightening on my own server, I was motivated to modify the source code of the server's kernel to detect attempts to connect on unused network ports, and to reject packets with `suspicious' options. These modifications allow me to observe many of the attacks currently in use and may also highlight those services which intruders believe to be vulnerable.

I've also modified my server to prevent IP and TCP spoofing attacks and to log incoming packets for services such as TFTP, NFS and RSH. Although none of these have been triggered yet, they should provide more information about the intentions of a would-be intruder.

2 Server Description

• Anonymous ftp to 500M of files, using WU-FTP 2.4;
• Web service to 100M of files, using Apache 0.8.8;
• Email using a `wrapped' Sendmail 8.6.10;
• NTP using Xntpd 3.3g,
• Telnet service with Berkeley Telnetd;
• Rsh and rlogin services (only on my subnet) using Berkeley Rshd and Rlogind;
• Converse service, a talk-like protocol used in amateur radio; and
• Domain Name service using Berkeley Named 4.9.3.

Minnie runs no other services. I use the wonderful tool called TCP_Wrappers, written by Wietse Venema, to protect services such as rsh, rlogin and converse. TCP_Wrappers allows or denies connections to particular services depending on the IP address from where the request comes. On Minnie, requests to these services from outside my department are rejected, and the requests are logged. TCP_Wrappers is a great utility, and if you're running any Internet connected Unix machine, I'd recommend using TCP_Wrappers on it.

3 Kernel Modifications

The following kernel modifications were made, adding 37 extra lines of C code to the system:

• Logging of ICMP packets with options unknown to the BSD networking stack.
• Logging and discarding ICMP packets with route redirect requests. These may be part of a network attack to redirect packets for a certain host through a machine which can monitor these packets.
• Logging and discarding IP packets with any options turned on. Most options are harmless except Source Routes (see below) and this option is probably not very useful.
• Logging and discarding IP packets with Source Route redirect requests. These may be part of a network attack to redirect packets for a certain host through a machine which can monitor these packets.
• Logging the fact that a certain TCP port has received too many SYN packets and had to throw connection requests away. This may indicate a TCP sequence number spoofing attempt.
• Logging connection attempts to unused TCP ports. This may indicate a probe to see what (vulnerable) services the server runs.
• Logging connection attempts to unused UDP ports. This may indicate a probe to see what (vulnerable) services the server runs.
• Changing the BSD networking stack's TCP's sequence number behaviour to foil TCP sequence number spoofing attempts. This vulnerability has been well-known for 11 years now without being fixed; see Morris 1985 for the original description.
• Logging all exec()s where the new program is running with an effective user-id of root (that is, 0). This option will produce an awful lot of logs if the machine runs many programs from root. Its advantage is that it may give an audit trail of root commands if an intruder breaks into the system.

These changes don't log the contents of the packets for two reasons. Firstly, logging packet contents will slow down the kernel and lower the system's performance. Secondly, the amount of data logged will be enormous; Minnie receives broadcast packets to UDP port 520 every few minutes from the subnet's router, not something that I'm particularly interested in logging.

4 Packet Suckers

To remedy this, I have written my own replacements, tcpsuck and udpsuck which are run by inetd to log TCP and UDP packets on particular unused ports. For ports which do have services on them, TCP_Wrappers can divert suspicious requests to these programs while still servicing legitimate users.

None of these programs have yet been triggered by would-be intruders. However, here are some example logs where I have tried to break in to Minnie from an external site:

• An attempt to TFTP my /etc/passwd file:

  Jan 25 12:15:10 Pkt from some.where.com port 1062 to tftp
   0- 00012f65 74632f70 61737377 64006e65	    ../etc/passwd.ne
  16- 74617363 696900			    tascii.
  

• A root rlogin attempt:

  Jan 25 13:35:03 Data from some.where.com port 1016 to login
   0- 00776b74 00726f6f 74007874 65726d2f	    .wkt.root.xterm/
  16- 39363030 00				    9600.
  

Due to the design of the Sun RPC protocols, a simple UDP packet logger won't capture the attempted RPC request. A program like Bellovin's portmopper program is required. I've altered Wietse Venema secure portmap daemon to perform RPC logging. Here's an example attempt to mount Minnie's root filesystem:

Feb 16 12:32:34 bad getport(nfs)    from happy.cs.adfa.oz.au
Feb 16 12:32:34 bad getport(mountd) from happy.cs.adfa.oz.au
Feb 16 12:32:35 fake connect from happy.cs.adfa.oz.au to mountd
 0- 322031d2 00000000 00000002 000186a5	    2 1.............
16- 00000001 00000001 00000001 00000030	    ...............0
32- 3123dd3a 00000013 68656e72 792e6373	    1#.:....happy.cs
48- 2e616466 612e6f7a 2e617500 00000000	    .adfa.oz.au.....
64- 00000000 00000002 00000000 00000000	    ................
80- 00000000 00000000 00000001 2f000000	    ............/...

5 Syslog Problems

• to listen on a UDP port other than 514, or no port at all; and
• to forward log messages to another machine's UDP port other than 514.

This minimises the chance of receiving fake log messages.

6 Reading the Logs

All system administrators need a tool which summarises the log files and highlights what the system administrator considers suspicious activity. How frequently the tool runs and what it highlights depends on the system's security requirements. On Minnie, a cron job reads through through the logs daily looking for suspicious behaviour. The result is emailed to me on another machine. A part of the summary shows TCP_Wrapper's activity and the results of my kernel modifications. Here is an example report:

From: Minnie Bannister
Date: Fri, 26 Jan 1996
To: wkt@cs.adfa.oz.au

TCP Connections
---------------
      ftp connections:	 378	      ftp refusals:	0
 sendmail connections:	  10	 sendmail refusals:	0
   finger connections:	   0	   finger refusals:	2
   rlogin connections:	   0	   rlogin refusals:	0
      rsh connections:	   0	      rsh refusals:	0
   telnet connections:	   2	   telnet refusals:	0

	08:53:49 fingerd[13074]: ccadfa.cc.adfa.oz.au
	10:28:07 fingerd[18732]: rudolph.north.pole.com


Attempted TCP Port Connects: 1
--------------------------------
Port 1713: 1 attempt

	14:21:48 TCP proto 1713 from 206.20.189.10, port 1075
	

Attempted UDP Port Connects: 3
--------------------------------
Port 33492: 1 attempts
Port 33493: 1 attempts
Port 33494: 1 attempts

	10:50:12 UDP proto 33492 from 129.72.40.4, port 41776
	10:50:13 UDP proto 33493 from 129.72.40.4, port 41776
	10:50:14 UDP proto 33494 from 129.72.40.4, port 41776

The first section summarises the information from TCP_Wrappers. There were only two connection refusals, both to the finger service. The next two sections show the attempted connects to unused TCP and UDP ports. The UDP connections show a traceroute to Minnie. I have no idea what's on TCP port 1713; the program which produced the summary would have given the name of the port if it was in the file /etc/services.

7 Seven Months of Network Activity

7.1 Logged TCP Packets

There is not much to report here. There were a few rlogin attempts, but none as root. Throughout May and June there was a person trying to ntalk to Minnie on the POP3 port, with no success.

7.2 Logged UDP Packets

The main suspicious activity here was SNMP packets. On 12 April 1996 I received two SNMP requests for Minnie's `public' SNMP information. On 15 June 1996, I had 16 SNMP packets for the following information: public, private, noAuth and medusa. These must be considered suspicious. There were no SunRPC requests, surprisingly.

7.3 Connection Packets to Unused TCP ports

The connects to auth are legitimate Ident requests which are mainly sent by remote mail servers. Connects to the nntp, gopher and alt-www ports failed because I don't run these services, and my Web server runs on port 80, not 8080 or 8001.

Some of the connection attempts were the result of TCP port strobes; I had ten strobes from five machines in the seven-month period, with one machine performing six of the strobes. I cannot explain why the remaining connect attempts were done.

7.4 Packets to Unused UDP ports

I have no idea why there is much more unwanted UDP traffic than TCP traffic. Packets to submitserver and bootserver ports might be suspicious, and I will add a UDP sucker to these ports. The packets to ports 33,480 and up are the result of traceroutes to Minnie and are normal. Again, I am at a loss to explain why the other packets were sent.

7.5 New Kernel Messages

Dropped half-open TCP connections:

Average of 250 per day, with peaks from 1,000 to 2,600 per day.

Closed half-open TCP connections:

Average below 3 per day, with peaks from 5 to 23 per day.

Unknown ICMP Packets:

Average below 5 per day, with peaks from 80 to 320 per day.

From January to July, bursts of TCP connection packets were seen 21 times. Most of these events were caused by over-enthusiastic Web robots or FTP catalogue agents. Three of these events, on 3 February 3, 23 February and 25 March 1996, were likely TCP sequence attacks. Because Minnie's TCP sequence number code has been fixed, the attacks failed.

7.6 Login Failures

7.7 Strange Mail Events

OAA25891: to=/root/dead.letter, ctladdr= root (0/0), delay=00:00:00, mailer=*file*, stat=Can't create output: Permission denied

On 9 April 1996, a remote user tried to use the EXPN and VRFY mail commands. These are disabled as Minnie has no users other than the super-user.

7.8 FTP Login Failures and Bad Commands

The number of misspellings of `anonymous' is quite incredible:

ANONYNOUS, ananonymous, anaoymous, annonymous, announimous, annoymous, anomous, anomymous, anon, anonimous, anonmous, anonomous, anonumouns, anonymosu, anonymouis, anonymouns, anonymous, anonymouse, anonymouys, anonymoyus, anonymozs, anonympp, anonympus, anonynous, anonyomos, anonyomus, anonyous, anoymous, anoynmous

As well as bad FTP logins, many anonymous logins tried to create new files and directories, change permissions on and/or delete existing files and directories. Only one anonymous login downloaded /etc/passwd and /etc/group. This didn't matter, as the copies used by the anonymous login don't contain any passwords and have several user-ids removed.

7.9 HTTP Errors

Minnie does not attract much attention as she is a small server on the Internet. Results from machines like archie.au or munnari.oz.au would be much more interesting.

8 Conclusion

I have added code to the FreeBSD kernel to detect connections on unused TCP and UDP ports and other suspicious network packets. Results to date show little suspicious behaviour. I have also written programs to log actual packet contents for suspicious connection attempts which can give even more information about attempted breakins.

Logging information is useless unless the information it contains is brought to the system administrator's attention in a useful and timely way. Automated and semi-automated tools are needed to summarise the logged data and present it to the administrator in a way which highlights activity which she/he considers suspicious.

Finally, individual system administrators can only monitor their own systems. They are not in a position to perceive trends in intruder activity. The collection of suspicious activity logs from several machines by an organisation such as AUSCERT may help to detect potential intruders before they wreak too much havoc. This can only be done if the information can be provided in a rational and machine-parseable format, and if the collection and analysis is performed in a (nearly) fully-automatic manner.

Acknowledgments

Appendix - References and Where to Get It?

If you have control over your network connection, and in particular the routers, you should consider implementing some form of firewall. There are several commercial and freeware products to help you do this. Before you start, get one of the following two books:

Chapman, Brent & Zwicky, Elizabeth (1995) Building Internet Firewalls, O'Reilly & Associates, Inc., USA.
ISBN 1-56592-124-0.

Cheswick, William & Bellovin, Steven (1994) Firewalls and Internet Security, Addison-Wesley, Reading, MA.
ISBN 0-201-63357-4.

These books outline what a firewall is, what it can and cannot do, how to set up firewalls, routers and bastion hosts, how to construct a network security policy and what to do if you are attacked. Cheswick & Bellovin's book was reviewed in the February 1995 AUUGN, and Chapman & Zwicky's book was reviewed in the October 1995 AUUGN.

The kernel patches and user-mode tools I've described in this paper are available at ftp://minnie.cs.adfa.oz.au/pub/NetSecurity/ In there you will find the following tar files:

kernmods.tar.gz:

These are the FreeBSD kernel modifications to log connections to unused ports, log root execs and log/prevent IP spoofing.

pktsuckers.tar.gz:

These are user-mode programs to log the contents of packets to suspicious ports. To get the most out of these programs, you should get the TCP_Wrappers from AUSCERT's FTP server.

wktportmopper.tar.gz:

This is a replacement portmapper daemon which will log unauthorised RPC requests and also their contents. You will need the TCP_Wrappers to use this program.

syslogd.tar.gz:

This is a modified syslogd which can use UDP ports other than 514.

Bibliography

1

Bellovin, S. M. (1992) There Be Dragons in Proceedings of the 3rd Usenix UNIX Security Symposium, Berkeley, CA, USA.

2

Bellovin, S. M. (1993) Packets Found on an Internet, Computer Communication Review, 23(3):26-31.

3

Bellovin, S. M. (1989) Security Problems in the TCP/IP Protocol Suite, Computer Communication Review, 19(2):32-48.

4

Morris, R. T. (1985) A Weakness in the 4.2BSD Unix TCP/IP Software
URL: ftp://minnie.cs.adfa.oz.au/pub/NetSecurity/117.ps.gz

Return to Conference Proceedings