Advertisment

Five UTMs Attacked

author-image
PCQ Bureau
New Update

We have tested a lot of enterprise network equipment in our Labs. From

Servers to NAS boxes to Workgroup printers and so on. However, testing a UTM

device is amongst one of the most time consuming tasks. The reason for this is

very simple.

Advertisment

In just one device or box we have to test at least five or ten different

components. So testing a single UTM device will include testing an anti-virus,

spam filter, content filter, firewall, IDS/IPS, DMZ, etc.

For our shootout we created a test bed for testing four of the most important

components-anti-virus, anti-spam, IDP and firewall. In the following sections we

unfold our experiences with these UTMs and describe in detail how we went about

testing them. You can use a similar setup to test not only UTMs but also other

security devices. However, a word of caution! These tests are potentially

damaging for your network as they involve attack these devices with real life

viruses, spam and hacking tools. So, never carry them out on a production

network.

Anti-virus tests



Testing for anti-virus capability is the easiest amongst all tests. We

simply need to create a Web, FTP and SMB server, and a set of different types of

viruses on top of it.

Advertisment

We used a Linux machine to host these viruses so that the hosting machine

itself doesn't get affected by them. The viruses that we used had old 16-bit

viruses to the latest Trojans and malware. We used a set of viruses with around

1000 virus files. This set was kept constant for all UTM devices.



Once the host machine was ready with all viruses hosted on top of it, we
connected it to the public port of the UTM devices one after the other and tried

downloading all viruses from the private network. Once done, we counted the

number of viruses which bypassed the UTM and got downloaded on the public

network.

Anti-spam tests



These tests are pretty much similar to the anti-virus tests, but more

categorized. We setup a machine with a POP3 Mail server running on it and dumped

around 2000 different spam mails on it. However, before dumping the spam, we

categorized it into text, image and PDF. Then we connected the machine to the

Internet and gave it a public IP address which is mapped with the MX record of a

domain. We took the UTM devices one by one and connected their WAN port to the

Internet.

Advertisment

We then connected a few machines to its private network and started

downloading the spam using Outlook Express. Once done we checked how many spam

the devices had missed; to either tag or block, and counted the number for all

devices. Again, to compare the performance of all devices we kept the set of

spam identical for all devices.

Firewall tests



To test the firewall, we ran the industry standard vulnerability assessment

tool Nessus; and a standard DOS attack. For running the DOS attack, we used

ettercap's Nice DOS plugin.

Advertisment

The test was pretty simple. We connected the WAN port of the UTM device to

the Internet with a public IP, ran Nessus and then the DOS attack, sitting on a

machine connected to the Internet from a different gateway.

To interpret the results, we counted the numbers of warnings, and issues

discovered by Nessus. And for the DOS attack we checked whether the device was

able to log and drop the attack or not.

Advertisment

IDS/IPS tests



To test the IDS/IPS functionality, we focused on the capability of the

device to detect internal attacks, or attacks that are generated from a

trusted/private network.

To test this we ran an ARP spoofing tool on the IP address of the public port

of the device (the IP which is used as the gateway address for the network), and

we tried to check what exactly the device does to prevent such kind of attacks.

ARP spoofing is a mechanism by which one can compromise the ARP cache of

switches, and divert all traffic intended for some other IP, to one's own IP.

This technique is also known as 'Man in the Middle Attack' or 'ARP flip-flop

attack' or 'ARP Poisoning Attack'.

We ran the tests in two modes. First, we spoofed the gateway IP and then

explicitly forwarded the data coming to the hacking machine, to the destination

gateway. And in the second mode we stopped forwarding all the data to the actual

IP.

Advertisment

Surprisingly, very few UTMs were able to detect and log this attack in the IP

forwarding mode. And none of them were able to prevent or take a precautionary

step.

At the same time, access to a UTM's private or gateway IP completely stopped

when we ran the test in a 'non-IP forwarding' mode. This shows that even now, a

'Man in the Middle Attack' is one of the most dangerous attacks from inside the

network and one of the stealthiest as well.

Performance



After all it comes to performance in the end. So to test and compare the

throughput from all boxes, we used a tool called QCheck. This was done by

connecting one machine to the public port of the device and another to the

private port. Then we installed QCheck on both the machines and ran it with a

packet size of 1Mb. The result was recorded in the form of throughput in Mbps.

Advertisment