NetScope Blog

Why deep packet inspection is so important for detecting web traffic



Difficulties of enforcing web policies on business Internet

Content delivery networks (CDNs) make enforcing web browsing policies difficult

When a user requests a site such as Facebook their computer will first ask for an IP address from the local Domain Name Server (DNS). Queries for the IP address that matches the domain name will continue up the chain of DNSs until an answer is found. For a lot of queries this will match one IP address and the web content will be served from that page. Click here if you'd like to read up on DNS lookups in more detail.

Bigger sites like Facebook, Twitter, YouTube do something different

Facebook for example, at the time of this writing, uses Round-robin DNS. Which means that a DNS lookup can match to any number of different IP addresses. This means that as soon as the lookup is performed and the web browser / computer is given an IP address to get content from there is often no domain name associated with that IP any more. Other large sites use content delivery networks to distribute their traffic to users.

This makes enforcing web browsing policies difficult

Sure, you can block websites at the DNS level, enforcing a draconian Internet use policy where only a few selected websites are allowed. But what if you've got groups of users who need to legitimately use social media like marketing, or you want to allow people to browse the web during their breaks?

How can you enforce a more open web browsing policy?

It's very hard to do with at layer 4 (OSI model)

At layer 4 we see the top domains for the selected time period. Here we can see some domains that make sense, like 'msn.com' and 'google.com.au', but some that are a little more obscure (these are delivered by CDNs).

Your average switch, router or web proxy reporting tools will give you information at this level.

Some useful domains but a lot of Content Delivery Networks. Not very useful.

We know Facebook traffic is present in the above chart

Why can't we see the Facebook traffic?

As we mentioned above, Facebook DNS uses a round-robin method to return naked IP addresses, so it is not unusual for domain information to be absent. This is because there is either no domain name associated with that IP address, or there is no domain returned on a reverse look-up on that IP.

However, if we use deep packet inspection we see Facebook traffic as well as other useful information
Deep packet inspection side-steps domain lookups by getting information directly from the packet

Without deep packet inspection Facebook traffic appears as IP addresses

Facebook traffic seen without deep packet inspection

All we see is IP addresses. There are no domains associated with this traffic unless we look into the packet itself. Deep packet inspection is the only real way to tell this information apart, without keeping track of all the IP addresses that Facebook uses.

Without deep packet inspection we don't get nearly as much useful information about our web traffic

What can we do with this new level of web traffic information?

A few ideas

  • Control access to different websites by time of day
  • Allow only certain groups, like Marketing, to access social media sites
  • Create alerts to notify the network admin if video sites exceed a certain bandwidth
  • Protect access to important websites like Salesforce (or your CRM) with QoS rules
  • Block access to groups of websites

What NetScope will do for you