Checking Bandwidth Usage

Your bandwidth usage is the sum total of all traffic from and to your account on the server, including visitors accessing your site, FTP uploads and incoming/outgoing emails.

The server will automatically send bandwidth warnings at 70%, 80%, 90%, 95%, 97%, 98%, 99% and 100% usage to the contact email specified in cPanel under "Preferences -> Update Contact Details".

Checking Bandwidth Usage
  1. Log into cPanel. If you are unsure about your login details, please contact support http://www.ashhost.co.nz/support.html 

  2. On the left hand side, you can see a summary of your current usage 

    Bandwidth Usage is shown on left side of cPanel

  3. For details, please go to "Logs-> Bandwidth" 

    Click on Disk Space Usage in cPanel

  4. Scroll down the page where you can see the summary stats for each month. For details, click "Total (all services)" under the relevant month. 

    View details at bottom of the page

Checking Awstats
  1. In your site's cPanel go to the Awstats link and click on it. 

    Click on Awstats link 

  2. At the top of Awstats you can review the "Viewed" vs "Not Viewed" traffic. Not Viewed traffic includes all robots and if unusually high, may indicate a problem with excessive spidering. 

    Awstats Traffic Type

  3. Further down, you can also review the type of files being accessed. Which file types dominate will depend on your site's coding but any unexpected or unusually high results may indicate an area for optimisation. 

    Awstats Traffic Type

Commmon Problems
SPIDERS

Search engines, price comparison sites and other companies index the internet with the use of automated spiders. Most reputable companies will do this spread out over a period of time, but some spiders can be aggressive and move through your site at high speed. This can quickly rack up bandwidth.

To check whether a spider is responsible for unexplained bandwidth usage, please first check the stas as explained above. Then if the unexplained traff is HTTP, you can use Awstats to further drill down into how the bandwidth is being used. Awstats is accessible from cPanel under "Logs -> Awstats".

Click on the Awstats link in cPanel

EMAIL AND ATTACHMENTS

Incoming and outgoing email is part of the bandwidth. For users with large attachments or frequent email usage, this can lead to high bandwidth even if the website itself receives little traffic. Email usages will show in the stats as "POP3" for incoming, "SMTP" for outgoing and "IMAP" for IMAP services.

IMAP

Every time the server is accessed with a new device for IMAP, the entire mailbox is being synced. For users with large mailboxes, this can lead to high bandwidth usage and will show up in the stats under "IMAP".

Analysing the Raw Access Log

The raw access log will show you exactly who accesses your site where, but due to the number of entries it can be tricky to distinguish patterns or major bandwidth users.

The easiest method to analyse the log is via MS Excel or a similar spreadsheet program. To use Excel to analyse the log:

  1. From within cPanel download the raw access log, save to your PC/Mac and unzip. 

    Click on Raw Access Logs in cPanel

  2. Open Excel and from within Excel open the log file. This should bring up the import options. Choose "delimited". 

    Delimited

  3. On the next screen choose "Delimiters: Space". Click "Finish". 

    Space delimited

  4. On the Excel page, delete all columns except the first (IP number) and the last (description). Insert a new row right at the top, and enter a name for each column (eg IP, Description). 

    Excel columns

  5. Highlight the entire sheet, click "Insert -> Pivot Table". Under Values, enter IP count, under Row Labels enter IP. 

    Pivot Table Configuration

  6. This should now give you a quick overview of the key IPs accessing your site - the higher the count, the more often this IP accessed your site. You can then look back at the raw access log to see details for this IP and find out its name (eg google bot etc).

    Pivot Table

TWENGA BOT

The twenga bot from twenga.com has caused considerable bandwidth usage on some sites on our servers. It indexes sites aggressively, sometimes with multiple simulatenous connections/IPs and at a fast rate. This can lead to high bandwidth usage with little benefit to the site owner.

Twenga Bot

For a list of current IPs used by twenga, see eg http://user-agent-string.info/list-of-ua/bot-detail?bot=TwengaBot.

Blocking Twenga in robots.txt via the usual deny rule is an option, but various sites have reported that twenga does not obey robots.txt so this may not stop the problem.

For a complete block, either block the IP (see below) or block the bot via .htaccess user agent rule. In your root .htaccess file add:

1
2
3
4
# let's block twenga
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^twenga
RewriteRule ^.*$ - [F]

Related article: http://my.ashhost.co.nz/knowledgebase.php?articleid=4&catid=14

BLOCKING IPS

IPs can be blocked from a website via the IP Deny Manager in cPanel. Once you have identified an IP that is causing you problems, enter it into the IP Deny Manager and this IP will be blocked from accessing your site.

IP Deny Manager in cPanel

Bear in mind that bots can operate from an IP range, so you may need to keep an eye on your access logs and block troublesome IPs as they appear.

  • 0 Users Found This Useful
Was this answer helpful?

Related Articles

Hosting Guide: cPanel

This article explains how to manage your site with ASH HOST, how the control panel works and how...

ns Name Servers

ns1.ashhost.co.nzns2.ashhost.co.nzns3.ashhost.co.nz