Performance Test Information
Overview
This page explains why you may want to configure a Performance test in Highlight and gives guidance of which test to use.
There is also detail of what each test does in practice.
Customers typically want to know for example if the voice services they use are fit for purpose, why at certain times services are running slowly, or if critical services are 100% available. Highlight can help answer these questions by implementation of performance testing to provide a complete picture of a customer's services.
Test results are used in three principal ways:
Status monitoring:
showing in real time how well all the tests are complying with the given target service levels and (optionally) generating alerts when compliance is broken.
Visualisation:
providing a graphical picture of results over any day, week or month
Service reviews:
monthly reports will give an overview of a customer's services which can then be a used at a service review with their provider.
Performance testing is supported on Cisco and Juniper devices. See table in next section for individual test support
Which test to use?
This section guides you through a choice of which test suits a given requirement
Are my voice services are working efficiently?
The Highlight MOS test measures the effectiveness of your network infrastructure to support voice
How reliable is my web service?
For an individual server within a web cluster, a TCP test will confirm if the server is ready to service web calls
How stable is my network?
The Highlight Precision delay test will assess if the network is offering consistent end to end transmit times by giving a real-time view of network reliability (are any packets dropped?) and consistency (do the round trip times vary greatly?) in addition to the average round trip delay
How can I monitor a device that does not support SNMP?
The Highlight Ping test detects the up/down status of any IP capable device
How can I tell if my network is available end to end, and gauge response times?
There are two test types available- For Cisco networks, the Highlight UDP echo test offers true network latency
- For other networks the Highlight Ping test offers round trip delay statistics
Summary of tests and their uses
| Test to measure... | Test type | Test description | Manufacturers supported |
|---|---|---|---|
| Reachability | ICMP Ping | Tests if a remote device responds, and measures round trip delay | Cisco, Juniper |
| UDP echo | Measures network delay between locations | Cisco, Juniper | |
| Application availability | TCP open | Determines if application is active (“listening”) | Cisco, Juniper |
| Network performance | Precision delay | Measures realistic jitter and delay statistics | Cisco, Juniper |
| Voice quality | MOS | Assesses a network path for voice quality | Cisco |
ICMP Ping
ICMP Ping test description
Ping is a computer network administration utility that is used to test the reachability of an end device or host and measure the round trip time (RTT) if a response is received. This can help to determine if there are any delay or latency problems on the network, although you should be aware that networks may treat ICMP Ping packets differently from packets carrying user traffic (for example, they may give them lower priority) and so performance results using pings may not exactly mirror the performance seen by applications. Even so, pings are a universal and handy baseline to establish connectivity.
ICMP Ping test default settings
Default information when setting up a Ping test:
Thresholds:
- Delay
- default is 400 milliseconds
- Packet loss
- default is 0.1%
- Jitter
- default is 15 milliseconds
For each test:
- Packet size
- default is 128 bytes
Highlight collection
The router will run the test at a test frequency specified when setting up a test. The default is every 30 seconds but can be changed.
Then at every polling cycle Highlight gathers up to 6 results for analysis and presentation.
UDP Echo
UDP Echo test description
The UDP Echo operation measures round trip time (RTT) between a router and a device configured to respond to these tests. It is similar to a ping, but uses a UDP packet which is closer in profile to an application packet. Some 'far end' devices may not respond to a UDP Echo test.
In the following diagram Router A is configured with the UDP Echo test. Response time (RTT) is computed by measuring the time taken to send a UDP Echo request from A to B and back again.
When running this test between Cisco devices, accuracy is enhanced by configuring Router B as an “IPSLA Responder”. This feature will record the time to process the UDP Echo request in its reply packet (can be as much as 100 milliseconds), which allows the router processing time to be excluded in the reported network response time.
The results of a UDP echo test can be useful in troubleshooting issues with business-critical applications by determining RTT attributable to the network.
UDP echo test default settings
Thresholds:
- Delay
- default is 400 milliseconds
- Packet loss
- default is 0.1%
- Jitter
- default is 15 milliseconds
For each test:
- Packet size
- default is 128 bytes
Highlight collection
The router will run the test at a test frequency specified when setting up a test. The default is every 30 seconds but can be changed.
Then at every polling cycle Highlight gathers up to 6 results for analysis and presentation.
TCP Open
TCP open test description
This test is used to measure the availability, or response time, of a server or service on a specific TCP port. For example, you can test whether or not a SAP system is responding, or whether your mail servers are responding via SMTP on port 25. This allows you to build a picture of application availability, potentially by testing from multiple sites, or to get an alert when an application starts performing slowly or fails.
The test will periodically attempt to open the specified TCP port using a ‘Connect’ packet, and record the time taken for the receiver to respond to the request. TCP Open tests are dealt with as a higher priority than ICMP Ping tests. Therefore a device may ignore a ping if it is busy whereas a TCP Open test would get put in a queue to be responded to.
TCP open test default settings
Thresholds:
- Delay
- default is 400 milliseconds
Highlight collection
The router will run the test at a test frequency specified when setting up a test. The default is every 60 seconds but can be changed.
Then at every polling cycle Highlight gathers up to 6 results for analysis and presentation.
Precision Delay
Precision delay test description
The Precision Delay test is a more powerful way of checking end-to-end performance of a data network. While the ICMP Ping test sends a series of ‘echo’ requests across the network and measures response time, Precision tests send a closely packed burst of tests which more accurately mirror real network traffic. This generates more background load on the network, but it gives more detailed results. Use Precision tests if you want to go beyond straightforward availability and testing to verify that your network can support specific, demanding applications.
Must have Cisco IPSLA responder as destinationPrecision delay test default settings
Thresholds:
- Delay
- default is 20,000 microseconds
- Packet loss
- default is 0.1%
- Jitter
- default is 10,000 microseconds
For each test:
- Packet size
- default is 128 bytes
- Total packets
- default is 50
- Packet interval
- default is 20 milliseconds
Highlight collection
The router will run the test at a test frequency specified when setting up a test. The default is every 175 seconds but can be changed.
Then at every polling cycle Highlight gathers a single result for analysis and presentation. A test run significantly more frequently than the polling cycle would be inefficient as only one result is collected.
Precision delay comparison with ICMP Ping
| Comparison of tests for a typical 3 minute polling cycle | ICMP Ping test | Precision delay test |
|---|---|---|
| Test Details | 6 ICMP Pings 30 seconds apart | 50 UDP Echoes 20 milliseconds apart (configurable) |
| Reporting of | Delay, Packet Loss, Jitter (calculated by Highlight) | More accurate Delay, Packet Loss, Jitter (reported by the router) |
| Settings available | Delay only | Delay, Packet Loss, Jitter |
| Typical use | To verify availability, and compliance with targets in terms of delay | To verify wider target compliance and monitor suitability for specific applications such as SAP, Citrix, Video |
| Requirements | Tests to any ICMP-enabled device | Requires Cisco IPSLA Responder |
MOS
MOS test description
When running Voice calls over an IP network (VoIP), it’s the quality of that network which dictates whether the user’s telephone experience is a good or bad one. Delay, loss and jitter will all have an effect.
Besides the qualitative description a user would give like 'quite good' or 'very bad', there is a numerical method of expressing voice quality: Mean Opinion Score (MOS). MOS gives a numerical indication of the perceived call quality, by giving a score from 1 to 5, as follows:
Perfect, like face-to-face conversation or digital radio reception
Fair, imperfections can be perceived, but sound still clear; supposedly the range for cell phones
Annoying
Very annoying, nearly impossible to communicate
Impossible to communicate
The values do not need to be whole numbers. For instance, a value of 4.0 or higher is referred to as toll quality, the normal value for the public telephone network. Many VoIP services aim for this, often with success, while values dropping below 3.5 are deemed unacceptable by many users.
The router running the MOS test sends simulated voice traffic across the network every few minutes, and calculates the MOS a phone user would experience based on an algorithm related to the user's codec. Call quality will depend heavily on the codec (Coder / Decoder) which is used to send an ‘analogue’ voice call over a digital network.
The destination device must be Cisco running IPSLA Responder.
MOS test default settings
Thresholds:
- Delay
- default is 400 milliseconds
- Packet loss
- default is 0.1%
- Jitter
- default is 15 milliseconds
- Score
- default is 4
- Codec
- default is G.711 uLaw with G.711 aLaw and G.729A also available
Highlight collection
The router will run the test at a test frequency specified when setting up a test. The default is every 175 seconds but can be changed.
Then at every polling cycle Highlight gathers a single result for analysis and presentation. A test run significantly more frequently than the polling cycle would be inefficient as only one result is collected.
Information required to set up a new test
Highlight supports tests via SNMP v2c and v3. Information needed to create a test:
SNMP v2c
- Device IP
- the Highlight accessible IP address of the device running the test
- RO community
- as configured on the device
- RW community
- as configured on the device
SNMP v3
- Device IP
- the Highlight accessible IP address of the device running the test
- Security level
- only authPriv is supported
- Username
- as configured on the device
- Auth key
- as configured on the device, MD5 or SHA1 selected
- Priv key
- as configured on the device, DES or AES 128 selected