How tutela is performing network quality test?

Listen it!

Tutela Support Centre Help Center home page

 

Download Throughput Test Methodology

 

When a download throughput test is performed the Tutela SDK determines (based on the current configuration) how to execute the throughput test.

Download throughput tests are performed using Amazon’s CloudFront service using HTTPS. This allows test files to be downloaded with low latency through one of the edge test servers located around the world. This service is commonly used to host popular websites and other web applications.

By default, Tutela uses a 2MB file to perform download throughput speed tests. This simulates typical user download behavior (downloading a web page). In general, the average throughput speed for small file-sizes can often be lower than the throughput speed for larger file sizes and customers may, therefore, notice a difference in the results from Tutela’s test results than from drive-testing or other measurements which use different file sizes or configurations.

A throughput test occurs as follows:

Amazon’s Cloudfront service determines the closest edge test server to serve the download test file based on the IP address of the device.
The SDK opens a TCP connection to the test server to start downloading the test file. The download test file size is a 2MB file to simulate loading a webpage or beginning a video session.

Note: This is not intended to calculate the maximum download throughput obtainable on the network. Instead, this is designed to be representative of the throughput a user may experience through using a typical app on a network.

The current time is then recorded in nanoseconds, as well as a received (Rx) byte count. The received byte count is a numerical count of how many bytes have been delivered over the wireless radio at a given time. This is the last step performed before the actual bytes are streamed from the server, to ensure that the measurements have the highest level of accuracy.

Data bytes are streamed-in as 1024 byte blocks until either the file has completed downloading or a timeout or a cancel test event is triggered.
The connection is then closed, the current time and the device’s current received byte count are recorded.

The download throughput of the connection is calculated by computing the difference of the received byte counts from after and before test and dividing by the time duration of the test using the formula below:

 

Upload Throughput Test Methodology

When an upload throughput test (using HTTPS) is performed the Tutela SDK determines (based on the current configuration) how to execute the throughput test.

By default, Tutela uses a 1MB file to perform upload speed tests. This simulates typical user behavior (uploading an image). In general, the average throughput speed for small file-sizes can often be lower than the throughput speed for larger file sizes and customers may, therefore, notice a difference in the results from Tutela’s test results than from drive-testing or other measurements which use different file sizes or configurations. 

The edge test servers are located in the following locations: 

     North America     Europe     Asia
Ashburn, VA (3)Atlanta, GA (3)Boston, Chicago, IL (2)Dallas/Fort Worth, TX (5)Denver, CO (2)Hayward, Jacksonville, FLLos Angeles, CA (4)Miami, FL (3)Minneapolis, Montreal, QCNew York, NY (3)Newark, NJ (3)Palo Alto, Philadelphia, PAPhoenix, AZSan Jose, CA (2)Seattle, WA (3)South Bend, INSt. Louis, Toronto, ONAmsterdam, The Netherlands (2)Berlin, GermanyCopenhagen, Denmark Dublin, IrelandFrankfurt, Germany (8)Helsinki, FinlandLondon, England (7)Madrid, Spain (2)Manchester, EnglandMarseille, FranceMilan, ItalyMunich, GermanyOslo, NorwayPalermo, Italy Paris, France (4)Prague, Czech RepublicStockholm, Sweden (3)Vienna, AustriaWarsaw, PolandZurich, SwitzerlandBangalore, India Chennai, India (2)Hong Kong, China (3)Kuala Lumpur, Malaysia Mumbai, India (2)Manila, PhilippinesNew Delhi, India (2)Osaka, JapanSeoul, South Korea (4)Singapore (3)Taipei, Taiwan(3)Tokyo, Japan (9) 
     South America     Australia     Africa
São Paulo, Brazil (2)Rio de Janeiro, Brazil (2) MelbournePerthSydneyJohannesburg, South AfricaCape Town, South Africa
      Middle East 
 Dubai, United Arab EmiratesFujairah, United Arab Emirates 

An upload throughput test occurs as follows:

  1. The closest test server used for upload throughput tests is determined by the IP address of the device. This server is then used to execute the upload test.
  2. The SDK opens a TCP connection to the test server to start uploading the test file.
    The default upload test file size is a 1MB file to simulate uploading a medium definition image to social media.  
    Note: This does not calculate the maximum upload throughput obtainable on the network but the representative throughput a user may experience through using a typical app on a network.
  3. The current time is then recorded in nanoseconds, as well as a transmitted (Tx) byte count.  The transmitted byte count is a numerical count of how many bytes have been sent over the wireless radio at a given time.  This is the last step performed before the actual bytes are streamed from the server, to ensure that the measurements have the highest level of accuracy.
  4. Data bytes are streamed out in 1024 byte blocks until either the file has completed uploading or a timeout or a cancel test event is triggered.
  5. The connection is then closed, the current time and the current devices transmitted byte count are recorded.  
  6. The upload throughput of the connection is calculated by computing the difference of the transmitted byte counts from after and before test and dividing by the time duration of the test using the formula below:

UDP Latency, Jitter and Packet Loss Test Methodology

A server response (SR) test is defined as a test that determines the latency, jitter and packet loss of the current connection using UDP.  When a server response test is performed, the Tutela SDK determines (based on the current configuration) how to set up the server response test.

Server response tests are performed against one of twenty test servers located around the world. Multiple servers are used to provide low latency based on geography. The test servers are located in the following locations:

     North America     Europe     Asia
Virginia, USA Oregon, USA California, USAOhio, USATexas, USA (Azure)Montreal, CanadaDublin, IrelandFrankfurt, GermanyLondon, England Paris, France Stockholm, SwedenSeoul, KoreaTokyo, JapanSingaporeMumbai, IndiaBahrain
     South America     Australia     Africa
São Paulo, BrazilSydney, AustraliaJohannesburg, South Africa (Azure)

To accurately measure these KPIs, a predetermined number of packets are sent to the test servers, usually 20 packets. Each of these packets contains a 1-byte payload which holds a number indicating the ordering of that packet in the sequence of packets sent out.

The payload size per packet and the number of packets sent are configurable parameters set by the Tutela DSC.

These tests are then performed as follows:

  1. The SDK chooses the closest test server based on the geographical distance between the device and the server locations.  
  2. Each packet is then sent to the test server via UDP which is connectionless. This means that, unlike TCP connections, there is minimal handshaking involved which could produce measurable results that indicate slower-than-actual speeds.  
    UDP protocol is also used to simulate latency-sensitive applications, such as video and VoIP calling.  
  3. The precise time that each packet is sent out is recorded to nanosecond precision and stored for later calculations.
  4. The test server receives the packets sent from the SDK and then immediately echoes them back to the listening device. The precise time that each packet is received back on the device is also recorded along with the specific payload of each packet.  This payload indicates the ordering of the packets returned to the device.
  5. The Metro Ethernet Forum (MEF) 10 specification is used to calculate the latency, jitter, and packet loss.  The MEF 10 specification is known to accurately measure jitter in real networks where packets can be sent with any uniform or bursty traffic patterns.  
    1. Latency is determined as half of the round-trip travel time of each packet from the time the packet is sent from the SDK to when the same packet is received.  
    2. Jitter is calculated by measuring the change in latency from packet to the packet received by the SDK.  
    3. Packets that are never received on the device are classified as lost, while we categorize packets that were either duplicate or out-of-sequence as discarded as they can no longer be used for latency or jitter measurement calculations.

Leave a Reply