I recently journeyed around Seattle to get a sense of the impact motion has on the network performance of mobile applications.

Request and response times touring Seattle

The image above is a graph of how long it took to download the homepage of Google News (588 KB) while I was traveling by bus and train throughout the city. Blue bars are successful downloads and red bars (and time entries with no bars at all) represent errors. The height of the bar shows how long it took to find out if it was successful download or an error.

I wrote the test app using MonoTouch and ran it on an iPhone 5 with Verizon LTE.

Observations

When you’re standing still, performance is pretty consistent and error free

Whenever I gave the phone time enough to pair with a cell tower, the performance was pretty consistent with 2.5 second response times with 588 KB of data (240 KB/s). I never received errors (go TCP!).

You will get spikes errors when moving

While most downloads took 2.5s, I received many 5s spikes. This is 100% increase in time. There was a even a time (look in the last 1/3rd of the chart) where 5s became the average and I received 10s spikes.

The spikes happened quite spuriously so I can only guess that they occured in cell tower transition zones.

While TCP is an insanely resiliant protocol, it would still fail from time to time. In a 2 hour interval, I recorded 2 errors that weren’t tunnel-related. This means that there was a 1% error rate while moving. My only guess is that these happened whenever I was switching towers and the request happened just then.

3G sucks compared to LTE

Do you see those blue spikes near the center just after a little gap (the Mount Baker tunnel)? The one that peaks at more than 50s? That’s 3G my friends: slower downloads by an order of magnitude and a huge variance increase. I can only wonder what this graph would be like if I was stuck on 3G the whole time…

Timeout and ReadWriteTimeout control these spikes

For the test, I used a WebRequest.Timeout of 20 s and a ReadWriteTimeout of 20s.

You can see the affect of Timeout with all the errors that cap at 20s. Mono gave a “System.Net.WebException: The request timed out” at that mark. This is great because it’s very predictable.

So, how do bars go above 20s? That’s the effect of ReadWriteTimeout. This is the timeout that affects each Read and Write call to the response object. I was using a 16 KB buffer for my reads so this means that I did about 37 read operations. Since each of these had a timeout of 20s I could have ended up waiting up to 740s (12 minutes)! Thankfully that never happened.

About 50% of the time, ReadWriteTimeout was able to finish the download. On the other 50%, I ended up receiving “System.Net.WebException: The operation has timed out.” exceptions. I’m not sure if I see much value in setting it up - I would prefer that errors come sooner rather than later.

The system default of 300s (5 minutes) seems beyond ridiculous to me. Don’t use that value.

TL;DR ReadWriteTimeout is an important value that you should think about.

Thank God for ConnectFailures

While travelling through a tunnel, the request would fail immediately with “System.Net.WebException: Error: ConnectFailure (No route to host)”.

These errors are represented by the white gaps in the chart. In the face of those 38 s errors, this is wonderful.

All errors were represented by WebExceptions

I’ve always wondered if a try-catch block of:

try {
    // Do some System.Net.WebRequest stuff
} catch (System.Net.WebException) {
    // OMG. OMG. OMG.
}

one that caught only WebException - was sufficient to catch all network related errors when using WebRequest. It turns out that yes, yes it is. One less thing to think about…

Here are all the errors MonoTouch emitted during this test (in Type.Name + Exception.Message format):

  • System.Net.WebException: Error: ConnectFailure (No route to host)
  • System.Net.WebException: The request timed out
  • System.Net.WebException: The operation has timed out.

Code Setup

The actual code for the netowork request can be found in the HandleTimer method in the gist for the test app.

The app disabled the idle timer of the phone so that it could run continuously in the foreground. Every 30 seconds, it would download the full 588 KB of Google News. In all, I figure I burnt through 8% of my monthly data allowance.

Test Procedure

  • Device: iPhone 5 (white)
  • Carrier: Verizon (LTE, usually)

I started the test by getting a slice of pizza at Pagliacci’s in the University District of Seattle. I figured this would be a good way to establish a baseline for staying still, and I was very hungry. I then took a bus to Westlake, downtown, wherein we entered the transit tunnels and I transferred to the light rail. I took the light rail through the tunnel and back into the open air as far as Columbia City. This journey consumed the first hour of the test. There, I switched trains to head back into the city. The trip back was essentially the same and even included another stop for pizza in the name of consistency (at Sbarro this time).