3G mobile data speeds in Sydney on the iPhone

If you’re interested in mobile Internet and the iPhone (and, really, who isn’t?), the Byteside ByteBlog has posted the results of its “Australian iPhone data test”.  The researchers measured 3G download speeds, upload speeds, and ping times, using the iPhone’s Speedtest.net application connected to four major mobile ISPs around Sydney.

We had concurrent access to four iPhone 3GS handsets, one on each of the four Australian networks — Telstra, Optus, Vodafone, and 3.  Travelling around the Sydney CBD and Sydney suburban areas, we ran close to 150 individual speed tests.  Tests ranged from Manly to Homebush, Annandale to North Sydney, and plenty in between.

Byteside have kindly made the raw data from their study available for download.  To analyse the results in more detail, I thought it would be an interesting exercise to try out the Instat statistical analysis package.  Instat can be downloaded for free (for non-commercial use) from the Statistical Services Centre.

Importing Byteside’s raw data (available as an Excel file) into Instat was fairly straight forward.  It just needed a bit of cleaning up.  To start proceedings, Instat’s Summary Tables was used to produce some descriptive statistics of download speeds.

Table 1: Statistical summary of download speeds by ISP

Mobile
ISP
obs
(no.)
min
(kbps)
mean
(kbps)
median
(kbps)
max
(kbps)
st.dev
(kbps)
Optus 110 0 1637 1903 3654 1080
Telstra 149 0 2681 2416 6151 1554
Three 149 0 625 525 2290 471
Vodafone 151 0 1283 1125 3349 961
TOTAL 559 0 1550 1271 6151 1329

Telstra looks the clear winner in terms of average and maximum download speeds, followed by Optus, Vodafone and Three.  However, I don’t think a summary analysis can tell the whole story.  I wanted to have a look at the plots of the distributions and run a few tests.  Instat obliges with the first request by providing quite a comprehensive graphing capability.

Intstat’s boxplot feature seems to cover all the basics (there’s also scope for plenty of customisation).  Below is a comparison of download speeds between the four carriers.

Figure 1: Boxplot of mobile download speeds by ISP

3G_download_speeds_boxplot

Now let’s have a look at the histograms…

Figure 2: Histogram of mobile download speeds by ISP

Fig. 2a: Optus
optus_download_hist
Fig. 2b: Telstra
telstra_download_hist
Fig. 2c: Three
three_download_hist
Fig. 2d: Vodafone
vodafone_download_hist

Telstra really does own the opposition, but not consistently so.  In fact the most frequent download speed for “The Big T” was less than 2Mbps.  All four carriers recorded their fair share of quite cruddy speeds, and all four recorded at least one instance where a download attempt didn’t work at all.  The reliability of 3G mobile broadband in this country (or Sydney, at least) has some way to go, apparently.

I also ran a couple of statistical tests.  As the data aren’t normally distributed, or the ISP download speed variances equal, I chose the non-parametric tests that Instat offers.  Firstly, to ask a somewhat redundant question, does network make a difference to download speed?

Kruskal-Wallis Test

Sample   n   Median   Ave rank    z
1    110  1902.50    305.0      1.81
2    149  2416.00    401.2     10.70
3    149   525.00    163.2    -10.30
4    151  1125.00    257.4     -2.01

H = 167.42 (adjusted for ties) with 3  d.f
Probability > 167.42  = 0

When you ask an obvious question, expect an obvious answer, I suppose.  We can comprehensively reject the null hypothesis that all the medians are equal.  Yes, download speeds depend on the network that you’re on.

Looking at the histograms above, the Optus and Vodafone distributions share some similarities.  Is there a statistically significant difference between these two carriers?  Again, the answer looks fairly obvious before the test is even run, but what the hell…

Two-sample test for independent data

Sample   n   Median   Rank sum
optus_d 110  1902.50  16047.0
voda_d 151  1125.00  18144.0

Mann-Whitney U = 9942.00    U’ = 6668.00
Wilcoxon T     = 16047.00

On HO: Mean (for U) = 8305.00
Mean (for T) = 14410.00
s.d. = 602.09  (adjusted for tied ranks)

Hence z = 2.72      Significance level is 0.33% for one-sided test
Significance level is 0.66% for two-sided test

0.33% (p=0.0033, one-sided) is highly statistically significant.  So we must conclude that the Optus mobile data network does indeed provide faster download speeds on the iPhone than Vodafone in the Sydney areas covered in Byteside’s study.  That’s when the download does actually work, of course.

So this post is really more a brief review of the Instat package than mobile network download speeds in Sydney.  I really enjoyed using it for the small amount of statistical analysis presented in this post.  Instat was capable and quite fun to drive.  Although Instat doesn’t have the polish or bells and whistles of a package like SPSS, it does have an intuitive GUI; a useful tool-set; and doesn’t come with the steep learning curve of R.  Free for non-commercial use, the price is right too.

Finally, in a conclusion keeping with the overall theme of this post, Telstra is the obvious choice for mobile data carrier in terms of download speeds and reliability.  But you’ll certainly pay for the privilege.  Optus and Vodafone both represent good value alternatives.  Based on the results collected by Byteside, it’s hard to recommend Three.  It is also a particular concern that all four networks recorded nil or negligible speeds to a significant proportion of download attempts in the survey.

——

About these ads

6 Responses

  1. Forever jealous of your statistical skills. I haven’t stopped by here in a while – for shame – I’m now going to work my way backwards :)

    • Thank you, Katie. I always come away from your blog the better for it. Keep up the great work.

  2. The similarities between Optus & Vodafone mght well be due to the fact that they share base station sites. Different equipment on the sites, but they both use an identical set of sites in Metro areas for 3G.

  3. I plan to travel to Sydney for 8 days around the New Year. What would be the best option for getting 3G data access on my phone? Is there a pre-paid option? I’m a US resident and I imagine a contract is out of the question?

    • I think the best thing to do would be to go to a Telstra retail outlet (“T-Life”) when you arrive and purchase a prepaid voice SIM with a “Browse Plus Pack” optional add-on for your 3G data needs. You’ll have 30 days to use it with no contract.

      http://telstra.com.au/mobile/prepaid/latestoffers/
      http://telstra.com.au/mobile/browsing_packs.html

      Hope this helps.

  4. [...] have been on Sydney’s Mobile Broadband Network all the time we lived in Australia, and the speed was never optimal in Internet peak-hours. [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: