More on maps
We got a chance to see the early-stage maps from ConnectMN last Friday. So far I'm a little confused as to how I, as a policy-maker, can use them. I'm not sure they can be used for anything, but hey I'm not the brightest bulb in the box...
They're not detailed enough -- I checked a few places that I know don't have any high-speed Internet available. The maps show availability in each. Lots of other people have written me with the same finding. My guess is that this is because the maps aren't being drawn with sufficient granularity to show the dead spots.
They're using a low-bar definition of broadband -- They're using the FCC 750k/bps definition. How quaint.
The underlying data is secret -- They've collected all the provider-based information under non-disclosure agreements. Shame on us (the State) for putting up with that. We have no way to verify their results.
They aren't using the speed-test data they've collected, they're relying on Speedtest.net's data instead -- Due to the rumpus Aileen Horwath kicked off a week or so ago, they've thrown out the data they collected and are only using data from SpeedTest.net for their measurements.
Speedtest.net is a biased measurement that favors high-speed users over dialup, which skews the data -- Go ahead, try to go through Speedtest.net's tool from a dial-up connection. I dare you. Speedtest has this cool graphical front-end that works great if you're on a high-speed connection, but takes forever to load if you're on dial-up. I mean, 10-20 minutes. So dialup-users aren't represented in Speedtest's data, a huge "sample bias" that any statistician will tell you is a fatal flaw and completely invalidates any conclusions that are drawn from the data.
"Averages" are misleading -- ConnectMN came in to our meeting all perky about how zippy Minnesota is. "This is the fastest state we've ever measured" they chirped during their presentation. They had already been dinged about this in a prior presentation, so they're going to back the Comcast-dominated Twin Cities area out of the number and try again. They were quizzed about the way they constructed those averages a lot by other task force members during our meeting (I was dialed in over a dreadful connection, so I just listened). I didn't come away with warm and fuzzies after hearing their explanation.
Their staff seem biased against municipal projects -- I don't think the State paid these guys to tell us what to do, they got paid to produce maps. But during their testimony one of their staff folks stoutly announced that "all municipal projects have failed." HUH?? what up with that? I bet there are a bunch of folks in various municipalities around the country who would beg to differ. I don't have a problem with people who have that opinion, but I'm concerned when it's my supposedly-unbiased contracted map-maker who's expressing it.
So like I said, I'm puzzled about what use these maps will be for making policy. So far I'm in the "did we really get what we paid $164,000 for?" camp.
February 16th, 2009 at 5:51 pm
Mike,
Some interesting info.
An Orono LMCC Commissioner identified an area of his city shown with broadband DSL availability on the map, but which, in fact, has neither DSL available (he checked with Qwest) nor Mediacom cable.
He ran the Connected Nation speedtest and the results were:
XXXX Watertown rd orono
down 44kbs up 312 kbs latency 311
XXXX Co Rd 6 orono
down 34kbs up 116 kbs latency 243
Of course, the upload speeds are nonsensical (they are 5 times faster than the modem could possibly support). So, I decided to try from my house with my laptop modem on my dial-up (XXXX Enchanted Cove, Shorewood).
The connection speed as reported by the dialer was 26.4kbps. I tried five different tests. The results follow. It seems that several are using the test engine from Netmetrics called Ookla. Connected Nation uses this one. This engine gives wacky results for upload speed.
ConnectMN.org (Connected Nation) test
down - 24kbps
up - 132kbps (nonsense -- shows 5 times as fast as the connection could be, given that I'm connecting by 26.4k modem)
Speakeasy.net (Chicago server, uses Okla engine)
down - 24kbps
up - 123kbps (again, nonsense -- same problem)
Speedtest.net (Twin Cities, MN server)
down - 24kbps
up - 122kbps (same problem -- 5x the possible speed)
Foxvalley.net/speedtest
down - 35kbps (almost nonsense -- unsure how this can be more than reported modem connection of 26.4kbps)
up - 37kbps (Also unsure of how this can be, but it's more reasonable that the above tests)
speedtest.tds.net (New York server)
down - 22kbps
up - 22kbps
So, of all of these, only the speedtest.tds.net seems to report reasonable results with a slow dial-up connection (they also report reasonable results for my Mediacom cable Internet connection). My conclusion is that all dial-up speed test results collected by Connected Nation should be thrown out. All other results should be audited to look for the wacky upload speeds vs. download ones.
Finally, there is positive proof in Orono that the map is incorrect. I'm still working on getting other city test results to confirm that only dial-up is available in other areas where CN's map shows DSL.
Dick Woodruff