Thursday, July 28, 2016

The digital divide has persisted over the life of the Internet.

National Bandwidth Potential, a novel Internet diffusion metric indicating application feasibility, shows a persistent digital access divide.

People have been trying to measure the global diffusion of the Internet and the digital divide between rich and poor nation for twenty five years. The first to do so was Larry Landweber, who noted whether or not a nation had an Internet (or other) connection. It was a binary metric -- yes or no -- and it was suitable to its time because there were only a handful of users who were restricted to teaching and research, using a few applications like email, file transfer, news groups and remote login.

1991 Internet diffusion (purple)

Five years later, the Internet had many more users and applications in commerce, government, entertainment, etc., so my colleagues and I developed a multidimensional Internet diffusion framework. One of our dimensions was pervasiveness, based on the number of users of the Internet per capita.

That made sense in 1995 since there were relatively few applications available for the slow dial-up connections of the time. A few people had faster ISDN or DSL connections and an organization might connect over a faster digital link, but most users were running the same few applications over analog phone lines.

Today, users per capita is pretty well meaningless. A Cuban who accesses email using a 2G cell phone and a Google Fiber user who has symmetric gigabit access to multiple computers and devices on a home LAN are clearly not equal.

To some degree, we anticipated this sort of thing via the connectivity infrastructure dimension in our framework. It considered international and intranational backbone bandwidth, Internet exchange points and last-mile access methods, but it was an imprecise measure -- mapping a nation into five levels -- and data was not readily available. (Our case studies typically required two weeks of in-country interviews).

Skipping ahead twenty years, a paper by Martin Hilbert uses an interesting diffusion metric -- nationally installed bandwidth potential (BP), which is a function of the number of telecommunication subscriptions (fixed and mobile), the kind of access technology per subscription (cable, DSL, GSM, etc) and the corresponding bandwidth per access technology. Their estimation of the latter is quite complex, taking factors like data type, upload/download speed, compression, etc. into consideration. The methodology is described in a ten page supplement to the paper. (It is behind a paywall -- let me know if you would like a copy).

Hilbert computed the BP of 172 countries from 1986 to 2014 and observed that the digital access divide is persistent. It is true that wireless connectivity is relatively inexpensive and mobile Internet use is growing rapidly in developing nations, but it is just as clear that many applications are precluded by the speed and form factor of mobile devices. A WhatsApp chat with a friend is not the equivalent of watching a high-resolution movie on a large screen TV and I am confident that Hilbert did not conduct his research or write the paper I read on a mobile phone. Even reading this blog post, following its links to other documents and taking notes on it would be tedious on a phone.

I expect this imbalance to persist because improved technology is costly and it enables ever more complex, demanding applications. The only trend I see that may in part reduce this feasible-application gap is the move to server-side processing for big data and AI applications, but even then interaction and the display of results will require bandwidth.

Hilbert's data also shows global shifts in application feasibility. As shown below, BP dominance has shifted from the US in the early, NSFNET days to China today. Korea has joined the top ten and the shares of Japan and Western Europe have dropped. The share of the bottom 162 countries rose slightly in 2001, but had fallen below the 1986 level by 2014.

Ten countries with most installed bandwidth potential

Income differences explain much of the persistence of the digital divide, but policies regarding Internet infrastructure ownership and regulation are also important. For example Estonia ranks 40th in the world in GDP per capita, but is ranked 20th on the International Telecommunication Union ICT Development Index.

Policy choices may play an even larger role among the top ten nations. The US ranks 9th in GDP per capita and Korea is 30th, but my son, who lives in Korea, pays $22 per month for symmetric, 100 mbps connectivity and has a choice of several competing Internet service providers. I live in the US and pay considerably more than he does for considerably slower service and have no ISP choice -- I am stuck with Time Warner Cable.

While we are waiting for enlightened policies, we can hope for technical change like the OneWeb and Spacex satellite Internet projects.


Tuesday, July 12, 2016

Coverage of the 2016 Tour de France -- big data

Gathering real time data on each rider enables a clean video user interface, real time presentation of the race status and post race data analysis.

For several years, I wrote posts on streaming coverage of the Tour de France, Olympic Games and the Tour de California. Those posts focused on topics like user interface, ads, video quality and comparison of NBC's coverage with that of the BBC.

I missed last year due to travel, but am watching the current Tour de France, and there have been significant changes for the better.

For a start, NBC now bundles coverage of the Tour de France with several other races, so one purchases an annual subscription. That means cycling fans can see more races and, presumably, that the archive footage will remain accessible at least during the year.

(In the past, both NBC and the BBC have deleted their archives some time after the end of the Tour. I believe they have an information stewardship obligation and should maintain the archives of important events for analysis by journalists, scholars, fans, remixers, etc. The cost of doing so would be low and, if they were not behind a paywall, they could be found by search engines.)

The video quality is also better than I recall -- a consistent 2.2 mbps stream with none of the dropouts we saw during 2014.

The user interface has been simplified since 2014 when it had five modes -- live video, standings, stages, riders and more:

2014 Five viewer modes

and you spent most of your time in the four-frame Live Video mode:

2014 four-frame Live Video user interface

By contrast, the live video UI this year is simple, with a small race status indicators like the time between the race leader and peleton in the screenshot below, popping up from time to time on a full video screen with customary controls at the bottom:

2016 live video user interface

At first stripping out ancillary information might seem a step backward (or forward if you are an Apple minion), but it is not. Much more ancillary information is available this year and it is accessed through a "Tour Tracker" site. The Tour Tracker allows you to see in-depth information for each stage, with tabs for Teams, Stages, Standings, Results, Recaps, Replays and Photos and a link to the live video window shown above.

2016 Tour Tracker user interface

All of that data is available because the race is now very well instrumented. Each bike has a small GPS transponder affixed to the seat.

GPS transponder

The data from the transponders is uploaded to the mobile data center of Tour partner (and team sponsor) Dimension Data, enabling them to provide live data during race -- check out the following video (2m 50s).


This data collection enables Dimension Data to provide real time status of the race, individual riders, teams, etc. In the example below, we see the speed of several riders, the time gaps between them and the distance from the leader to the finish superimposed on the live video window.

Status update on Live Video viewer

In addition to real-time status statistics, Dimension Data is able to analyze data after a stage is complete. For example, the following image shows that the stage 6 sprint winner, Mark Cavendish, accelerated a little bit later than the second and third place finishers. I would expect that this sort of data is helpful to the racers and their managers. (The teams receive some information that is not available to the general public).

Post stage analysis

GoPro cameras are another source of Tour data. Since 2015, GoPro has been a Tour partner and they had cameras on cameras on official cars and motorbikes, team cars, mechanics and selected bikes. Fellow Tour enthusiast Jim Rea spotted some live GoPro footage during stage 1, but has not seen any subsequently. That being said, you can see archived video after the stages are complete by searching on Google for "GoPro: Tour de France 2016 - Stage n Highlights" site:youtube.com, where n is the stage number.

The following video is not from the Tour de France, but it shows what it is like to be in the sprint at the end of a race.


And the following video shows a crash from a mechanic's point of view:


(A 360 degree virtual reality versions would be cool).

The NBC package also includes a free mobile app. The live video on the Android app has a simplified user interface with only a pause/play toggle. The data view offers the options shown below, but it is not as complete as that of the Tour Tracker Web site.


I watch The Tour on a wide-screen laptop with a 3200 by 1800 pixel display and toggle back and forth between the Tour Tracker and video windows. An alternative would be to run the mobile app on a smart phone, casting it to a TV set and using a laptop or tablet for the Tour Tracker.

The bottom line is that race coverage had improved significantly since I last watched The Tour. The video quality has improved noticeably and the addition of real-time GPS data has added to the experience. I didn't even bother trying a VPN tunnel to check the BBC coverage.