Monday, January 30, 2017

Do-it-yourself rural fiber

M-PAC cable
I doubt that any elementary school in the US has fiber to the premises, but, in 2013, an elementary school in rural Bhutan was connected to the Internet using optical fiber in the "last mile."

They were able to connect the school because the cabling they used, metal-packed armored cable (M-PAC), which is modeled on undersea cables, does not have to be in a protective duct. It is 4mm in diameter, light and flexible, so it can be installed by supervised volunteers or unskilled workers.

As shown below, a portion of the cable to the school is buried in a hand-dug ditch and another link is suspended overhead:

The cable used in this installation was supplied by OCC Corporation, but last June the International Telecommunication Union (ITU) adodpted a standard for "low-cost sustainable telecommunications infrastructure for rural communications in developing countries," L.1700.

As a framework standard, L.1700 is largely technology-neutral. Technology-specific best practices are provided by supplement texts such as ITU-T L Supplement 22, which specifies the design of a low-cost, terabit-capable optical cable that can be deployed on the ground’s surface with minimal expense and environmental impact. For more on the standard and it's intended application, check this post.

We have major fiber backbones in large cities -- might we also have do-it-yourself backbones in rural villages?

Thursday, January 05, 2017

History is written and revised by the winners -- can the Internet Archive change that?

Kremvax during the Soviet coup attempt
I was naively optimistic in the early days of the Internet, assuming that it would enhance democracy while providing "big data" for historians. My first taste of that came during the Soviet coup attempt of 1991 when I worked with colleagues to create an archive of the network traffic in, out and within the Soviet Union. That traffic flowed through a computer called "Kremvax," operated by RELCOM, a Russian software company.

The content of that archive was not generated by the government or the establishment media -- it was citizen journalism, the collective work of independent observers and participants stored on a server at a university. What could go wrong with that?

Mumbai terrorist attack
The advent of the Web and Wikipedia fed my optimism. For example, when terrorists attacked various locations in Mumbai, India in 2008, citizen journalists inside and outside the hotels that were under attack began posting accounts. The Wikipedia topic began with two sentences:
The 28 November 2008 Mumbai terrorist attacks were a series of attacks by terrorists in Mumbai, India. 25 are injured and 2 killed.
In less than 22 hours, 242 people had edited the page 942 times expanding it to 4,780 words organized into six major headings with five subheadings. (Today it is over 130,000 bytes, revisions continue and it is still viewed over 2,000 times per month). What could go wrong with that?

The Arab Spring
The 2011 Arab Spring was also seen as a demonstration of the power of the Internet as a democratic tool and repository of history. What could go wrong with that?

What went wrong

The problem is that the Internet turned out to be a tool of governments and terrorists as well as citizens. Furthermore, historical archives can disappear or, worse yet, be changed to reflect the view of the "winner."

Our Soviet Coup archive was set up on a server at the State University of New York, Oswego, by professor Dave Bozack. What will happen to it when he retires?

If someone tried to delete or significantly alter the Wikipedia page on the Mumbai attack, they might be thwarted by one of the volunteers who has signed up to be "page watchers" -- people who are notified whenever the page they are watching is edited. We saw a reassuring demonstration of the rapid correction of vandalism in a podcast by Jon Udell. That was cool, but does it scale? Volunteers burn out. The page on the Mumbai attacks has 358 page watchers, but only 32 have visited the page after recent edits.

Even if a Wikipedia page remains intact, links to references and supporting material will eventually break -- "link rot." If our Soviet Coup archive disappears after Dave's retirement, all the links to it will break.

By the time of the Arab Spring, we were well aware of our earlier naivete -- the Internet was already being used for terrorism and government cyberwar and the dream of providing raw data for future historians and political scientists was fading.

The Internet Archive

Soviet coup archive from Internet Archive
I was slow to understand the fragility of the Internet, but others saw it early -- most importantly, Brewster Kahle, who, in 1996, established the Internet Archive to cache Web pages and preserve them against deletion or modification. They have been at it for 20 years now and have a massive online repository of books, music, software, educational material, and, of course, Web sites, including our Soviet Coup archive. As shown here, it has been archived 50 times since October 3, 2002 and it will be online long after Dave retires -- as long as the Internet Archive is online.

Khale understands that saving static Web sites like the Soviet Coup archive only captures part of what is happening online today. Since the late 1990s, we have been able to add programs to Web sites, turning them into interactive services. As such, he has recently begun archiving virtual machine versions of interactive government services and databases.

Khale is understandably concerned by the election of Donald Trump, who has demonstrated a keen ability to exploit the Internet and a disregard for truth. As such, he is raising money to create a backup copy of the Interent Archive in Canada and working to archive US Government Web sites and services.

The Internet is inconceivably large and growing exponentially. There is no way the Internet Archive can capture all of it, but it is the leading Internet-preservation organization today. Khale and his staff will continue their work and will inspire and collaborate with other relatively specialized efforts like that of climate scientists who are working to preserve government climate-science research results, data and services.

For more on the Internet Archive check out the following PBS News Hour segment (9m 12s):

You can read the transcript here.

I'd also recommend listening to this short (5m 14s) podcast interview of Brewster Kahle. He describes the End of Term project -- a collaborative effort to record US government (.gov and .mil) Web sites and services when a new administration takes over. He describes deletions and modifications from 2008 and 2012 and feels a special urgency today for obvious reasons.

You can read a transcript of the interview here.

Update 1/6/2017

The Internet Archive has launched the Trump Archive with 700+ televised speeches, interviews, debates, and other news broadcasts. Mention by a fact-checking site was the "signal" used for inclusion of a video and links to the fact-check document are included in a companion spreadsheet. I hope they use speech recognition to produce searchable transcripts as well.

Too bad we did not have Trump and Clinton archives during the campaign -- I hope we will have similar, timely archives in the future. One can even imagine similar archives for state and local campaigns if a crowd-sourcing system were developed.

Update 1/7/2017

There is an annotated PowerPoint presentation on citizen journalism here. I use it in teaching an Internet literacy class and there is a note on my PowerPoint presentation style here.

Update 2/5/2018

Speaker of the House Paul Ryan posted a clueless tweet bragging that the Republican tax-cut bill had saved a secretary at a public school $1.50 per week -- enough to cover her $60 Costco membership for the year. The tweet was soon deleted, but the Internet Archive had cached a Czech retweet:

Note that embarrassing news travels fast -- the day after the initial tweet, the Czech tweet had 954 retweets and 1,466 likes.

Note also that $1.50 per week will cover the secretary's $60 Costco membership with $18 to spare -- she can use that to cover the tax on her extra income.

Ryan is a more selective tweeter than Trump. For example, he did not Tweet anything when his fund-raising committee received a $500,000 campaign contribution from the Koch brothers shortly after the tax bill passed in the House.

Update 7/17/2018

Yesterday, the Trump Administration shut down the National Guideline Clearinghouse, a database of medical guidelines that for nearly 20 years has been a critical resource for doctors, researchers and others in the medical community.

The shutdown was necessitated by budget cuts -- I guess they had to make a tough choice between paying for the database or Tom Price's charter flights.

The Internet Archive Wayback Machine archived the database on July 15, but that is a static copy that will not be updated.

The administration has also been removing the term "climate change" from Web sites and there have been several efforts to back up client-science data, for example this and this.

Tuesday, January 03, 2017

Package delivery -- the other "last mile" problem

We've had bad luck with package delivery during the last six months:
  • An order of kids rain boots from Walmart was stolen from our front porch. Unfortunately, a small box containing a ring from TheRealReal was delivered at the same time and was also stolen.
  • Walmart replaced the rain boots and TheRealReal gave us a refund, but my wife was disappointed not to get the ring.
  • We received a package from TheRealReal via Federal express. It should have contained a bracelet, but it was empty. Again, we received a refund, but not the gift. It may have been taken by someone at TheRealReal or Federal Express.
  • We ordered a pressure cooker from Amazon. The package it came in was marked "fragile" but was in poor condition. We opened it, saw two dents in the pressure cooker and returned it for a refund.
  • We ordered a blanket from SweetDreamsHome, an Amazon Marketplace retailer. The order was placed on December 14 and scheduled for delivery. We planned to be out of town on the scheduled delivery date, so requested a different and were assured it would arrive on December 23. It did not arrive on that date, so we contacted Amazon. We were assured that it would arrive on December 28th. It did not. When it did not arrive on the 29th, we cancelled the order. It arrived on the 30th.
Amazon and the others were extremely polite and responsive and we received prompt, no-hassle refunds, but we were disappointed, a Christmas gift was late and we had to be worrying that a package might come while we were out and unable to sign for it or, worse, that it would be stolen.
I checked the American Customer Satisfaction Index of the Consumer Shipping and Internet Retail industries and found that their scores of 80 out of 100 put them in the top six of 43 industries surveyed. (Internet service providers were ranked last because there is little competition in the industry).

That being said, the survey only considers the US Postal Service (74), UPS (80) and Federal Express (82). The private companies are rated higher than the Postal Service and all three have been relatively stable over time. (The US Postal Service moved up in the late 1990s, while UPS and Federal Express have slipped a little).

Fortune magazine says Silicon Valley venture capitalists are giving up on on-demand delivery and I am not expecting on-demand drones or robots or self-driving delivery trucks any time soon. (If they do, the thieves may start stealing drones and robots as well as packages). Are vendor-agnostic local pickup locations a solution?

Maybe this was a run of bad luck and we plan to keep shopping online, but not as frequently.