Maplight.org and other public data mashups

We discuss mashups, and public data is becoming available for mashing up on the Internet.

Consider Maplight.org. Using Maplight, one can correlate voting on a bill with contributions. For example, the Medicare Prescription Drug Price Negotiation Act of 2007 requires the Secretary of Health and Human Services to negotiate lower drug prices on behalf of Medicare beneficiaries. As we see below, interests wanting this bill to pass contributed more to legislators voting yes than to those voting no.



Maplight currently has data for the Federal and California legislatures, with other states under construction. For more on their capability, see this video tour.

Prominent blogger Jon Udell has a strong interest in the availability of public data on the Internet. You can see what he is finding and learning by tracking his public data bookmarks.

Who are your representative and senators? Which special interests are supporting them? How are they voting on issues of concern to you? What data does your local government publish on the Internet? Do they make it easy for users to access it and create mashups?

Creating and sharing slidecasts

A slidecast is a slide presentation with synchronized audio, and Slideshare.net is an Internet service for creating and sharing slidecasts. As shown here, the user can watch continuously or skip from slide to slide using the viewer controls.

One creates a slidecast by uploading a PowerPoint, PDF or OpenOffice presentation with the slides and an mp3 file with the audio track. The site provides editing tools for selecting and adjusting the portion of the audio file to play with each slide. You can see how it is done in this how-to screencast.

The slideshare site also has the features of a typical social networking site, allowing viewers to comment on, evaluate and search for slideshows. You can also upload and share a slide presentation with no audio track.

Do some of your professors use slide presentations in class? Would it be helpful to have them available for viewing after class as slidecasts? Have you made a presentation that you could have recorded as a slidecast?

Exponential growth

I added a new course note explaining the idea of exponential growth -- growth at a constant rate.

Storage progress -- every movie ever made on an iPod

We discuss the progress in electronic, communication and storage technology that underlies advances in network-based applications. Robot designer Rodney Brooks uses the Apple iPod to illustrate improving storage capacity. If the current exponential rate of growth continues, an iPod will be capable of holding every movie ever made by 2020. (That includes all the bad movies -- he figures all the worthwhile movies will fit by 2016).

Brooks made the point in a talk outlining his work on robotics. You can hear an excerpt on the growth of iPod storage here, listen to the entire talk here, or watch videos of a robot in Brooks' lab interacting with humans here.

How old will you be when your iPod has the capacity to hold every movie ever made? What other applications might it be used for? Will it be useful to you in your work and career?

Why US connectivity is lagging

We have discussed the decline of the US in terms of connectivity. A recent article in the Economist asks why some developed nations are doing better than others, concluding that "The best explanation, in fact, is that broadband thrives on a mix of competition and active regulation, to ensure an open contest".

In South Korea, where half the population lives in large apartment buildings, each owns its own internal cabling and allows rival operators to install equipment in the basement. Tenants choose which to use. In Japan, politicians pressured the dominant operator, NTT, to connect people's homes by high-speed fiber lines. The Communications Ministry has indicated that it will make NTT open those lines to rivals.

In theory, France, Canada and the US also passed laws requiring incumbent telephone companies to allow competitors access to their local lines. Those laws have succeeded in generating competition in France and Canada, but not in the US where regulatory and court decisions have enabled the telephone companies to keep competitors out.

Average advertised download speed in Japan is 95 megabits per second. How does that compare to the speed in your home? What sorts of applications would such high speed make possible? In Denmark, the Netherlands and Switzerland, over 30% of inhabitants have broadband. In the US it is around 22%. How might that impact the economy?

WiMax deployment begins

We have seen that DSL and Cable are the two most common technologies for home connectivity, but there are alternatives. WiMax wireless is beginning to roll out.

TDS Telecom has just announced the availability of fixed WiMax service to 55,000 households and 10,000 businesses in Madison Wisconsin. Residential service is $45/month for 2 Mbps, $50 for 4 Mbps, and $55 for 6 Mbps.

Note that this is connectivity to a fixed location, an alternative to DSL or cable. Mobile WiMax, which will compete with cellular service, has not yet been deployed.

TDS also offers DSL connectivity in Madison. The rates are $30 for 768 Kbps, $40 for 1.5 Mbps, and $50 for 4 Mbps. At these rates, which would you prefer? Are both available throughout the Madison area?

Infrastructure services -- for developers, not users

We have discussed many Internet services intended for applications like editing documents or developing simple applications like blogs or online databases. These are generally intended for users -- people who use the Internet in their work or play, not professional developers.

We can contrast these user services with infrastructure services, which are intended for professional developers building user services or custom applications. Amazon has been a leader in this field. They learned to operate vast, efficient, reliable server "farms" in support of their shopping site, and decided to allow others to use the same infrastructure. Amazon Web services include virtual servers, storage space for large amounts of data, and storage and retrieval of data from flat files.

3Tera's Applogic offers a graphical user interface for designing a virtual server farm and deploying applications. In this example, the drag and drop interface was used to create a site with four firewalls (gray), a load balancer (yellow), two Web servers (blue), a MySQL server (red) and servers storing configuration files and logs (orange).



Each of these is an independent virtual server running Linux. Developers can configure these machines and load software on them as if they were in sitting on their desks. They create "dashboards" to monitor them, and can increase or decrease capacity in seconds. (Check out this Applogic video demonstration).

These services allow one to develop applications without hardware or datacenter cost. They bill for resources used -- CPU time, storage, bandwidth. That means there is essentially no cost while an application is being developed and debugged since there is no traffic. When the application goes live, the capacity, and hence cost, grow and shrink dynamically depending upon utilization.

Storage and computation are not the only infrastructure services. For example, Amazon offers a service for handling monetary payments from computers or users to other computers or users and Lypp.com offers telephony services. For example, a developer can give the user the ability to initiate a call between their telephone and a company. Lypp servers establish the call, interface with the switched telephone network, and bill for minutes used. Again, there is no traffic, so no cost, until the application is put in use.

Infrastructure services are the latest step in server evolution. Would infrastructure services appeal more to large or to small and medium sized organizations? Why? We offer courses today on programming for vendor-specific platforms like ASP.NET. Will we one day offer a course in programming for Amazon Web Services? Would you like to take such a course?

Best practice: what to do when a service outage occurs

We have discussed the pros and cons of relying on service providers. Reliability is one issue, and, when something goes wrong, you want fast action and continuous status reports. 37Signals, provider of Basecamp project management and other services, responded well during a recent two hour service interruption. The following log was posted on their Web site, fully informing users of the problem and progress toward its solution.

• All systems are currently offline as we're experiencing network outage from our provider. We're working on it right now. No data has been lost, all our machines are still working, but they're not accessible from the internet. Sorry for the inconvenience — 10:03am CST (16:03 GMT) on January 18, 2008.
• We have located the problem to be with the load balancer setup. A new unit is being installed. We should be back shortly. Again, we're terribly sorry for this disruption of service — 10:28am CST (16:28 GMT) on January 18, 2008.
• The technicians at our service provider are still working on installing the new load balancer. We're breathing down their neck as heavily as we can. And we profusely apologize for this unacceptable interruption of service — 10:56am CST (16:56 GMT) on January 18, 2008.
• The load balancer has been swapped and is currently being configured. We should be in the home stretch now. Again, we're incredibly sorry for this disruption. This is not how Fridays are supposed to be — 11:18am CST (17:18 GMT) on January 18, 2008.
• Our service provider is still working on the configuration of the new load balancer. We're on their case every few minutes to get updates. It's hard not to be very disappointed that a simple load balancer replacement can take this long for someone who's supposed to be the best in the business. We're out of new ways to say we're sorry, so we'll just say it again: We're so sorry for this — 11:50am CST (17:50 GMT) on January 18, 2008.
• The latest from our service provider is that the reason it's taking so long is because the configuration for the load balancer lives on a CF card that has also gone bad. The configuration is currently being rebuilt from scratch — 11:57am CST (17:57 GMT) on January 18, 2008.
• Now we finally have access to the servers with remote access. We should be able to bring the services back one by one now. Looks like there's finally light at the end of the tunnel — 11:59am CST (17:59 GMT) on January 18, 2008.
• Basecamp is now back. We're bringing back everything else we can as fast as possible. It might take a few minutes for the DNS access to update, but it's coming back — 12:05am CST (18:05 GMT) on January 18, 2008.
• All the products should be coming back online now as soon as the DNS updates. It seems like we're coming out of the woods entirely7 — 12:13am CST (18:13 GMT) on January 18, 2008.

Could an outage like this occur if the user ran their own application? Would in-house staff have responded as well as 37Signals staff did to this outage?

Video of a Google Android cell phone prototype

We have seen that a dumb, open network encourages investment and hardware and software innovation. We have contrasted this with the telephone and cellular networks, where application and hardware innovation are controlled by the network operators. We've also followed Google's efforts to open the cellular network by possibly purchasing spectrum and by developing and distributing Android, an open source mobile phone operating system bundled with Google applications like maps and mail.

Google has released some videos demonstrating prototypes of Android phones. For example, the following video shows a touchscreen phone reminiscent of the Apple iPhone.



(You can find other Android-related videos at the Android Web site linked to above or by searching on Youtube).

The most important feature shown in the video is not clever hardware or software -- Apple already did that with the iPhone -- but the open platform. The video begins and ends with Google co-founder Sergey Brin inviting developers to build Android applications. Based on his Internet experience, he feels that openness will be critical to the success of Android, and has kicked things off with a ten million dollar contest for application developers.

If he is right, third party applications will spur Android sales, and each Android phone sale will spread Google applications -- and ads. Not satisfied with being on your desktop, Google also wants to be in your pocket.

After viewing the prototype video, can you think of applications you would like to have on your next cell phone? Would you be willing to watch targeted advertisements in return for free cell phone service?

The OLPC XO, a LAN machine in the Internet era

We have talked about the One Laptop per Child (OLPC) computer called the XO. Sales have been slower than hoped and Intel has left the OLPC coalition because they were unwilling to stop selling their low cost Classmate PC in developing nations. An Intel sales person tried to unhook the OLPC sale to the government of Peru, and OLPC head Nicholas Negroponte compared that to McDonald's competing with the World Food Program.

But, the Classmate is proving a strong competitor to the XO in spite of its higher price. Is Intel selling it below cost to eliminate a competitor or is the Classmate a better product in the eyes of the education ministers and other government officials who are the target market. (Both are being marketed, appropriately, as ubiquitous educational infrastructure).

I've not seen the Classmate, but have an XO, which I find disappointing. The hardware is innovative and appropriate for kids and rural areas in developing nations, the environment for which it was designed, but, it is a pre-Internet machine. We have outlined the evolution of application development and delivery platforms from batch processing to the Internet, and the XO is still primarily a LAN machine in the Internet era.

It can connect to the Internet via WiFi (if you have an open access point), but the browser cannot handle mp3 and Flash files, foregoing tons of kid-oriented material. Such shortcomings may be overcome with software upgrades, but that will make a more fundamental problem apparent -- XO desktop applications like image, audio and video processing will seem lame compared to their Internet-based counterparts. You can't keep them down on the farm once they've seen PBS Kids or Jumpcut.

One can argue that most XO users will be on XO LANs in locations without Internet connectivity so they won't know what they are missing. That is true, but sad -- second rate technology for the third world.

Mr. Negroponte discussed Internet connectivity in a presentation at the MIT Emerging Technologies Conference in 2005, stating that

It is not a solved problem, but there are many people and many systems working on it. Wifi, WiMAX, GPRS, 3G, 4G, fiber, on and on and on. ... (Connectivity) is happening. It doesn't need me, it doesn't need MIT, it doesn't need the media lab.

Negroponte believed that local interest, competition and regulatory reform would take care of the connectivity issue. Unfortunately, as I have argued elsewhere, global competitiveness and regulatory change is not leading us to connectivity, but to a growing gap between developed and developing nations.

Uruguay, an OLPC customer, recognizes the need for connectivity. They will be providing connectivity to OLPC schools. I hope the XO is a better Internet access device by the time those Uruguayan children get theirs.

Where the Internet is not available, the XO seems appropriate. It is designed for the LAN. The concept of connecting to and sharing with local XOs and a local server is "baked in." The only flaw there may be speed, which I was not able to test. (A mesh network with single radios in each machine could be mighty slow).

Connectivity is beyond the control of OLPC, but I was also disappointed with aspects of the user interface. Kids who are forming a mental model of the machine and network need immediate feedback. When a kid is exploring, every click and gesture is an experiment. The cursor must always change shape when over a hot spot, every click has to be executed immediately or, if that is not possible, an "hourglass" or explanation of what is happening needs to appear, etc.

If the target market is the LAN, the XO is a reasonable alpha-test in need of rapid software upgrades. The only way to achieve that in a timely manner is for a strong open source development community to form around the machine. OLPC would do well to encourage the formation of a social network of developers and support them with Windows and Mac-based development tools.

In the (slightly) longer run, it needs to make the transition from a LAN machine to an Internet machine, and, as in Uruguay, the rollout needs to be coupled with connectivity.

Earth View, a cool astronomy service

We discuss both mashups and location-based applications. This image shows the earth from a 1,000 kilometers above our campus. It will be dark or light depending upon the time of day.

It was generated using Earth Viewer, which can display various views of the Earth from various vantage points, for example, from the Sun or Moon. I used Geocoder to get our campus coordinates.

With the coming of low cost, location-aware cell phones and other portable devices, people predict that we will see a lot of location-based advertising. Can you think of other location-based applications? Can you think of applications you could build on top of Earth Viewer?

An excellent computer history course

We cover the history of technology and networks, and the University of Washington, UCSD and UC Berkeley offered an excellent course on the history of computing in fall 2006.

The course material is online, and includes short essays by the faculty, links to original material, and presentations by faculty and prestigious guest speakers, including many computing and networking pioneers.

Many of the presentations include photos of historic equipment and events. For example, the photo shown here was taken behind the scene of Doug Engelbart's historic 1968 demonstration of his work on personal and collaborative computing.

Note that the course material is organized using a wiki. It illustrates the use of a wiki as a simple, flexible platform for creating a Web site to support an ad-hoc project or organization.

Historic prototypes look old fashioned to us, like early cars or the Wright Brother's first plane. How will we see today's technology in the future? Is technology changing as fast today as in the past?

SlideShare.net, a service for making and hosting narrated slide show presentations

Intellectual property expert Larry Lessig became known for presentations built around narrated slide shows delivered as Flash files. This well-known presentation on US copyright law illustrates his style. Doug Kaye and Paul Figgiani used the same technique in narrating a presentation on recording telephone calls during interviews.

Kaye and Figianni used SlideShare.net to convert a Keynote presentation into a Flash movie (PowerPoint also works). The presentations are also hosted at SlideShare.net, and the user can watch straight through, pause the presentation or skip forward or backward.

What do you think of the narrated slide show as a presentation format? What are its pros and cons? Are there topics you would like to present using this technique?

A terrific how-to on recording broadcast quality telephone interviews with Skype

We cover audio processing, and Doug Kaye and Paul Figgiani have put together a terrific 22.5 minute presentation on recording interviews using Skype. You no longer need expensive equipment to make broadcast quality recordings of phone calls. A Mac or PC running Skype, a USB head set, a low-cost audio capture program, and this presentation are all you need. (Even if you are not recording telephone interviews, much of what they say applies to audio recording and processing in general).

For discussion of the presentation and more on audio recording and editing, see Kaye's Conversations Network forums.

Are there people you would be interested in interviewing?

A Web 2.0 book

Susan Kelley spotted Dive Into® Web 2.0, a book by Deitel & Associates. The book covers many of the same topics we do, and there is also an extensive online resource center -- you should check it out.

Like other publishers we have discussed, Deitel offers electronic and print versions of the book. You can read the HTML version free online or order a PDF file or a hard copy from on-demand publisher lulu.com. It will be interesting to see if they are able to keep the HTML and PDF versions synchronized.

This 81 page book is $19.95 in hard copy, $9.95 as a PDF file and the HTML version is free. Assuming you want a book, which version would you select at those prices? Would you like to read it on a portable device?

Google and the opening of the cellular network

We discuss the open architecture of the Internet, a "dumb" network, that connects smart devices. Users, not the network operators, decide what hardware to use and which applications to develop. As such, we see rapid innovation and massive investment. We have contrasted this with the cellular telephone network in which the network operators control the hardware and applications that are developed.

The cellular network is beginning to open. New spectrum is being auctioned off and Google is lobbying the to open the cellular network. They are dominant in desktop Internet services and advertising, and would like to exploit an open cellular network to dominate mobile Internet services and advertising.

During the last month, the New York times has published four articles on the opening of the cell network and Google's role in it.

• I, Robot: The Man Behind the Google Phone
• Google Enters the Wireless World
• Cellphone Straitjacket Is Inspiring a Rebellion
• Google Says It Will Bid for Wireless Spectrum

Individuals, organizations and the economy would benefit from an open cellular network, but do you worry that Google, if successful, may become too powerful?

The Internet and urban migration in developing nations

We have discussed the digital divide between rich and poor nations and proposals for closing it. We have argued that closing the digital divide would improve the quality of rural life in developing nations, thereby reducing the impetus for migration from rural areas to urban slums.

A recent estimate holds that half of the world population now lives in urban areas, and the UN predicts that will grow to 59.9 % by 2030.



This will lead to increased crowding and the growth of megacities. For a description of one such megacity, Bombay, see this book by Suketu Mehta, or listen to this interview.

Do you believe the Internet could improve life in rural areas of developing nations? Do you believe that would cut migration to cities?

Running applications from flash drives

We have been able to use the limited-function computers in campus labs by running applications like Filezilla and GIMP from flash drives. Fred Langa has published a two part series on the topic: Part 1 and Part 2.

The articles review available flash drives, sources of portable software (like PortableApps), and operating concerns like security, privacy and backups.

Portable devices -- innovation and form factors

In discussing mobile and portable connectivity, we talked about competing device form factors and innovation. We have also discussed new devices which may influence future form factors and user interfaces: the One Laptop Per Child Foundation (OLPC) laptop, the Apple iPhone, and, most recently, Amazon's Kindle.

Each has hardware and user interface innovations that may become standard fare on future portable devices. For example, the Kindle and OLPC laptop have low power displays that are legible in daylight, and the iPhone has an elegant touch screen interface and can be programmed to automatically switch between displaying documents in landscape or portrait mode if it is rotated. The OLPC laptop automatically forms a mesh network with nearby machines; the Kindle is directly connected to Amazon via Sprint's cellular network; and the iPhone can move between WiFi and AT&T's cellular network. The list goes on.

They each deliver different functions. The Kindle is a book reader, period. The iPhone a phone, Internet browser, email client, camera, etc., and it is now open to third party developers. The OLPC laptop is a PC running Linux. It comes preloaded with applications and there are tools for what I hope will become a vibrant developer community. It was conceived of as a computer for school children in developing nations, but in many ways it is more attractive than a "serious" compact laptop like a Sony Vaio for a business traveler.

Each is physically different:

	iPhone	Kindle	OLPC
Height (in)	4.5	7.5	9.5
Width (in)	2.4	5.3	9
Depth (in)	.46	.7	1.25
Weight (oz)	4.8	10.3	51
Pixels/inch	183	167	200
Resolution	420x320	600x800	1200x900
Screen diag (in)	3.5	6	7.5

The iPhone is a smart phone with a bright and large enough screen to browse the Web and read email fairly comfortably. The Kindle is designed for easy reading, and the OLPC laptop's high resolution display may be even better.

Which form factor do you favor? Are you willing to carry around a 3 pound OLPC laptop? Is the iPhone screen large enough for your portable applications? Would you rather carry a Kindle or a paperback book and a magazine when you board a plane?

Portable devices -- the Kindle book reader

Over the years a number of vendors have marketed portable devices for reading books, magazines and other material, but none have caught on. The latest attempt is Amazon's Kindle. Kindle is both hardware -- the portable reader -- and a service. The service includes an online store with 90,000 books, magazine and blog subscription and free downloading using Sprint's cellular network. It also includes a backup copy of everything you buy or transfer to your Kindle in case it is lost or damaged. That is the good news.

The bad news is that the Kindle service is a "walled garden." It is like having a cell phone that can only call one number -- the Amazon book store. Furthermore, the cost of downloading and backup must be covered, so a subscription to a blog which is free on the Internet might cost $1 per month, and you must pay to put your own Word files or other documents on your Kindle. The charges cover the backup and downloading service.

Would you be willing to have a specialized book reader or would insist upon a portable device capable of other functions like listening to music, Web browsing and email? Would you want a portable device that was tied to a single vendor, Amazon?

Who needs the broadcast television networks?

The Los Angeles Times and the New York Times both ran recent stories on entertainment programs being produced for the Internet by successful, mainline TV producers and writers. They are attracted to the new medium by creative control and cost savings on distribution. You can check episodes of two of them at The Fantastic Two and Qarterlife.

For more insight into the background of TV production and what is motivating these successful men to move their talents from television to the Internet, see the companion LA Times article Are Corporate Suits Ruining TV?". (You can guess the answer to that question).

Watch a couple episodes from one or both of these programs. How do they compare to TV? How do you think they could be improved?

Will open WiMax disrupt the closed cellular networks?

As we have seen, WiFi and 3rd generation cellular networks are the primary options for mobile and portable connectivity today. But, WiFi covers only limited areas (though there are millions of them), and cell networks are closed to innovation and focused on a business model of selling voice and other expensive services rather than pure Internet connectivity.

One day, Google and others may force the cellular networks to open up, but it will be a struggle. Another possibility is that a new standard, WiMax, will provide viable wireless Internet access.

We have spoken of WiMax earlier, and the WiMax leader at this time seems to be Clearwire Communication led by industry veteran Craig McCaw. You can read of McCaw, Clearwire, and their possibly foundering negotiations to combine WiMax networks with Sprint in this Wall Street Journal article.

If WiMax sounds promising, you can follow standards and industry progress in blogs written by Glenn Fleishman and Steve Stroh both of whom know a lot about wireless technology and business. Fleishman has just written a positive review of Clearwire's test deployment of pre-standard mobile WiMax in its home city of Seattle.

Do you plan to subscribe to either Stroh's or Fleishman's blog? What industry blogs do you subscribe to?

New notes on mashups and rating, identity and reputation

I posted three new class notes with assignments today. Two are on mashups between Blogger blogs and Talkr, a nifty text-to-speech service. Talkr essentially converts your blog into an audio podcast using a one-line API. Users also have the option of listening to rather than reading individual posts.

The other new note is on rating, identity and reputation. It gives several examples, and the accompanying assignment has the students add a rating widget from Outbrain.com to their blogs.

The notes and accompanying assignments are listed under Applications in this list of notes and assignments.

Can Google open the celllular network?

As we have seen, the cellular network is closed. On the Internet, anyone can deploy an application, but most cellular operators control applications. This problem was highlighted by John O’Rourke, general manager of Microsoft’s Windows Mobile business, who pointed out that it had taken Microsoft more than half a decade to get to its current level -- doing business with 160 mobile operators in 55 countries. Even Microsoft has to negotiate with mobile operators one at a time.

Mr. O'Rourke was quoted in a New York Times article on Google's plan to provide open-source telephone software and application development tools. They have formed a coalition to develop and promote this software. The coalition, called The Open Handset Alliance, includes a number of large telephone manufacturers and network operators, but others, like Apple, Verizon and AT&T are noticeably missing.

Columnist Robert Cringely predicts that Google will go further and become a cellular operator by bidding in the FCC auction of 700 MHz spectrum which will be freed up by the switch to digital television broadcast. If that happens, he expects them to offer free, ad-supported cell phone service.

Given the Apple iPhone and Google's plans, what do you expect your cell phone and service to be like in five years?

Using a wiki for a presentation

Raquel Mireles built a presentation for another class on a wiki. She included videos (stored on Youtube, a Web service) as well as links and notes.

How does using a wiki for a presentation compare to using Powerpoint? How does it compare to using a specialized online presentation service like Thinkfree Show or Zoho Show?