MOOC monetization -- a free sample strategy

The high cost of education has been one of the drivers of interest in MOOCs, and nearly all MOOCs have been free up to this point. We are in the "invest to get users" stage.

I've heard speculation on ways to monetize MOOCs, and would like suggest one I've not encountered -- the free sample model.

We are starting to get some data on MOOCs. For example, Katy Jordan pulled together completion rate data on 26 of the MOOCs that have been offered by Coursera, EDx and Udacity:



MOOC critics point to these low completion rates as evidence that MOOCs are a Bad Idea, but those large enrollment numbers are still encouraging, so let's look further. Here is a plot of the number of views of the first and last videos each week in the Bioelectricity course at Duke University:



As you see, it is long-tail, y=1/x type graph. One way to look at the long tail is to conclude that the majority of enrollees were dissatisfied and the course was a failure. But an alternative explanation is that the folks were not really enrollees, they were browsers, and, like shoppers in any situation, most decide not to buy.

We can also think of their motivation. Maybe some were not interested in "completing" the course and getting "credit" -- they wanted something less complete, less formal. We have data on that from the Duke course as well. As shown here, fun and enjoyment was the strongest motivator.



Similar motivation was reported by students in a programming class. Sixty four percent of the students said they were interested in the course out of personal interest and curiosity, 33% for professional advancement and only 3 percent for university studies.

These three studies are small and limited, but perhaps we should be thinking about MOOCs as entertainment and lifelong learning as well as for formal education and vocational training, which brings us to free sample pricing. We can view enrollment over time as follows:



The spike in the first few weeks is mostly browsers -- people who want to see what the course is like and what the workload is. They would be getting a free sample.

After a few weeks, the enrollment decline slows and we are left with the students who are interested in completing the course. As we have seen, some are seeking formal credit and others entertainment and the joy of lifelong learning. Those people would presumably be willing to pay for the course. How much?

San Jose State University is currently running a trial in which students will get credit for Udacity courses for $150. The results of that pilot study are not yet in, but $150 is less than the University price.

How about the non-credit students? Let's say a typical MOOC runs 10 weeks and the first three weeks are free. What would people pay for seven weeks of entertainment or lifelong learning? Would it be as much as a movie? A hardback book? A steak dinner?

We can noodle around with a simple spreadsheet and get some napkin-sketch revenue estimates. For example, let's assume a for-credit price of $50 for a certificate of completion and a non-credit price of $20. The median number of students completing the MOOCs shown above was 2,777. Assuming that ten percent of those are for-credit students and the MOOC can be offered four times a year, we get annual revenue of $214,000. If we use the mean completion rate of 4,447, the revenue estimate increases to $343,000.

Note that under these assumptions, the bulk of the projected revenue is from the entertainment students, not for credit students.

If an organization offers several courses, are numbers like that sufficient to cover the overhead of management and administration, production, software refinement, community facilitation, hosting and bandwidth, etc. and pay a teacher who is full time on a single course?

A question for any good teacher reading this -- how effective a teacher would you be if you only taught one course and spent full time delivering and improving it?

-----

Update, 4/17/2013

Coursera revealed that they had revenue of $220,000 in the first quarter after starting to charge for verified completion certificates (http://bit.ly/12nd8Rb).  The income was from students paying between $30 and $100 per certificate.  Coursera co-founder Daphne Koller said prices were "around" $50.

This gives us a first cut estimate of what someone hoping to get job or school credit for a course might be willing to pay.  No doubt, the going price for certified completion will be higher than the price for un-certified completion for entertainment and curiosity.

(Coursera uers who pay for certification have to submit a photo ID of themselves to the company and are also tracked based on their “unique typing pattern” to ensure that people who take tests or turn in assignments are who they say they are).
-----

Update 5/23/2013

I posited a simple characterization of people signing up for MOOCs as either "browsing" or "enrolling." Phil Hill presents a more nuanced characterization in his blog post on blog student types, classifying students as no-shows, observers, drop-ins, passive participants and active participants. Details are presented in the paper called Deconstructing Disengagement: Analyzing Learner Subpopulations in Massive Open Online Courses.
-----

Update 11/27/2013

This post in the Chronicle of Higher Education argues that MOOCs have failed in higher education, and are better suited to vocational training and lifelong learning.
-----

Update 12/1/2013

Prominent MOOC provider Udacity has shifted emphasis from college credit courses toward vocational training and lifelong learning.
-----

Update 12/9/2013

A University of Pennsylvania study of a million enrollees in 16 MOOCs shows they have relatively few active users and low completion rates.

• Course completion rates are very low, averaging 4% across all courses and ranging from 2% to 14% depending on the course and measurement of completion.
• Across the 16 courses, completion rates are somewhat higher, on average, for courses with lower workloads for students and fewer homework assignments (about 6% versus 2.5%).
• Variations in completion rates based on other course characteristics (e.g., course length, availability of live chat) were not statistically significant.
• The total number of individuals accessing a course varied considerably across courses, ranging from more than 110,000 for “Introduction to Operations Management” to about 13,000 for “Rationing and Allocating Scarce Medical Resources.”
• Across all courses, about half of those who registered viewed at least one lecture within their selected course. The share of registrants viewing at least one lecture ranged from a low of 27% for “Rationing and Allocating Scarce Medical Resources” to a high of 68% for “Fundamentals of Pharmacology.” 

Two MOOC case studies and some completion rate data

Here are links to case-study descriptions of two MOOCs. They are both informative, interesting descriptions of the projects, students and outcomes. For example, one thing that caught my eye was the fact that a significant number of participants in both classes were motivated by personal curiosity. (You will have to read them to see what catches your eye :-).

1. Functional Programming Principles in Scala: Impressions and Statistics

2. Bioelectricity: A Quantitative Approach Duke University’s First MOOC

IEEE Spectrum Podcast interview of Professor Roger Barr, who taught the Bioelectricity course (13m 42s).

3. Katy Jordan has compiled MOOC completion rate and assessment type data and displayed it in an interactive graph. (Be sure to visit the interactive graph -- you can filter it in several ways).

-----

Update 3/31
Jordan looked at 26 MOOCs, and using her data, I put together this completion rate table:

The average lifespan of canceled Google services and APIs is 4 years

A lot of people, myself included, are upset at Google's decision to eliminate their RSS reader, Google Reader. Perhaps Charles Arthur was one of them, because he has just written an article reporting the lifespan of 39 canceled Google services and application programming interfaces (APIs).  It turns out that the mean life span of canceled projects is 1,459 days, just under four years. The article has a table listing the start and end dates of the 39 services and APIs.

I use Google Reader in my work and, until now, have been having my students use it.  We use other Google services even more -- Google Docs, Blogger, Google Plus and Hangouts are central to my teaching and other work. I would be happy to pay a reasonable fee to use those services, but would think twice about requiring my students to do so.

Google had their reasons to shut Reader down -- it saves them some (very small) percent of their infrastructure cost and frees up some people for other work, but it harms their reputation. The folks who are now having second thoughts about relying Google services and APIs tend to be active -- educators, journalists, IT professionals, application developers, etc. Alienating one of these folks is a bigger loss to Google than losing a casual user.

IP TV is taking off -- which organizations will run the next global "networks?"

I recently wrote a post on Netflix's made-for the Internet series, House of Cards, saying that I did not like it as much as the HBO series The Sopranos. I've since finished watching House of Cards, and, even if I did not find it as compelling as The Sopranos, I was hooked and enjoyed watching it.

I ended that review by saying I hoped House of Cards would succeed and Netflix would give us more high production value entertainment online. It did succeed -- as you see here, it has an average rating of 4.6 stars.

In retrospect, that is not such a surprise. As David Carr points out, Netflix's use of big data pretty well guaranteed them a hit. Before starting production of the series, they knew that people liked the movies of director David Fletcher and star Kevin Spacey as well as the British version of House of Cards, upon which this series was based. Given that history, they were confident the series would be a success, so they produced 13 episodes without a pilot test.

As you see here, the episodes vary in length from 46 to 56 minutes -- the writers were freed from the constraint of broadcast television episodes, which must fit into time slots.

Having 13 episodes recorded ahead of time, meant that viewers, including me, could watch two or more together. We were freed from the weekly release schedule of broadcast television. The season constraint is also gone -- it could have been 12 episodes or 14 -- whatever the writers felt worked well.

We were also freed from commercials, which I really hate now after a year or two as a cord cutter.

They could have also dropped the episode constraint. One can think of House of Cards as a 661-minute movie. A viewer could pause whenever he or she felt like it and resume later or the writers could have inserted suggested pause points. I am not sure how well that would work out for viewers or how the writers would have handled the 11-hour format, but it would have had one positive advantage -- the viewer would not have to watch the series introduction and credits 13 times.

Netflix has committed to more Internet production, and they are not alone. YouTube is bankrolling productions and their audience exceeds 1 billion views per month.

The BBC has announced plans to produce Internet programs, and they are experienced content producers. Netflix jumped out to an early lead, but the BBC commitment reminds us that the Internet is global and we will see global productions as well as global audiences.

The times they are a'changing. The situation is well summarized in a quote by Netflix's chief content officer Ted Sarandos, who said "The goal is to become HBO faster than HBO can become us." Will Netflix, YouTube, the BBC and others become HBO-like content producers before HBO is freed of contractual obligations and moves their content to the Internet?

Historians look back at movies like Birth of a Nation, with its use of panorama shots, panning, night photography, a musical score and a large battle scene, or The Jazz Singer, with its sound track, as production technique breakthroughs. We may one day look back on House of Cards as a distribution breakthrough.

-----

Update, 3/28/2013

Amazon has committed to production of five children's programs in addition to six comedies they announced earlier.

Netflix is not standing pat -- they announced an 8-episode series to be called, Sense8, by the creators of the Matrix movies and Babylon 5. As with House of Cards, they are betting on a near-sure thing.

Update, 4/22

Even Twitter wants in on the video goldrush. Next week, Comedy Central will host a comedy festival on Twitter. That sounds like a long shot, but I love comedy, so will give it a shot. The hashtag is #ComedyFest.

Update, 4/29

Amazon has debuted pilots of their initial productions (http://indy.st/11doxAs). Netflix accurately predicted the success of House of Cards using past history and was confident in producing 13 episodes at one time. Amazon will use the public as a very large "focus group" in deciding which shows to produce and which to drop.

Netflix has released its second complete series, Hemlock Grove (http://nflx.it/14HdTIg). It has a four star rating -- Netflix mitigates their risk by mining their Big Data before producing the episodes.

Netflix and Amazon have an advantage over traditional producers in their ability to predict the likely success of new productions.

Update, 7/18

House of Cards was nominated for best drama for the 65th Primetime Emmy Awards.  The series earned nine nominations overall, including lead acting nods for Kevin Spacey and Robin Wright.

Update, 7/27/2013

Netflix profit grows but stock dropped because the number of subscribers was disappointing.  Subsequently, they got a big boost in trials when Google bundled a 3-month subscription in with their new Chromecast device.

-----

Update 10/7/2013

HBO is offering access to seven series in the Google Play store. They cost between $2 - $3 per episode and $19 - $29 for a full season. They say more will come.

Netflix said they wanted to become HBO before HBO became Netflix -- the race is on. (This looks good for Google too).

-----


Update 10/23/2013

As we see above, Netflix's chief content officer, Ted Sarandos, said "The goal is to become HBO faster than HBO can become us." It seems the race is getting close and hot. Consider these recent developments:

• You no longer have to be an HBO subscriber to see HBO programs -- you can order individual episodes or full seasons ala carte from Google. (Individual episodes cost between $1.99 and $3.99 and full seasons are between $14.99 and $38.99, depending upon the show and the video quality.) The selection is limited today, but HBO says they will add more -- this must be a delicate marketing and contractual issue with their cable and satellite carriers.
• Bloomberg projects that Netflix is poised to pass HBO in paid subscribers and is in talks about offering their content through cable providers like Cox Communications.

Consumer choice is growing and we are seeing more and more content on the Internet, but, will we settle into the usual ologopoly pricing situation?

Today, we notice some pricing differences and some similarities. HBO charges for episodes or seasons. Amazon offers current season releases for $1.99 (or $2.99 for HD). Amazon offers their Prime customers a lot of free content, but it is limited. A Prime subscription costs $79 per year, but it also includes fast shipping on things you purchase from Amazon. Netflix offers all you can stream for $7.99 a month, but its streaming service is limited -- for example they offer movies on DVD that are not available for streaming.

Amazon and HBO charge an additional dollar per episode for high definition video, and my guess is that is a lot more than the extra bandwidth cost and that difference will drop as bandwidth becomes cheaper. It will be interesting to see if they keep that differential. (That may seem like gouging, but it is nothing like the phone company pricing for text messages).

It would be cool if you could get any content from any "channel" and they were all competing on price, but with a relatively limited number of channels and production companies, I expect the market will eventually settle into a comfortable oligopoly/oligopsony.

Update 11/13/2013
-----

Netflix rolls on:

• Netflix introduces a new user interface -- it shows how their recommendation algorithm came up with suggestions for you.
• Netflix and YouTube combine for over 50% of North American traffic.
• Netflix added 1.3 million new US subscribers in Q3 and is now larger than HBO

Update 11/13/2013
-----

HBO Go shows up as a supported Chromecast app in the Google Support page.

_____
Update 10/15/2014

HBO CEO Richard Plepler announced that the pay-TV channel would launch a stand-alone, online streaming version of its service next year.

This move was predicted by Netflix's chief content officer Ted Sarandos, who, nearly two years ago, said "The goal is to become HBO faster than HBO can become us."

This will make cord cutters happy, but, Plepler made the announcement at a Time Warner Cable meeting. As long as ISPs maintain their monopoly/oligopoly market positions, they will be able to raise their Internet service prices as consumers shift away from bundles of TV channels toward Internet streaming.

-----
Update 7/11/2015

Linear TV viewing is down 15% over last year and Netflix is the number three broadcaster in the US.

Carna, a benign Internet census botnet

This map shows the locations of 460 million pingable IP addresses identified between June and October 2012 by a benign botnet called Carna:

My first reaction -- wow that is cool -- I like global visualizations of the world and the Internet -- check out the hot spots in Silicon Valley, Southern California, back east and in Asia. Look at India and China.

My second reaction -- aren't botnets evil?

Not this one. Follow the link above and you will find a full description of this ethical, benign botnet. It gathered interesting information and did no harm.

This map is only one Carna botnet result. For example it found 141 Million firewalled IP addresses and 729 million more with reverse DNS records for a total of 1.3 Billion used IP addresses.

Another example -- Carna counted the number of hosts in each top-level domain -- here are the top 20:

(Domain host counts are also reported in the Internet Systems Consortium survey, which they have run since the beginning of the domain name system. For you old folks -- ISC used to be called "Network Wizards.")

You can see all that and more in the Carna paper, and all the data is available for download.

The author does not identify him/herself for reasons of modesty and perhaps fear of upsetting people, but, whoever you are, thanks for your effort and it would be great if you would it on a regular basis.

Google fiber will expand to Olathe, Kansas

Google chose Kansas City for its first gigabit fiber network, and they just picked the second city, nearby Olathe, Kansas.

Google skipped over Kansas City suburbs like Prairie Village, Mission, Shawnee and Overland Park to bring their gigabit fiber to Olathe, a separate city with 125,000 residents. I lived in Prairie Village while in high school, and at the time, Olathe was way out in the sticks. Times have changed.

Does this portend more Google Fiber cities? Are they just trying to make a point to get the incumbent cable companies off the dime? Time will tell.

-----

Update 4/9/2013

As rumored for some time, Google has announced that they will extend Google Fiber to Austin Texas.  Austin is about the same size as Kansas City and it a hi-tech hotbed.  Google will gain insight into future applications of high speed networks and it will not cost them much.  In the wake of the Austin rumors, we also saw speculation on the cost of the Kansas City networks and of a nationwide buildout.  Eric Schmidt has been quoted as saying that gigabit Internet is a real business.  Maybe they will focus on somewhat small, hi-tech cities??

Chronicle of Higher Education survey of MOOC professors

The Chronicle of Higher Education has published the results of a non-scientific survey of professors who have taught a MOOC. The online questionnaire was sent to 184 professors in late February, and 103 of them responded.

My favorite result was the answer to the question "Has the experience of teaching a MOOC inspired you to change the way you teach the traditional classroom version of the course" -- 73.7 percent of the respondents answered "yes."

Even if MOOCs are a flash in the pan, they trigger innovation in pedagogy, technology and certification, just as online teaching material will take us beyond the traditional textbook.

I was also struck by the completion rate. The median number enrolled was 33,000 and the median number who completed the course with a passing grade was 2,600. This was achieved by a median of 1 teaching assistant and 8 hours per week of the professor's time.

In a recent blog post, I asked whether a MOOC with 700 active students was a bad deal. If we can teach 2,600 students, we can justify a full time professor and production costs -- what could you accomplish if you worked full time on teaching one class and had good support?

Check the Chronicle article to see the rest of the results, and be sure to note that there are three tabs at the top of the page -- an article, the survey results and quotes from eight of the professors.

The California legislature may endorse credit for MOOCs.

Governor Brown is following in his father's footsteps.

Governor Pat Brown is remembered for developing the California master plan for higher education in 1960. His son, Jerry Brown, the current Governor of California, may be remembered (for better or worse) as an Internet-era reformer of higher education.

Last fall, Governor Brown signed a bill designed to generate open text books, in order to alleviate the rapid rise of textbook prices compared with consumer prices, shown below. (Click images to enlarge).

This spring, San Jose State University is running a trial offering of credit for massive online courses from Udacity, a private company. The hope is that this will reverse the rising cost of tuition and fees.

Now the California legislature is considering a bill requiring the state’s public colleges and universities to give credit for faculty-approved online courses taken by students unable to register for oversubscribed classes on campus.

For a glimpse of the thinking behind these measures, this is what the Governor had to say higher education in his January State of the State speech:

With respect to higher education, cost pressures are relentless and many students cannot get the classes they need. A half million fewer students this year enrolled in the community colleges than in 2008. Graduation in four years is the exception and transition from one segment to the other is difficult. The University of California, the Cal State system and the community colleges are all working on this. The key here is thoughtful change, working with the faculty and the college presidents. But tuition increases are not the answer. I will not let the students become the default financiers of our colleges and universities.

And, it's not just California. In the supplemental notes to President Obama's State of the Union speech, under the heading "Holding colleges accountable for cost, value and quality," we read a call for revision of the federal student aid system, allowing for new measures of value and new system of accreditation:

The president will call on Congress to consider value, affordability, and student outcomes in making determinations about which colleges and universities receive access to federal student aid, either by incorporating measures of value and affordability into the existing accreditation system; or by establishing a new, alternative system of accreditation that would provide pathways for higher-education models and colleges to receive federal student aid based on performance and results.

If you want to dig deeper, you can see the text of the ambitious bill introduced February 21 and some March 8 amendments.

Tuition and fees

Textbook prices

-----

Update 4/6/2013

The California State Senate unanimously passed Senate Bill 520, authorizing the granting of credit for courses taken online. The bill now goes to the Assembly.

-----

Update 8/2/2013

California puts SB 520 on ice. State Senator Darrell Steinberg, who introduced the bill, has put it on hold in the face of faculty opposition and to take time to evaluate new online programs proposed by the University of California, California State University and California Community College system. For example, the California State University will offer 35 online courses statewide.

Google Glass -- why just capture reality when you can mediate and improve it?

There is a lot of hype surrounding wearable computing these days and Google Glass gets a lot of that attention with demonstrations of video and image capture and augmented reality, as shown in this video, which has been viewed over 16 million times.

But University of Toronto professor Steve Mann has been wearing electronic glasses and working with mediated reality for over 30 years. Mann differentiates between "augmented reality," where text or graphics are displayed over your normal vision and more general "mediated reality," in which vision is improved -- for example, detecting infrared radiation to see temperature differences, zooming in on distant objects, or extending the dynamic range of a photographic image by combining various exposures. He discussed these applications and more in an recent Techwise Conversations podcast interview.

Mann has pioneered mediated reality, but he is not alone in the field. For example, professor Gabby Sarusi and his colleagues at the at Ben Gurion University of the Negev are working on a nano layer coating for detecting infrared radiation in night vision goggles. Work on high dynamic range photography has also been with us for some time.

Journalists are beginning to report on initial experience with Google Glass prototypes and Google is fishing around for a killer app, but I suspect that Google Glass will turn out to be a niche product. We will not be wearing them 24-7, but some of us may put them on for special tasks -- like a surgeon entering the operating room.

That being said, research prototypes and high-value niche applications have a way of turning into ubiquitous, game-changing products, but it takes a while -- as you see in this photograph of Ivan Sutherland, inventor Sketchpad, the first object oriented computer graphics program, which he developed more than twenty years before MacDraw came out. (Ditto, any early prototype).

What about fifty years from now? Will Google Glass 2063 interface directly with the retina?

The access and bandwidth divide in online education

What if online classes and MOOCs really do take over the world? Will that exacerbate the "digital divide?" Will poor and rural US students have to seek out public libraries and other public access spots? An article in the Chronicle of Higher Education focuses on that question.

What about students in developing nations? Good luck taking a Coursera course in, say, Myanmar or Cuba. In much of the world, Internet access is too expensive for the average person and, even if they can get online, too slow to effectively use a modern Web site.

As online courses increase in sophistication, bandwidth will become an even greater impediment to participation.

The Guinean students shown here are studying under street lights (click to enlarge). Will they have access to global online education?
-----

Update 3/6/2013

A followup article talks about the impact of data caps for online students in the US. Again, the situation is much worse in developing nations.
-----

Update 12/8/2013

A University of Pennsylvania study of a million users who signed up for one of 17 courses they offered through Coursera showed that few were from developing nations.

Survey of employer hiring criteria for bachelor's degree graduates -- internships pay off

The Chronicle of Higher Education and NPR Marketplace have reported results of a survey they commissioned of employers of college graduates.

They found that where a student goes to school is not as important as you might think. Graduates of "flagship" state universities are only seen as a little more "preferable" (on a 5-level Likert scale) than private or regional public campuses.

They also reported that, with the exception of education majors, the perceived value of a bachelor's degree was somewhat lower than it was five years ago.

Online degrees were rated "undesirable" (2.84 on a 5-level Likert scale), but I suspect that is based online private schools like the University of Phoenix.

Universities are moving increasing numbers of their classes online and that trend is stronger in public than private schools Will employer's perception of their graduates decline? What of fully online public programs like Calstate Online?

For me, the most interesting result was that internships were rated as more important than where one went to college, their major or GPA. Completing an internship was reported to be the most important credential for recent college graduates.

A related survey of 2012 college graduates reinforces the value of internships. They found that 55% of the students had internship and/or co-op experience and 51% of interns were offered jobs. Payed interns fared better than volunteers. They did less busy work and 63% got at least one job offer, compared to 41% of unpaid interns.

The Chronicle/Marketplace survey was conducted by Maguire Associates, but I could not find a report of it on their Web site. I would like to see the actual survey questions and analysis, but this article is worth reading. You should also listen to or read the transcript of the Marketplace podcast (2m 46s) on the study.

-----

Update 3/8/2013

College students or even high school students planning their college career should crawl around the NACE Web site. There is a lot of useful information there, including a salary calculator for many different jobs in many different locations, based on your background.

Actually, the NACE calculator is hosted on the Web site of their partner Jobsearchintelligece.com. JSI maintains a database with salary data for up to 1,000 occupations in 560 regions of the U. S. - with a minimum of 25 respondents for each occupation in each region to ensure statistical significance. You can easily query the database to find, for example, starting salaries for specific full time jobs given specific majors along with salaries for those same jobs ten years after graduation.

I decided to goof around with an example, and searched for data on modeling jobs. I said I was a recent grad with a GPA between 3.5 and 3.9 who had gone to CSUDH and majored in language and literature.

I learned that there are 60 persons employed as models in the Los Angeles area, and starting, median and top salaries are $27,120, $43,810 and $72,290. Based upon the background I provided, they estimated my starting salary at $38,900, but after looking at their overall employment picture for models, shown here, I decided to look for a different career.

Remember that college is about a lot more than hoped-for jobs and salary. The best thing that can happen at college is not finding a major that leads to job offers with good starting salaries, it is finding something you are passionate about. That is way more important -- put your energy there.

Columbia University study slams traditional online classes -- we need to move beyond traditional

Researchers at Columbia University’s Community College Research Center have released a working paper "Adaptability to Online Learning: Differences Across Types of Students and Academic Subject Areas", based on a study of nearly 500,000 courses taken by over 40,000 community and technical college students in Washington State. The study was based on traditional online courses, which typically have about 25 students and are run by professors who often have little interaction with students. (I will return to this point below).

Here's the bottom line -- the study concluded that “the online format had a significantly negative relationship with both course persistence and course grade, indicating that the typical student had difficulty adapting to online courses.”

This is consistent with the anecdotal reports of my colleagues who teach online -- they have much better experience with masters level classes than undergraduate.

Let's drill down a bit.

The study found that the negative results for online classes held across the board, but the effect was stronger for some subgroups – “males, Black students, and students with lower levels of academic preparation experienced significantly stronger negative coefficients for online learning compared with their counterparts, in terms of both course persistence and course grade.”

They also note that "performance gaps between key demographic groups already observed in face-to-face classrooms (e.g., gaps between male and female students, and gaps between White and ethnic minority students) are exacerbated in online courses."

They found that older students adapted more readily to online courses than did younger students. (Again, consistent with my colleagues' observations).

The study also found that "the relative effects of online learning varied across academic subject areas ... two academic subject areas appeared intrinsically more difficult for students in the online context: the social sciences (which include anthropology, philosophy, and psychology) and the applied professions (which include business, law, and nursing)."

The authors of the study went on to suggest policies to cope with the problems they identified:

• Screening: Only allow certain students to take online classes, for example those with a 3.0 or better GPA or those who successfully complete a workshop on online learning skills.
• Scaffolding: Incorporate the teaching of online learning skills into online courses in which less-adaptable students tend to cluster.
• Early warning: Identify and intervene with students who are having difficulty adapting.
• Wholesale improvement: Improving the quality of all online courses taught.

The first three feel like fingers in the dike to me, but how about wholesale improvement? At the start of this post, I noted that these were traditional online courses. They are often quick makeovers of classroom courses -- using standard textbooks and ancillary materials, but substituting threaded discussion for classroom sessions. Old wine in new bottles.

This is understandable when there are, say, 25 students in a class. There is no time for "wholesale improvement."
However, if we scale the online class up, say enrolling 500 active, for credit students, we can afford to create "new wine."

We see experimentation along these lines in MOOCs -- highly modular, interactive presentations created specifically for online delivery, peer interaction, instrumentation of material, crowd-sourced feedback from students, high production values, frequent revision of teaching material, etc.

None of this is affordable if we are teaching sections of 25 students, but for 500 students, we can afford full time staff producing and maintaining a single course. Hey, with 20,000 students we can afford Disney Studios.

Is a MOOC with "only" 700 active students a bad deal?

The Chronicle of Higher Education reported this morning that a Coursera professor, Richard McKenzie, had decided to withdraw from his course Microeconomics for Managers. (The course will run to completion with a new instructor using McKenzie's videos).

Professor McKenzie was disturbed by the level of student engagement and commitment . In a note he posted on January 30, McKenzie noted that just under 37,000 had enrolled, but “fewer than 2 percent have been actively engaged in discussions,” and he worried that unprepared students were wasting the other's time "The problem is especially acute when students who have not watched the videos and have not done the readings contribute comments that we all have to read at some level."

In the forum thread discussing his withdrawal, there is little if any criticism and a lot of appreciation and explanation for low participation rates. McKenzie stated that “I will not give on standards, and you also should not want me to, or else the value of any ‘certification’ won’t be worth the digits it is written with."

I did not take the course, but watched some of the video. The format was uninterrupted lecture for around 30 minutes followed by multiple choice questions. The lectures I watched were engaging and presented well. He used easily followed examples in which rational economic thinking often leads to non-intuitive, perhaps politically incorrect, conclusions.

So, what went wrong?

Were there economic problems? The Chronicle reported that some students complained about the amount of work McKenzie assigned and others balked at the price of the textbook "The New World of Economics: A Remake of a Classic for New Generations of Economics Students," which Amazon sells for $79.95 in paperback or $63.96 for the Kindle.

I don't know what Professor McKenzie's business relationship was with Coursera, but the lectures were produced and copyrighted in 2011 by RBMCourses.com (which is not a registered domain name on the Internet).

The teaching material was re-purposed and so were professor McKenzie's expectations for the class. He was not prepared for the variance in online student commitment compared to his classroom at UC Irvine. Enrollment numbers like 37,000 have to be understood as browsing, not commitment. He has learned that lesson, writing:

As it has turned out, the enrollment count is meaningless, with fewer than 40 percent of the students actually logging in during each of the first two weeks. Only a fourth of the enrolled students have done as much as watch a single video lecture over the last week.

But, how bad is that? Even if, say, only 700 students actively participated, the course seems to have made economic (and social) sense. The course will be picked up by Melissa Loble, Associate Dean for Distance Learning at UC Irvine, and run to completion using Professor McKenzie's videos, quizzes, midterm and final exam. The videos and textbook were sunk costs, so the MOOC was a way to generate some extra revenue and offer a course at the same time at relatively little marginal cost.

Based on my sampling of professor McKenzie's lectures, I would guess that his MOOC was as effective as the typical (perhaps not the best) microeconomics course for business students. If that is the case, a low marginal cost class with "only" 700 students sounds viable to me. It even leaves some slack for continued investment in the teaching material or additional presentation resources.

Measuring and predicting cyber security

Cybersecurity is a hot topic these days as the number of attacks grows and governments try to defend against them. There are questions of policy as well as technology -- which policies work?

Aaron Kleiner, Paul Nicholas and Kevin Sullivan of Microsoft Research have tried to answer that question by looking at the results of scans using MSRT, Microsoft's malicious software removal tool. The MSRT reports back when it removes malicious software and, using this data, Microsoft estimates computers cleaned per mille (thousand) or “CCM,” the number of computers cleaned for every 1,000 times that the MSRT is run. For example, if 50,000 scans resulted in 200 cleans, the CCM would be 200/50 = 4.

The researchers gathered data during the fourth quarter of 2011 and published the results in a report entitled Linking Cybersecurity Policy and Performance. Here you see a visualization of CCM levels for countries in the fourth quarter of 2011.

Analyzing the data, they looked for correlation between policies and CCM. For example, countries signing the Council of Europe Convention on Cybercrime do better than countries that do not. In addition to policies, they looked at various indicators and found correlations as shown below.

Using a model based on demographic and policy variables, the were able to predict CCM well, as shown here.

The authors emphasize that correlation does not imply causation, but this is an interesting big data application. Microsoft provides a valuable service for free, and in return gathers massive amounts of data on incidents of malicious software detection and removal. It is a win-win situation.

The State of the Union Address -- Internet related points

I was in class during the State of the Union Address last night, so read the transcript online today. I zeroed in on the Internet-related points.

The President mentioned the threat of hackers and called for legislation that would "give our government a greater capacity to secure our networks and deter attacks." Hey, who can argue against that? But, the curmudgeon in me notes that he spoke only of defending against attacks, which leaves me wondering what are we doing with respect to offensive attacks. I also have a slight worry that some of the attacks we spend taxpayer money deterring might be attacks on the intellectual property of movie studios, record companies, book publishers, etc.

He also announced that he had issued an executive order on Improving Critical Infrastructure Cybersecurity, but I have not yet read it.

The President also raised this rhetorical question "Ask any CEO where they'd rather locate and hire, a country with deteriorating roads and bridges or one with high-speed rail and Internet, high-tech schools, self- healing power grids." High speed Internet and high tech education are clearly on his mind. (For more concrete evidence of this, see FCC Chairman Genachowski's recent Gigabit City Challenge.

A comment which indirectly impacts the Internet was a call for stepped up energy and medical research -- "Now is the time to reach a level of research and development not seen since the height of the space race."

That is what struck me in the speech. Now for a couple of meta comments.

1. I chose to read the transcript rather than watch the video. It was faster and I could copy the above quotes as I went along. I was also able to quickly link to the executive order on security and the Gigabit City Challenge, but, I missed the emotional "content" of both the President and the audience. Perhaps that diminished my impression of the importance of the speech.

2. As soon as I finished reading the speech, I skipped over to FactCheck.org and found that they did their thing and concluded that "The president spins his accomplishments on jobs, health care and deficit reduction in annual address." How cool was that?

3D Printing technology marches on -- for better or worse

While Congress and the President and the NRA haggle over gun control laws, Defcad.org is distributing 3-D printing specs for guns and parts of guns.

Check the following video for a demonstration of a printed 30-round magazine for a semi-automatic weapon.



------

Update, April 1

It may be an April Fool joke, but the Defcad,org web site has a notice saying the site and domain name have been seized by the Federal Government and Grand Jury indictments were issued.

Their latest video, which is still up on YouTube, announces Defcad Search:



The video shows off some guns with 3D-printed parts, denounces the intellectual property and promises Defcad Search, a search engine with CAD files from which there will be no takedowns, ever.

----

Update 5/7/2013

Defcad.org has now released a complete, single shot pistol. You can read more about it in this Forbes article. Is this any more a threat than home made "zip" guns made from common hardware parts?

Update 5/10/2013

The State Department requested that Defcad stop distributing their pistol plans, which they did. However, they report that there had been over 100,000 downloads and a quick Google search turns up sites which are now offering copies. Once something is out, you cannot put it back.

Update 5/12/2013

Leland Yee, a California State Senator who is alarmed by 3D printing of guns, has called for regulations of 3D printers.

Update 5/24/2013

Moving away from the dark side -- Michigan Tech University is running a 3D printers for peace contest. It's your chance to win a 3D printer -- what would Mother Theresa have done with a smart printer?

Update 7/26/2013

The 3-D printed pistol is not yet ready for prime time.  Matt Ratto, a professor at University of Toronto printed a handgun -- it took  27 hours, a fair amount of computer know-how, a $50,000 printer, and $300 worth of plastic.  What will it cost in ten years?

Update 1/19/2014

Here is an article (with links) on a low-cost, open-source 3D metal printer from Michigan Technological University. It's a work in progress, but it is less expensive than off-the-shelf commercial plastic 3D printers.

Why I signed a petition to name Susan Crawford next head of the FCC

I just electronically signed the Whitehouse.gov petition to appoint Susan Crawford as the next head of the Federal Communications Commission. Let me tell you why.

The Internet was invented and deployed in the US. At one time, nearly all international traffic flowed through our National Science Foundation network. Today, our Internet is mediocre by the standards of developing nations and slipping.

In a recent interview by Bill Moyers, Crawford says U.S. Internet access is slow, costly and unfair and tells how we got in this fix. (The video is embedded below).

Here are a few quotes from the interview:

"What's happened is that these enormous telecommunications companies, Comcast and Time Warner on the wired side, Verizon and AT&T on the wireless side, have divided up markets, put themselves in the position where they're subject to no competition and no oversight from any regulatory authority. And they're charging us a lot for internet access and giving us second class access."

"So there's been a division. Cable takes wired, Verizon/AT&T take wireless. They're actually cooperating."

"In almost 20 states in America it's either illegal or very difficult for municipalities to make this decision for themselves."

"This is a moment when we have to separate out content from conduit. It should not be possible for a local cable actor or any distributor to withhold programming based on volume ...Everybody should get access to the same stuff at the same price and they should be announced prices."

"Michael Powell, who served as F.C.C. chairman for four years in the mid-2000s, is now the cable and telecom industry's top D.C. lobbyist."

"Meredith Attwell Baker who was one of the F.C.C. commissioners who approved Comcast's merger with NBCUniversal, left the agency four months later to join Comcast as a highly paid lobbyist."

Further reading:

Dan Gillmor, director of the Knight center for digital media entrepreneurship at Arizona State University, explains why Susan Crawford should be the next head of the Federal Communications Commission (http://bit.ly/XjaNEm).

Cory Doctorow, Internet activist, journalist and science fiction, agrees that Crawford should run the FCC ( http://bit.ly/WYv8fm).

I searched for a rebuttal -- someone arguing against Crawford's appointment -- but was unable to find one. Please let me know if you know of one.
-----

Update 1/28/2014

How cable companies "compete" -- dividing up the cable market

Comcast is near a deal to buy New York City, North Carolina and New England cable assets from Charter Communications Inc., but that sale is contingent upon shareholders approving Charter’s takeover bid for Time Warner Cable. Since Comcast would no longer be in the running for Time Warner Cable, they will probably accept Charter's offer.

This trading of monopoly territory reminds me of the way rival drug gangs divide up corners and housing projects -- I wonder if these guys watch "The Wire."

NMC-Educause report on technologies that will impact higher education

The New Media Consortium and Educause have teamed up on a report that looks into the crystal ball to predict technologies that will impact higher education in the next year, the next two or three years and the next four or five years.

The technologies they are bullish on are listed below. Do you agree? Can you think of educational applications for each of them? Download the report and see the basis of their predicitions.

Time-to-Adoption Horizon: One Year or Less

• Flipped Classroom
• Massively Open Online Courses
• Mobile Apps
• Tablet Computing

Time-to-Adoption Horizon: Two to Three Years

• Augmented Reality
• Game-Based Learning
• The Internet of Things
• Learning Analytics

Time-to-Adoption Horizon: Four to Five Years

• 3D Printing
• Flexible Displays
• Next Generation Batteries
• Wearable Technology

Netflix's House of Cards disappointments (me)

Last night I streamed two episodes of Netflix’s miniseries "House of Cards."

House of Cards was produced by Netflix for distribution on the Internet, not via the traditional cable TV services. I am not a big TV watcher and am certainly not a drama critic, but I hoped it would be a terrific program, helping Netflix disrupt the status quo -- like comedian Louis CK's production of his own comedy show or Bill O'Reilly and John Stewart's comic debate.

I was disappointed. The protaganist, is Frank Underwood, a Machiavellian leader of the House of Representatives, who is out for revenge against the President who passed him over for Secretary of State.

I didn't get hooked because Frank Underwood is no Tony Soprano. Frank is one dimensional -- all selfish and plotting -- there is nothing to like about him. Tony is an evil murderer, but he is also loyal to his gang, loves and worries about his family and is a likable teddy bear of a guy.

The entire Netflix cast is one-dimensional -- the Lucretia Borgia wife, the hip reporter who blogs and knows that print is dead, etc.

There were a couple of innovations. I liked the way they showed people sending text messages on screen, and they released all of the episodes at the same time so people can watch them marathon-style if they do get hooked.

I hope I'm in the minority and most people love House of Cards, because I want Netflix to succeed and give us more high production value entertainment online.

Cyberwar is a trending topic -- New York Times, Wall Street Journal and Washington post hacked

Last week, the New York Times, Wall Street Journal and Washington Post each reported attacks by Chinese hackers trying to discover documents and the names of sources for coverage of political corruption in China. They also reported that Chinese hackers had targeted Bloomberg, Reuters and other media sites.

The Times said the intruders used phishing attacks to break in around September 13 and infiltrated computers of at least 53 users, including those of the Shanghai and Beijing bureau chiefs. It seems they were looking for email and documents that might identify the sources for The Time's coverage of business deals that left relatives of Chinese prime minister Wen Jiabao wealthy.

According to Mandiant, the security firm the Times hired for help on this case, at least thirty western news organizations have been targeted by China. Many private companies also say they have been attacked by China, but they will not give forensic proof because that would be bad public releations and also reveal security secrets.

The Times reported that the hackers used university computers as proxies and switched IP addresses in order to remain anonymous. Oliver Richwood has warned against using China as a scapegoat and assuming they are responsible for all cyber attacks, but, in this case, the Times seems certain that the attacks are Chinese, saying they "closely matched the pattern of earlier attacks traced to China." The reported choice of targets supports that claim.

The outline of Cliff Stoll's TED Talk

Eric Schmidt of Google has said ”It’s fair to say we’re already living in an age of state-led cyber war, even if most of us aren’t aware of it.” Schmidt is correct, but "cyber war" is too broad a term. We may distinguish between at least three forms of cyber attack (1) espionage -- government, personal and business spying (2) sabotage that destroys or modifies data and (3) sabotage that effects the physical world. We can also characterize cyber attacks by their intent -- monetary gain, industrial advantage, political repression, political advantage, etc.

In his article "Fear Pays the Bills, but Accounts Must Be Settled," security expert Bruce Schneier acknowledges that cyber attacks are commonplace, but points out that hype often outstrips reality and leads to major funding decisions. Many stories focus on hypothetical sabotage of the electric grid and other infrastructure, but, as far as I know, the only type 3 attack to date was the 2008 US-Israel Stuxnet worm, which damaged an Iranian nuclear enrichment plant. (The New York Times also reported that story).

Hype or reality, cyberwar funding is on the increase, and there is a lot of money to made. For example, the recently increased funding for the Army cyber command.

Topographer Stoll's Klein wine bottle

All this talk about government-sponsored hacking reminded me of Clifford Stoll, an astronomer and system administrator who tracked down a KGB hacker at Lawrence Berkeley National Lab in 1986. Stoll documented his detective work in an insightful and funny book "The Cuckoo's Egg: Tracking a Spy Through the Maze of Computer Espionage." Check it out and read the reviews at Amazon. I read it at the time and remember really liking both the book and Stoll himself. If you wonder why I said I liked him as well as his book, watch his 2006 TED Talk.

The New York Times has also recorded an 8-minute news video on the Chinese break ins featuring interviews of the Times CTO and Nicole Perlroth, who wrote the story.

-----

Update 2/19 -- The New York Times reports that Mandiant will soon publish a report stating that a specific unit of the Chinese Army, headquartered in a specific building in Beijing has been responsible for "an overwhelming percentage of the attacks on American corporations, organizations and government agencies."

-----

Update 2/20 -- Mandriant has published a report detailing their evidence against the Chinese and the Chinese have denied that the charges. They have also produced a short video showing screen captures of the hackers at work. (Yes, the video could have easily been contrived and bogus).

The video shows alleged Chinese hackers setting up a gmail account using a US IP address, spearphishing, taking control ov victim computers, accessing a victim's Microsoft Exchange email, copying hacking tools from their repository in Shanghai and stealing files.

Security gurus are now going through the Mondriant report to see which malware was being used and making sure that it is known to the security and antivirus community.

It is too bad that Cliff Stoll did not have access to YouTube when he was working on the Cuckoo's Egg. His video would have been a lot funnier!

-----

Update 2/24

Bloomberg has an article "How to curb Chinese cyberattacks." The title promises more than it delivers, but it differentiates between attacks against critical infrastructure and espionage and talks about what the administration has and should do.

-----

Update 2/25

The New York Times continues the discussion of Chinese cyber-attacks -- the "cyber-cold war" -- between the US and China. The article surveys the debate over the appropriate response to Chinese attacks -- from diplomacy to better defense to counter attack. There is no clear answer because of the deep inter-dependencies of the Chinese and US Economies.

-----

Update 3/1

The Chinese have countered US charges, saying hackers from the US have repeatedly launched attacks on two Chinese military websites, including that of the Defense Ministry.  They claim 144.000 attacks per month.  Do you believe the US is also conducting offensive attacks?  Do you favor our doing so?

-----

Update 3/8

I've always assumed that Skype calls were securely encrypted, but that turns out not to be the case. With Microsoft's assistance, the Chinese government is monitoring Skype calls. University of New Mexico graduate student Jeffrey Knockel, who discovered the monitoring and is tracking the tracker, maintains a list of keywords the Chinese use for both censorship and surveillance. There are over 1,100 keywords on the list and he updates it daily on his Web site. Be careful about saying things like "Reporters without Borders", "Amnesty International", or the Macdonalds in front of Chunxi Road in Chengdu" if you are using Skype to talk with someone in China.

-----

Update 3/9/2013

Chinese Foreign Minister Yang Jiechi says recent hacking allegations are on "shaky ground," and China opposes "turning cyberspace into a new battlefield or using the Internet as a new tool to interfere in the internal affairs of other nations.” He said that China’s government opposes hacking and has drawn up rules and laws to strictly forbid hackers and they have advocated and submitted specific proposals for a set of international rules governing the Internet under a United Nations framework.

Do they really want a truce? What I wonder is why we are not seeing stories of Internet espionage in China. I suspect it is going on.

-----

Update 3/15/2013

A high level official has now explicitly called upon China to curtail hacking and enter into talks on the topic. While this seems obvious to a layman, in the protocols of international diplomacy, this is apparently an important step.

The speech in which hacking is addressed was by Tom Donilon, National Security Advisory to the President.

The New York Times article stresses hacking, but it is worth noting that hacking was a small part of the speech addressing "The United States and the Asia-Pacific in 2013." The main point of the speech was that the US has shifted priorities, increasing emphasis on the Asia-Pacific region.

In fact the section on hacking was only 319 out of 5,317 words and 3 of 78 paragraphs -- 4-6%. That is perhaps indicative of the over-hyping of the importance of these "cyberwar" attacks.

-----

Update 3/15/2013

The US is talking stick as well as carrot.
http://nyti.ms/WtjCLX

General Keith Alexander, who runs both the National Security Agency and the new Cyber Command, told the House Armed Services Committee “This is an offensive team that the Defense Department would use to defend the nation if it were attacked in cyberspace. Thirteen of the teams that we’re creating are for that mission alone.”

The same day the nation’s top intelligence official, James R. Clapper Jr., warned Congress that a major cyberattack on the United States could cripple the country’s infrastructure and economy, and suggested that such attacks now pose the most dangerous immediate threat to the United States, even more pressing than an attack by global terrorist networks.

This is starting to sound like the "weapons of mass destruction" buildup to the invasion of Iraq.

-----

Update 5/7/2013
http://nyti.ms/16e8CYU

The New York Times has reported that the US has accused the Chinese military of "mounting attacks on American government computer systems and defense contractors, saying one motive could be to map “military capabilities that could be exploited during a crisis.”

-----

Update 5/20/2013

The New York Times reports that, after a three-month break, Chinese hackers have resumed their attacks on U.S. targets. The Chinese Foreign Ministry denies the accusation, and People’s Daily, which reflects the views of the Communist Party, called the United States “the real ‘hacking empire,’ ” saying it “has continued to strengthen its network tools for political subversion against other countries.” Other Chinese organizations and scholars cited American and Israeli cyberattacks on Iran’s nuclear facilities as evidence of American hypocrisy.

-----

Update 5/22/2013

This story keeps on going. The Washington Post reported that Chinese hackers breached Google security in an effort to learn the identities of Chinese intelligence operatives in the United States who may have been under surveillance by American law enforcement agencies. It seems they tried tried to do the same at Microsoft, but failed to gain access.

----

Update 5/23/2013

The New York Times has published an article on the culture of hacking in China -- it is widespread and brazen. Companies advertise their capability and sell at trade shows. The article covered a trade show and quotes a salesman as saying “We can physically locate anyone who spreads a rumor on the Internet.” The company’s services include monitoring online postings and pinpointing who has been saying what about whom.

Another quote from the article:

The culture of hacking in China is not confined to top-secret military compounds where hackers carry out orders to pilfer data from foreign governments and corporations. Hacking thrives across official, corporate and criminal worlds. Whether it is used to break into private networks, track online dissent back to its source or steal trade secrets, hacking is openly discussed and even promoted at trade shows, inside university classrooms and on Internet forums.

-----

Update 5/29/2013

The Washington Post reports that critical U.S. weapons system designs were compromised by Chinese hackers. Before things like reports of weapons in Iraq, I would have been more concerned about this report, but, now, I cannot help wondering whether it is hype to get congress to allocate funds for agencies and companies.

-----

Update 6/1/2013

A New York Times column argues that the example of the British use of Elizabethan pirates against the Spanish navy is more relevant to the budding cyberwar between the US and China than the cold war between the US and the Soviet Union.

_____

Update 6/1/2013

A New York Times article reports that The United States and China have agreed to hold regular, high-level talks on how to set standards of behavior for cybersecurity and commercial espionage, the first diplomatic effort to defuse the tensions over what the United States says is a daily barrage of computer break-ins and theft of corporate and government secrets.  That sounds hopeful.