Friday, February 22, 2013

Columbia University study slams traditional online classes -- we need to move beyond traditional

Researchers at Columbia University’s Community College Research Center have released a working paper "Adaptability to Online Learning: Differences Across Types of Students and Academic Subject Areas", based on a study of nearly 500,000 courses taken by over 40,000 community and technical college students in Washington State. The study was based on traditional online courses, which typically have about 25 students and are run by professors who often have little interaction with students. (I will return to this point below).

Here's the bottom line -- the study concluded that “the online format had a significantly negative relationship with both course persistence and course grade, indicating that the typical student had difficulty adapting to online courses.”

This is consistent with the anecdotal reports of my colleagues who teach online -- they have much better experience with masters level classes than undergraduate.

Let's drill down a bit.

The study found that the negative results for online classes held across the board, but the effect was stronger for some subgroups – “males, Black students, and students with lower levels of academic preparation experienced significantly stronger negative coefficients for online learning compared with their counterparts, in terms of both course persistence and course grade.”

They also note that "performance gaps between key demographic groups already observed in face-to-face classrooms (e.g., gaps between male and female students, and gaps between White and ethnic minority students) are exacerbated in online courses."

They found that older students adapted more readily to online courses than did younger students. (Again, consistent with my colleagues' observations).

The study also found that "the relative effects of online learning varied across academic subject areas ... two academic subject areas appeared intrinsically more difficult for students in the online context: the social sciences (which include anthropology, philosophy, and psychology) and the applied professions (which include business, law, and nursing)."

The authors of the study went on to suggest policies to cope with the problems they identified:

  • Screening: Only allow certain students to take online classes, for example those with a 3.0 or better GPA or those who successfully complete a workshop on online learning skills.
  • Scaffolding: Incorporate the teaching of online learning skills into online courses in which less-adaptable students tend to cluster.
  • Early warning: Identify and intervene with students who are having difficulty adapting.
  • Wholesale improvement: Improving the quality of all online courses taught.
The first three feel like fingers in the dike to me, but how about wholesale improvement? At the start of this post, I noted that these were traditional online courses. They are often quick makeovers of classroom courses -- using standard textbooks and ancillary materials, but substituting threaded discussion for classroom sessions. Old wine in new bottles.

This is understandable when there are, say, 25 students in a class. There is no time for "wholesale improvement."
However, if we scale the online class up, say enrolling 500 active, for credit students, we can afford to create "new wine."

We see experimentation along these lines in MOOCs -- highly modular, interactive presentations created specifically for online delivery, peer interaction, instrumentation of material, crowd-sourced feedback from students, high production values, frequent revision of teaching material, etc.

None of this is affordable if we are teaching sections of 25 students, but for 500 students, we can afford full time staff producing and maintaining a single course. Hey, with 20,000 students we can afford Disney Studios.

Monday, February 18, 2013

Is a MOOC with "only" 700 active students a bad deal?

The Chronicle of Higher Education reported this morning that a Coursera professor, Richard McKenzie, had decided to withdraw from his course Microeconomics for Managers. (The course will run to completion with a new instructor using McKenzie's videos).

Professor McKenzie was disturbed by the level of student engagement and commitment . In a note he posted on January 30, McKenzie noted that just under 37,000 had enrolled, but “fewer than 2 percent have been actively engaged in discussions,” and he worried that unprepared students were wasting the other's time "The problem is especially acute when students who have not watched the videos and have not done the readings contribute comments that we all have to read at some level."

In the forum thread discussing his withdrawal, there is little if any criticism and a lot of appreciation and explanation for low participation rates. McKenzie stated that “I will not give on standards, and you also should not want me to, or else the value of any ‘certification’ won’t be worth the digits it is written with."

I did not take the course, but watched some of the video. The format was uninterrupted lecture for around 30 minutes followed by multiple choice questions. The lectures I watched were engaging and presented well. He used easily followed examples in which rational economic thinking often leads to non-intuitive, perhaps politically incorrect, conclusions.

So, what went wrong?

Were there economic problems? The Chronicle reported that some students complained about the amount of work McKenzie assigned and others balked at the price of the textbook "The New World of Economics: A Remake of a Classic for New Generations of Economics Students," which Amazon sells for $79.95 in paperback or $63.96 for the Kindle.

I don't know what Professor McKenzie's business relationship was with Coursera, but the lectures were produced and copyrighted in 2011 by (which is not a registered domain name on the Internet).

The teaching material was re-purposed and so were professor McKenzie's expectations for the class. He was not prepared for the variance in online student commitment compared to his classroom at UC Irvine. Enrollment numbers like 37,000 have to be understood as browsing, not commitment. He has learned that lesson, writing:

As it has turned out, the enrollment count is meaningless, with fewer than 40 percent of the students actually logging in during each of the first two weeks. Only a fourth of the enrolled students have done as much as watch a single video lecture over the last week.
But, how bad is that? Even if, say, only 700 students actively participated, the course seems to have made economic (and social) sense. The course will be picked up by Melissa Loble, Associate Dean for Distance Learning at UC Irvine, and run to completion using Professor McKenzie's videos, quizzes, midterm and final exam. The videos and textbook were sunk costs, so the MOOC was a way to generate some extra revenue and offer a course at the same time at relatively little marginal cost.

Based on my sampling of professor McKenzie's lectures, I would guess that his MOOC was as effective as the typical (perhaps not the best) microeconomics course for business students. If that is the case, a low marginal cost class with "only" 700 students sounds viable to me. It even leaves some slack for continued investment in the teaching material or additional presentation resources.

Saturday, February 16, 2013

Measuring and predicting cyber security

Cybersecurity is a hot topic these days as the number of attacks grows and governments try to defend against them. There are questions of policy as well as technology -- which policies work?

Aaron Kleiner, Paul Nicholas and Kevin Sullivan of Microsoft Research have tried to answer that question by looking at the results of scans using MSRT, Microsoft's malicious software removal tool. The MSRT reports back when it removes malicious software and, using this data, Microsoft estimates computers cleaned per mille (thousand) or “CCM,” the number of computers cleaned for every 1,000 times that the MSRT is run. For example, if 50,000 scans resulted in 200 cleans, the CCM would be 200/50 = 4.

The researchers gathered data during the fourth quarter of 2011 and published the results in a report entitled Linking Cybersecurity Policy and Performance. Here you see a visualization of CCM levels for countries in the fourth quarter of 2011.

Analyzing the data, they looked for correlation between policies and CCM. For example, countries signing the Council of Europe Convention on Cybercrime do better than countries that do not. In addition to policies, they looked at various indicators and found correlations as shown below.

Using a model based on demographic and policy variables, the were able to predict CCM well, as shown here.

The authors emphasize that correlation does not imply causation, but this is an interesting big data application. Microsoft provides a valuable service for free, and in return gathers massive amounts of data on incidents of malicious software detection and removal. It is a win-win situation.

Wednesday, February 13, 2013

The State of the Union Address -- Internet related points

I was in class during the State of the Union Address last night, so read the transcript online today. I zeroed in on the Internet-related points.

The President mentioned the threat of hackers and called for legislation that would "give our government a greater capacity to secure our networks and deter attacks." Hey, who can argue against that? But, the curmudgeon in me notes that he spoke only of defending against attacks, which leaves me wondering what are we doing with respect to offensive attacks. I also have a slight worry that some of the attacks we spend taxpayer money deterring might be attacks on the intellectual property of movie studios, record companies, book publishers, etc.

He also announced that he had issued an executive order on Improving Critical Infrastructure Cybersecurity, but I have not yet read it.

The President also raised this rhetorical question "Ask any CEO where they'd rather locate and hire, a country with deteriorating roads and bridges or one with high-speed rail and Internet, high-tech schools, self- healing power grids." High speed Internet and high tech education are clearly on his mind. (For more concrete evidence of this, see FCC Chairman Genachowski's recent Gigabit City Challenge.

A comment which indirectly impacts the Internet was a call for stepped up energy and medical research -- "Now is the time to reach a level of research and development not seen since the height of the space race."

That is what struck me in the speech. Now for a couple of meta comments.

1. I chose to read the transcript rather than watch the video. It was faster and I could copy the above quotes as I went along. I was also able to quickly link to the executive order on security and the Gigabit City Challenge, but, I missed the emotional "content" of both the President and the audience. Perhaps that diminished my impression of the importance of the speech.

2. As soon as I finished reading the speech, I skipped over to and found that they did their thing and concluded that "The president spins his accomplishments on jobs, health care and deficit reduction in annual address." How cool was that?

Sunday, February 10, 2013

3D Printing technology marches on -- for better or worse

While Congress and the President and the NRA haggle over gun control laws, is distributing 3-D printing specs for guns and parts of guns.

Check the following video for a demonstration of a printed 30-round magazine for a semi-automatic weapon.


Update, April 1

It may be an April Fool joke, but the Defcad,org web site has a notice saying the site and domain name have been seized by the Federal Government and Grand Jury indictments were issued.

Their latest video, which is still up on YouTube, announces Defcad Search:

The video shows off some guns with 3D-printed parts, denounces the intellectual property and promises Defcad Search, a search engine with CAD files from which there will be no takedowns, ever.


Update 5/7/2013 has now released a complete, single shot pistol. You can read more about it in this Forbes article. Is this any more a threat than home made "zip" guns made from common hardware parts?

Update 5/10/2013

The State Department requested that Defcad stop distributing their pistol plans, which they did. However, they report that there had been over 100,000 downloads and a quick Google search turns up sites which are now offering copies. Once something is out, you cannot put it back.

Update 5/12/2013

Leland Yee, a California State Senator who is alarmed by 3D printing of guns, has called for regulations of 3D printers.

Update 5/24/2013

Moving away from the dark side -- Michigan Tech University is running a 3D printers for peace contest. It's your chance to win a 3D printer -- what would Mother Theresa have done with a smart printer?

Update 7/26/2013

The 3-D printed pistol is not yet ready for prime time.  Matt Ratto, a professor at University of Toronto printed a handgun -- it took  27 hours, a fair amount of computer know-how, a $50,000 printer, and $300 worth of plastic.  What will it cost in ten years?

Update 1/19/2014

Here is an article (with links) on a low-cost, open-source 3D metal printer from Michigan Technological University. It's a work in progress, but it is less expensive than off-the-shelf commercial plastic 3D printers.

Saturday, February 09, 2013

Why I signed a petition to name Susan Crawford next head of the FCC

I just electronically signed the petition to appoint Susan Crawford as the next head of the Federal Communications Commission. Let me tell you why.

The Internet was invented and deployed in the US. At one time, nearly all international traffic flowed through our National Science Foundation network. Today, our Internet is mediocre by the standards of developing nations and slipping.

In a recent interview by Bill Moyers, Crawford says U.S. Internet access is slow, costly and unfair and tells how we got in this fix. (The video is embedded below).

Here are a few quotes from the interview:
"What's happened is that these enormous telecommunications companies, Comcast and Time Warner on the wired side, Verizon and AT&T on the wireless side, have divided up markets, put themselves in the position where they're subject to no competition and no oversight from any regulatory authority. And they're charging us a lot for internet access and giving us second class access."

"So there's been a division. Cable takes wired, Verizon/AT&T take wireless. They're actually cooperating."

"In almost 20 states in America it's either illegal or very difficult for municipalities to make this decision for themselves."

"This is a moment when we have to separate out content from conduit. It should not be possible for a local cable actor or any distributor to withhold programming based on volume ...Everybody should get access to the same stuff at the same price and they should be announced prices."

"Michael Powell, who served as F.C.C. chairman for four years in the mid-2000s, is now the cable and telecom industry's top D.C. lobbyist."

"Meredith Attwell Baker who was one of the F.C.C. commissioners who approved Comcast's merger with NBCUniversal, left the agency four months later to join Comcast as a highly paid lobbyist."
Further reading:
Dan Gillmor, director of the Knight center for digital media entrepreneurship at Arizona State University, explains why Susan Crawford should be the next head of the Federal Communications Commission (

Cory Doctorow, Internet activist, journalist and science fiction, agrees that Crawford should run the FCC (
I searched for a rebuttal -- someone arguing against Crawford's appointment -- but was unable to find one. Please let me know if you know of one.

Update 1/28/2014

How cable companies "compete" -- dividing up the cable market

Comcast is near a deal to buy New York City, North Carolina and New England cable assets from Charter Communications Inc., but that sale is contingent upon shareholders approving Charter’s takeover bid for Time Warner Cable. Since Comcast would no longer be in the running for Time Warner Cable, they will probably accept Charter's offer.

This trading of monopoly territory reminds me of the way rival drug gangs divide up corners and housing projects -- I wonder if these guys watch "The Wire."

Tuesday, February 05, 2013

NMC-Educause report on technologies that will impact higher education

The New Media Consortium and Educause have teamed up on a report that looks into the crystal ball to predict technologies that will impact higher education in the next year, the next two or three years and the next four or five years.

The technologies they are bullish on are listed below. Do you agree? Can you think of educational applications for each of them? Download the report and see the basis of their predicitions.

Time-to-Adoption Horizon: One Year or Less
  • Flipped Classroom
  • Massively Open Online Courses
  • Mobile Apps
  • Tablet Computing
Time-to-Adoption Horizon: Two to Three Years
  • Augmented Reality
  • Game-Based Learning
  • The Internet of Things
  • Learning Analytics
Time-to-Adoption Horizon: Four to Five Years
  • 3D Printing
  • Flexible Displays
  • Next Generation Batteries
  • Wearable Technology

Sunday, February 03, 2013

Netflix's House of Cards disappointments (me)

Last night I streamed two episodes of Netflix’s miniseries "House of Cards."

House of Cards was produced by Netflix for distribution on the Internet, not via the traditional cable TV services. I am not a big TV watcher and am certainly not a drama critic, but I hoped it would be a terrific program, helping Netflix disrupt the status quo -- like comedian Louis CK's production of his own comedy show or Bill O'Reilly and John Stewart's comic debate.

I was disappointed. The protaganist, is Frank Underwood, a Machiavellian leader of the House of Representatives, who is out for revenge against the President who passed him over for Secretary of State.

I didn't get hooked because Frank Underwood is no Tony Soprano. Frank is one dimensional -- all selfish and plotting -- there is nothing to like about him. Tony is an evil murderer, but he is also loyal to his gang, loves and worries about his family and is a likable teddy bear of a guy.

The entire Netflix cast is one-dimensional -- the Lucretia Borgia wife, the hip reporter who blogs and knows that print is dead, etc.

There were a couple of innovations. I liked the way they showed people sending text messages on screen, and they released all of the episodes at the same time so people can watch them marathon-style if they do get hooked.

I hope I'm in the minority and most people love House of Cards, because I want Netflix to succeed and give us more high production value entertainment online.

Saturday, February 02, 2013

Cyberwar is a trending topic -- New York Times, Wall Street Journal and Washington post hacked

Last week, the New York Times, Wall Street Journal and Washington Post each reported attacks by Chinese hackers trying to discover documents and the names of sources for coverage of political corruption in China. They also reported that Chinese hackers had targeted Bloomberg, Reuters and other media sites.

The Times said the intruders used phishing attacks to break in around September 13 and infiltrated computers of at least 53 users, including those of the Shanghai and Beijing bureau chiefs. It seems they were looking for email and documents that might identify the sources for The Time's coverage of business deals that left relatives of Chinese prime minister Wen Jiabao wealthy.

According to Mandiant, the security firm the Times hired for help on this case, at least thirty western news organizations have been targeted by China. Many private companies also say they have been attacked by China, but they will not give forensic proof because that would be bad public releations and also reveal security secrets.

The Times reported that the hackers used university computers as proxies and switched IP addresses in order to remain anonymous. Oliver Richwood has warned against using China as a scapegoat and assuming they are responsible for all cyber attacks, but, in this case, the Times seems certain that the attacks are Chinese, saying they "closely matched the pattern of earlier attacks traced to China." The reported choice of targets supports that claim.

The outline of Cliff Stoll's TED Talk
Eric Schmidt of Google has said ”It’s fair to say we’re already living in an age of state-led cyber war, even if most of us aren’t aware of it.” Schmidt is correct, but "cyber war" is too broad a term. We may distinguish between at least three forms of cyber attack (1) espionage -- government, personal and business spying (2) sabotage that destroys or modifies data and (3) sabotage that effects the physical world. We can also characterize cyber attacks by their intent -- monetary gain, industrial advantage, political repression, political advantage, etc.

In his article "Fear Pays the Bills, but Accounts Must Be Settled," security expert Bruce Schneier acknowledges that cyber attacks are commonplace, but points out that hype often outstrips reality and leads to major funding decisions. Many stories focus on hypothetical sabotage of the electric grid and other infrastructure, but, as far as I know, the only type 3 attack to date was the 2008 US-Israel Stuxnet worm, which damaged an Iranian nuclear enrichment plant. (The New York Times also reported that story).

Hype or reality, cyberwar funding is on the increase, and there is a lot of money to made. For example, the recently increased funding for the Army cyber command.

Topographer Stoll's Klein wine bottle
All this talk about government-sponsored hacking reminded me of Clifford Stoll, an astronomer and system administrator who tracked down a KGB hacker at Lawrence Berkeley National Lab in 1986. Stoll documented his detective work in an insightful and funny book "The Cuckoo's Egg: Tracking a Spy Through the Maze of Computer Espionage." Check it out and read the reviews at Amazon. I read it at the time and remember really liking both the book and Stoll himself. If you wonder why I said I liked him as well as his book, watch his 2006 TED Talk.

The New York Times has also recorded an 8-minute news video on the Chinese break ins featuring interviews of the Times CTO and Nicole Perlroth, who wrote the story.


Update 2/19 -- The New York Times reports that Mandiant will soon publish a report stating that a specific unit of the Chinese Army, headquartered in a specific building in Beijing has been responsible for "an overwhelming percentage of the attacks on American corporations, organizations and government agencies."


Update 2/20 -- Mandriant has published a report detailing their evidence against the Chinese and the Chinese have denied that the charges. They have also produced a short video showing screen captures of the hackers at work. (Yes, the video could have easily been contrived and bogus).

The video shows alleged Chinese hackers setting up a gmail account using a US IP address, spearphishing, taking control ov victim computers, accessing a victim's Microsoft Exchange email, copying hacking tools from their repository in Shanghai and stealing files.

Security gurus are now going through the Mondriant report to see which malware was being used and making sure that it is known to the security and antivirus community.

It is too bad that Cliff Stoll did not have access to YouTube when he was working on the Cuckoo's Egg. His video would have been a lot funnier!


Update 2/24

Bloomberg has an article "How to curb Chinese cyberattacks." The title promises more than it delivers, but it differentiates between attacks against critical infrastructure and espionage and talks about what the administration has and should do.


Update 2/25

The New York Times continues the discussion of Chinese cyber-attacks -- the "cyber-cold war" -- between the US and China. The article surveys the debate over the appropriate response to Chinese attacks -- from diplomacy to better defense to counter attack. There is no clear answer because of the deep inter-dependencies of the Chinese and US Economies.


Update 3/1

The Chinese have countered US charges, saying hackers from the US have repeatedly launched attacks on two Chinese military websites, including that of the Defense Ministry.  They claim 144.000 attacks per month.  Do you believe the US is also conducting offensive attacks?  Do you favor our doing so?


Update 3/8

I've always assumed that Skype calls were securely encrypted, but that turns out not to be the case. With Microsoft's assistance, the Chinese government is monitoring Skype calls. University of New Mexico graduate student Jeffrey Knockel, who discovered the monitoring and is tracking the tracker, maintains a list of keywords the Chinese use for both censorship and surveillance. There are over 1,100 keywords on the list and he updates it daily on his Web site. Be careful about saying things like "Reporters without Borders", "Amnesty International", or the Macdonalds in front of Chunxi Road in Chengdu" if you are using Skype to talk with someone in China.


Update 3/9/2013

Chinese Foreign Minister Yang Jiechi says recent hacking allegations are on "shaky ground," and China opposes "turning cyberspace into a new battlefield or using the Internet as a new tool to interfere in the internal affairs of other nations.” He said that China’s government opposes hacking and has drawn up rules and laws to strictly forbid hackers and they have advocated and submitted specific proposals for a set of international rules governing the Internet under a United Nations framework.

Do they really want a truce? What I wonder is why we are not seeing stories of Internet espionage in China. I suspect it is going on.


Update 3/15/2013

A high level official has now explicitly called upon China to curtail hacking and enter into talks on the topic. While this seems obvious to a layman, in the protocols of international diplomacy, this is apparently an important step.

The speech in which hacking is addressed was by Tom Donilon, National Security Advisory to the President.

The New York Times article stresses hacking, but it is worth noting that hacking was a small part of the speech addressing "The United States and the Asia-Pacific in 2013." The main point of the speech was that the US has shifted priorities, increasing emphasis on the Asia-Pacific region.

In fact the section on hacking was only 319 out of 5,317 words and 3 of 78 paragraphs -- 4-6%. That is perhaps indicative of the over-hyping of the importance of these "cyberwar" attacks.


Update 3/15/2013

The US is talking stick as well as carrot.

General Keith Alexander, who runs both the National Security Agency and the new Cyber Command, told the House Armed Services Committee “This is an offensive team that the Defense Department would use to defend the nation if it were attacked in cyberspace. Thirteen of the teams that we’re creating are for that mission alone.”

The same day the nation’s top intelligence official, James R. Clapper Jr., warned Congress that a major cyberattack on the United States could cripple the country’s infrastructure and economy, and suggested that such attacks now pose the most dangerous immediate threat to the United States, even more pressing than an attack by global terrorist networks.

This is starting to sound like the "weapons of mass destruction" buildup to the invasion of Iraq.


Update 5/7/2013

The New York Times has reported that the US has accused the Chinese military of "mounting attacks on American government computer systems and defense contractors, saying one motive could be to map “military capabilities that could be exploited during a crisis.”


Update 5/20/2013

The New York Times reports that, after a three-month break, Chinese hackers have resumed their attacks on U.S. targets. The Chinese Foreign Ministry denies the accusation, and People’s Daily, which reflects the views of the Communist Party, called the United States “the real ‘hacking empire,’ ” saying it “has continued to strengthen its network tools for political subversion against other countries.” Other Chinese organizations and scholars cited American and Israeli cyberattacks on Iran’s nuclear facilities as evidence of American hypocrisy.


Update 5/22/2013

This story keeps on going. The Washington Post reported that Chinese hackers breached Google security in an effort to learn the identities of Chinese intelligence operatives in the United States who may have been under surveillance by American law enforcement agencies. It seems they tried tried to do the same at Microsoft, but failed to gain access.


Update 5/23/2013

The New York Times has published an article on the culture of hacking in China -- it is widespread and brazen. Companies advertise their capability and sell at trade shows. The article covered a trade show and quotes a salesman as saying “We can physically locate anyone who spreads a rumor on the Internet.” The company’s services include monitoring online postings and pinpointing who has been saying what about whom.

Another quote from the article:
The culture of hacking in China is not confined to top-secret military compounds where hackers carry out orders to pilfer data from foreign governments and corporations. Hacking thrives across official, corporate and criminal worlds. Whether it is used to break into private networks, track online dissent back to its source or steal trade secrets, hacking is openly discussed and even promoted at trade shows, inside university classrooms and on Internet forums.


Update 5/29/2013

The Washington Post reports that critical U.S. weapons system designs were compromised by Chinese hackers. Before things like reports of weapons in Iraq, I would have been more concerned about this report, but, now, I cannot help wondering whether it is hype to get congress to allocate funds for agencies and companies.


Update 6/1/2013

A New York Times column argues that the example of the British use of Elizabethan pirates against the Spanish navy is more relevant to the budding cyberwar between the US and China than the cold war between the US and the Soviet Union.


Update 6/1/2013

A New York Times article reports that The United States and China have agreed to hold regular, high-level talks on how to set standards of behavior for cybersecurity and commercial espionage, the first diplomatic effort to defuse the tensions over what the United States says is a daily barrage of computer break-ins and theft of corporate and government secrets.  That sounds hopeful.