San Jose State and Udacity release results for summer pilot courses

San Jose State University (SJSU) and Udacity have run two rounds of trial online courses. The pass rates in the first round were poor, but a press release from SJSU and a blog post from Udacity report improvement in the second trial, which was run this summer.

As you see, the pass rates were better in the summer than the spring for every class and the online students in two of the summer classes had higher pass rates than on-campus students.

Both Udacity and SJSU say they are "encouraged" by these results, but there were differences between the terms that confound the data and make it difficult to explain, say, the improvement in elementary statistics or the poor results in entry level math. The three courses that had been offered in the spring were revised, but the numbers of students in the classes and their backgrounds and motivations were also substantially different.

For example, the spring classes were relatively small and the students were all trying to earn credit. In the summer, there were more students and their motivations varied, as shown here:

This is not surprising. I have noted earlier that college credit motivates relatively few MOOC students and, in fact, non-credit students may turn out to be the most important audience for and customers of online classes.

The results of this trial are far from definitive, but this is a time of rapid innovation and such experiments are to be encouraged.

Speaking of innovation, I cannot bring myself to conclude this note without pointing out that "passing with a grade of C" seems like a poor metric for "success" in a course. That is the traditional university criteria, but the Internet gives us the ability to break courses into fine-grained modules and to provide more precise, informative measures of success.
-----

Update 9/12/2013

San Jose State has released a report on the Udacity experiment entitled "Preliminary Summary SJSU+ Augmented Online Learning Environment." The report author is not the preliminary investigator for the study, Elaine D. Collins, but was written "for" her by consultants at the RP Group.

It is reviewed in two posts:

1. Chronicle of Higher Education, "Few Surprises in NSF Report on San Jose State U. Test of Udacity Courses article"

2. Inside Higher Ed, "After weeks of delays, San Jose State U. releases research report on online courses"
_____

Update 11/18/2013

The academic senate at San Jose State is considering a resolution to restrict the power of the administration to unilaterally enter into course contracts -- to keep control with the academic departments.

The Academic Senate is expected to vote on Monday on a proposed policy that would forbid the university to sign contracts with outside technology providers without the approval of tenured and tenure-track faculty members in whatever department would be affected.

-----

Update 11/25/2013

The Chronicle of Higher Education interviewed two leaders on opposite sides of the MOOC debate at San Jose State and they discover common ground. Anti-MOOC professor Peter Hadreas learned that pro-MOOC professor Khosrow Ghadiri has been using MOOC material to run a flipped class -- not merely having students enroll in a MOOC -- and that it is very labor intensive, taking 80 hours per week. They agree that pedagogical innovation and experiments are good things.

I also noted that one of the professors is an engineer and one is a philosopher. Can you guess which is which? Which do you think spends 80 hours per week on his teaching?

-----

Update 12/13/2013

Georgia Tech designs its Udacity pilot to avoid failure. They distance the themselves from Udacity's debacle at San Jose State University.
-----

Update 12/18/2013

San Jose State will resume their testing of the material developed for three online courses -- Elementary Statistics, Introduction to Programming and General Psychology in the spring semester. Enrollment will be capped at 70 students for the statistics class, 150 students for the programming course and 35 students for the general psychology class. At least half of the seats for programming and statistics will go to SJSU students and the rest will go to CSU students on other campuses. The course will use SJSU's LMS, Canvas. This sounds like a traditional online class using the material that was developed for the Udacity trial.
-----

Update 2/2/2014

In the fall of 2012, students in two traditional sections of an introductory electrical-engineering course earned passing grades at rates of 57 percent and 74 percent, respectively. In an experimental third section, which was “flipped” to incorporate the MIT videos, the pass rate was 95 percent, but they did not adhere strictly to the MIT material.

In the spring of 2013 they also ran three sections, one of which used edX content. In the traditional sections, students passed at rates of 79 percent and 82 percent. In the experimental section, the pass rate was 87 percent.

The spring experimental section followed the MIT curriculum than did the fall 2012 section -- they covered more material.

The professor, Khosrow Ghadiri (mentioned above), said the pass rates of the spring-2013 trial should not be compared with those of the fall-2012 trial because the students learned different material and took different examinations. In the fall, he used the MIT content to help teach his own syllabus. In the spring, he used the MIT professor’s content and learning objectives.