Purdue Signals -- giving students feedback on how they are doing

I give my students feedback as to how they are doing relative to the class on assignments and quizzes throughout the semester. I also do informal, anonymous surveys to give them feedback on their relative level of effort, for example asking how many many actually watched a video that had been assigned or how long they spent studying a module or doing an assignment. These low-tech surveys take only a few minutes to administer and tabulate. My goal is to put information, and therefore responsibility for the outcome, in their hands.

Purdue University has a more ambitious effort for monitoring student progress and giving them feedback during the term. Since 2007, they have used Signals, a system that mines student data to predict their success in a course. They look at demographic, help-seeking and performance variables to judge how well a student is doing as the term progresses. Student feedback includes a red/green/yellow signal to indicate their overall progress as well as "canned" emails from their instructors and suggestions as to how to improve and where to get help.

Purdue reports that graduation rate, retention rate and grades have improved for students taking Signals courses. For example, Signals students got better grades in these courses:

This and the other results reported by the Signals team are encouraging, but there are many confounding variables. For example, professors choose whether or not to use Signals in their courses and those who choose to do so may be more more committed to teaching than those who don't.

If this sounds interesting, you can visit the Signals home page, read this short report on the project, read this discussion of ethical considerations in mining student data in this manner or watch these one minute videos.

Explanation of Signals for students:



Comments from two faculty members:



I am not sure whether I would use Signals if it were available to me as a teacher, but my low-tech approach to feedback on relative performance and effort is simple and I can do it on my own.

-----

Update 9/28/2013

Andrew Stewart suggest that we look at JISC's work in this area, writing:

Some interested findings emerging from Jisc's Assessment and Feedback programme around this area. As a distance learning student myself I'd love to have this kind of information at my fingertips.

Is Google turning evil?

Google recently announced that they will be using information from all their applications when ranking search results. (Google lists 35 data sources on their privacy page).

Their corporate motto is "don't be evil." Is Google becoming a bit evil?

When Google started, their page ranking algorithm was based on the number of links to a page. The reasoning was that a page many people linked to was more important than one with few incoming links.

Link count was the first indicator of the importance of a page, but over time, Google, and other search engines, has added ranking signals like the information in your Google profile, your language, location, previous searches, friend's "Plus 1" votes, Gmail messages, calendar entries and more. (The details of the Google search ranking algorithm are a closely guarded secret and it is frequently changed).

Google personalizes search results using these signals. You and I might get different results from the same search terms, and Google will be able to predict whether the "beetle" you are searching for is more likely to be a car or a bug. Privacy advocates are upset because Google uses information from your prior searches, location and language even if you are not logged in. Some also feel this new policy is a unilateral change in Google's terms of service.

Other critics, like Eli Pariser, argue that personalized search narrows our view of the world, creating a "filter bubble."

Google answers these critics in the following video, saying that personalized search results provide more relevant information for users:



Google has also begun promoting their own content in ranking search results. For example, I just did a Google search for Facebook founder "Mark Zuckerberg." The first result was a link to his Google Plus page, which has no posts. That was followed by images of him and a Web post about a fake Mark Zuckerberg. I had to scroll down to the second screen to see a link to his Facebook page. If I search on my own name, I am taken to my Google Plus profile, not my home page on the Web.

Google supporters point out that favoring their own content is reasonable, drawing an analogy to print publications. No one complains that we cannot see articles from the Los Angles Times sports section while reading the New York Times.

But, The Times could not do that even if they wished to, and it is easy to do on the Web. More important, Google has a conflict of interest. Unlike the New York Times, they are in the search business as well as the content business. Google runs the risk of losing brand credibility as a search engine if they start over-ranking their content. They will seem a bit greedy and evil.

And what of the future? Does Google favor YouTube videos over others? Even if they don't today, might they tomorrow?

I don't mind Google personalizing my searches, but they are not serving me well if they rank their information over better information on other sites. Google lets you show or hide personal results with a single click. Maybe they should do the same for "natural" search results in which Google content is not boosted.

Do you think Google has turned evil? Do you want personalized search? Would you want Google to rank their own content the same way they do other's content? If you are not sure, check the first half of this TWIT podcast, in which experts discuss the issue.

You can also see a pretty funny parody showing Hitler reacting to Google's policy changes, but be warned that it is full of profanity and geek in-group references.