Friday, January 02, 2009

Rating and reputation in scholarly publication

We discuss systems for rating and reputation. Such systems are common in e-commerce and social networks, and we are beginning to see them in scholarly publishing.

I have posted a number of the articles I've written on my Web site, but do not track downloads or citations. Today I received an email from a journal stating:

As a service to our authors, we are pleased to provide you with a monthly report tracking readership for your article "A Framework for Assessing the Global Diffusion of the Internet":

9 full-text downloads between 2008-12-02 and 2009-01-02 29 full-text downloads since date of posting (2008-09-03).
This count came from a service of Berkeley Electronic Press, and you can read about their methodology here.

Citations are also used as a measure of the value of an academic paper, and Google Scholar provides citation counts. If I go to Google Scholar, I see that the above article has been cited by others 77 times.

The Association for Computing Machinery (ACM) is the most important professional society in my field, and they also report scholarly metrics for articles in their journals and conference proceedings. You can query the ACM database and see the number of times an article has been cited by other ACM authors and the number of downloads during the past six weeks and during the past year.

These are "walled gardens." For example, ACM does not keep records on the article mentioned above because it was not in one of their publications -- they only record downloads and citations of ACM publications by ACM members. Still, they are valuable first steps in using electronic reputation systems to assess scholarly contributions.

Would you consider the rating of scholarly writing in deciding whether to take a class from a particular professor? Would you consider the student-contributed ratings at a site like Rate my professors?