Professor and Library Staff Join Forces to Measure Reach of Ideas in Fluid Digital Landscape

How do you measure the impact of an idea?  

For academics, this is a crucial question as they seek to track the influence of faculty research in an ever-changing digital landscape. A group of Wellesley faculty and staff are leading the way in answering this question within the field of pedagogical innovation.

Traditional methods of investigating the impact of research focus on tracking how many times a scholar’s work is cited in other publications. However, given the slow nature of academic publishing, “it can take a really long time to see if a paper is going to make an impact,” explained director of research services Megan Brooks. She noted that traditional tracking methods also don’t track how many times an article is downloaded, tweeted, or mentioned in blogs.

Adele Wolfson, Nan Walsh Schow ’54 and Howard B. Schow Professor in the Physical and Natural Sciences and Professor of Chemistry, was troubled by the limits of traditional research metrics, especially in the field of pedagogical research (the study of teaching methods). “If you’re working in a lab,” she said, “you’re always building on previous research and citing it in reports. But in pedagogical research, people use the ideas and methods discussed, but they don’t often build upon it with subsequent papers.” Consequentially, it is difficult to quantify the impact of innovative pedagogical research.

Wolfson was curious to find out whether there were other available methods of tracking research, and so she went to her colleagues at Library and Technology Services. It turned out that LTS staff members were already discussing the burgeoning field of alternative metrics or “altmetrics.”

Alternative metrics seek to address the limits posed by traditional methods by offering a faster, more multidimensional method of tracking the impact of an article. Companies like Zotero and Mendeley allow scholars to track how many times an article has been mentioned on blogs or Twitter, how often an article was downloaded, or whether data sets were used by other researchers. “The field of altmetrics has been developing over the past few years as social networking has grown,” said Deborah Lenares, manager of library acquisitions and resource sharing. “It’s something that librarians have been watching and have been looking for ways to support its growth.”

Together Wolfson, Lenares, Brooks, and librarian Alana Kumbier (who has since left the College) researched and identified ways that altmetrics could better track pedagogical research, given the unique characteristics of both the research and the metric. The group also sought to inform scholars what more they could do to effectively share their work with interested peers. The resulting paper, “Monitoring and Promoting the Impact of Pedagogically-Related Scholarship,” was published in the journal Biochemistry and Molecular Biology Education. (Fun fact: The paper was the first Wellesley article published under the new Open Access policy). On November 14, the group presented its work at the meeting of the Scholarly Communications Interest Group of the Association of College and Research Libraries (ACRSL) New England chapter.

Looking ahead, Wolfson, Lenares, and Brooks hope that their work will begin conversations on campus about new and dynamic ways to track the impact of faculty research. Now there’s an idea worth tracking.