A DC Primer: Measuring Grant Impacts at the National Library of Medicine

By: irinaz
Posted: March 17, 2012

Prior to my tenure at the University of Michigan’s School of Information, I was a grants officer, so I was thrilled that there was an Alternative Spring Break project that combined parts of my work experience and what I’m currently studying. My project at the National Library of Medicine (NLM) was collecting initial bibliometric data on their feasibility grants, the R21 award. These are NLM’s brief (up to 2 years) grants – which seek to quickly test ideas, models, or techniques that can potentially have big impacts in biomedical informatics fields. These are new awards that have been funding innovation since 2002.
Under the direction of one of NLM’s Associate Fellows, Suzy Roy, my research process involved:
• Pulling data from NIH’s incredible tool, NIH RePORTER (http://projectreporter.nih.gov/reporter.cfm) to get all R21 awards.
• Running searches in PubMed to retrieve all articles that attribute funding to any of these grants. Fun fact! Did you know that NLM keeps track of grant numbers and makes funding easily searchable and easy to track? One more argument for librarians running the world, if you ask me.
• Figuring out the number of times each article produced as a result of NLM R21 funding was cited (this is the bibliometric part), and ultimately how many citations each R21 grant produced through Scopus searches.
• Areas of further research, had I had more time, would have included searching Web of Science and Google Scholar to get a fuller picture of the articles’ citation impacts that R21 grants generate.

My project was only scratching the surface of the wealth of information that is related to these grants, and discussions with Suzy and her colleagues, as well as gathering the data brought up some really intriguing questions, foremost among them was how does an organization define success for these types of grants? Citations are one initial mechanism, but what about patents, or software developed as a result of NLM funding? These are weighty questions, but what I hope my work did was start off initial evaluation efforts and point to more defined areas for future research. There were clear standouts – in terms of citation impact – among the 46 grants I analyzed, and it would be fascinating to delve deeper into those works, identify measures of success, and see if they apply across grants or could possibly be used as an institute-wide metric. This path is one pretty quantifiable way that can really demonstrate a library’s value, which is one challenge librarians constantly grapple with.

Those were the nitty-gritty project specifics. Suzy also took pains to make sure I got big-picture details. I was very interested in finding out about NLM’s grantmaking priorities and processes. The fun facts continue! Did you know that as part of the U.S. Department of Health and Human Services, NLM (which is one of 27 institutes that comprise the National Institutes of Health) is part of the largest grantmaking arm of the US federal government? I was therefore thrilled to be able to quiz Dr. Valerie Florance, NLM’s associate director for Extramural Programs, with absurdly detailed grants and development questions.

All in all, my Alternative Spring Break at the National Library of Medicine was a perfect combination: I helped get the ball rolling on a massive analysis project, learned specific skills that I am taking back with me for my research at my current job, learned about grantmaking from one of the foremost national institutions, and most importantly, met an incredible array of people dedicated to and passionate about their work that – in so many ways – trickles down to affect the lives of countless people on a daily basis.

See more about NLM: http://www.nlm.nih.gov/about/index.html
NLM’s Associate Fellows Program: http://www.nlm.nih.gov/about/training/associate/proginfo.html
More on the R21 awards: http://grants.nih.gov/grants/funding/r21.htm
More on NIH: http://www.nih.gov/about/