Source: https://researchimpactacademy.com/blog/lessons-learnt-from-over-100-nhmrc-impact-case-study-reviews/
Lessons learnt from over 100 National Health and Medical Research Council (NHMRC) impact case study reviews
In 2018, the National Health and Medical Research Council rolled out its new grant scheme. Among those funding opportunities were the Investigator Grants where they introduced, for the first time, a three-part Impact Case Study to be reviewed as part of the applicants track record.
The case study requires applicants to write about the real-world impact from a past program of work in three sections of 2000 characters each: Reach and significance of the research impact; Research programs contribution to the research impact; Applicant’s contribution to the research program.
Since the introduction of the Investigator Grants, we have worked with over 160 researchers across health and medicine disciplines from more than 20 universities and institutes in Australia, to review and refine their impact case studies. Throughout this process, we have seen many different approaches and heard firsthand the misunderstandings and frustrations from the researchers perspective.
Common mistakes and misunderstandings
- Confusion as to what goes in each of the three sections, due to confusing terminology, inexperience in understanding impact, and limited best practice examples. Typically, we see researchers do a good job with Field 3, the “applicants contribution to the program”, as they are more comfortable talking about their own work.
- Researchers often misunderstand the meaning of impact and think that the results of their research project or study is the impact, rather than use of the knowledge or sharing of an output from that study, that leads to a change.
- We often see that researchers want to write about the latest research that they did or their prospective and newly funded research. However, it is best to use old work as that will have had the most significant time for uptake and use leading to impact.
- Despite being clearly outlined in the guidelines, many researchers believe that the impact sections need to be related to the knowledge gain section of the current proposal. The impact sections stand alone from the rest of the application.
Field-specific mistakes
Field 1: Reach and significance of the research impact
- Many applicants are writing a review of the literature from their field of research, rather than talking about their particular program of work.
- We often see broad-brush statements about how the work has impacted on something without numbers and evidence to back it up.
Field 2: Research programs contribution to the research impact
- All too often we see people rewriting the same content in Field 1 & 2 with minor changes, rather than providing details of the work that led to or underpinned the impact outlined in Field 1.
Field 3: Applicant’s contribution to the research program
- Writing about new work currently being done rather than their role in the program of work that led only to the impacts in Field 1, not their entire career.
- Talking about their entire career when specificity to the program is required.
- Repeating leadership sections (other parts of the grant application) verbatim.
Observations and opinions
Difference between the emerging leadership and leadership categories
Conversations with researchers and reviewers highlighted an expectation that those applying for the leadership levels, due to their seniority, will have had more than one type of impact, with some saying the more types of impact, the better the researcher. However, this is problematic on two fronts; first, not all kinds of research will generate multiple types of impact. Second, there is not enough room to include multiple impact examples, and if reviewers consider this necessary then there at least needs to be more space available for applications applying under the leadership category compared to the emerging leadership category.
Structure and writing styles
Everyone has a different opinion on how best to structure and write each of the sections. Some prefer dot point while others are embracing the narrative. Personally, narratives work better to provide context around the particular impact and how it came about. Use of dot points is attractive to some but, I believe, comes with the issue of needing to provide evidence against each point.
When it comes to writing styles, the past tense is the logical choice considering you are talking about a program of work that has already been completed. In the first two fields, it is best to use “the program”, “my program”, and then use “I” when talking about the applicant’s contribution to the research program.
Inconsistencies in peer review
Conversations with those in the sector, suggests some disparity in the expectations of reviewers based on their interpretations of the guidance documents. As an example, some reviewers believe that reach and significance of the impact is not about the applicant’s impact, but rather the applicant’s field of research based on other peoples research programs as well.
There is also considerable uncertainty about how the reviewers will, despite guidance, consider and evaluate multiple impact types versus one impact type. Additionally, how the reviewers can ensure consistency in ranking amongst the different impacts, in particular, how to assess knowledge impacts based on citations versus implementation in guidelines used in multiple countries.
The most valuable piece of advice
Read and use the following key elements from the guidelines
- Understand what is meant by a program of work. The definition clearly states that this is a cohesive body of research by the applicant.
- The definition of impact includes three key terms that help articulate the program’s impact. Use the terms, adopted, adapted, or used, where appropriate.
- Use the category descriptor tables that outline expectations across the different impact types.
Overall, it has been an absolute pleasure and privilege to work with so many talented researchers across Australia that have had incredible impacts from their work. As a nation, we should be incredibly proud of the depth, breadth and reach of our health and medical research contributions to society, I sure am.
Watch past webinars for the NHMRC Ideas and Synergy Grants on our membership site.
No comments:
Post a Comment