Forward Thinking: Analyzing gender equity across MSFHR award programs

As BC’s health research funding agency, with a focus on talent development, we think deeply about how to best support BC health researchers and balance important considerations like equity and diversity. Although these issues are not ones the Foundation can solve singlehandedly, we are considering our role – and our responsibilities – as a funder in fostering equity in health research.

Dr. Zena Sharman, MSFHR’s Director, Strategy, is an expert in gender and health. Here, she, and her colleagues Dr. Julia Langton, Manager, Evaluation & Impact Analysis and Amanda Paleologou, Research & Evaluation Specialist, outline what our data tell us about gender equity in MSFHR’s funding programs, what we’ve been experimenting with, and how we’re moving forward.

Forward Thinking is MSFHR’s blog, focusing on what it takes to be a responsive and responsible research funder.

---

Introduction

In the spring of 2018, we published a blog post on what we knew and what we wish we knew about gender equity in health research funding. We also committed to several action steps: analyzing applicant gender, piloting unconscious bias training, developing an equity strategy and defining equity. In the spirit of accountability and making our learning visible, we wanted to share what we’ve learned since then, what we’ve been experimenting with and how this is informing our next steps.

What we’ve learned since 2018

Gender breakdown for applicants

We previously reported success rates by gender for our two longest running funding programs, the Research Trainee and Scholar awards. We were pleased to see that our data showed similar success rates by gender, recognizing that in absolute numbers, we funded fewer female early career researchers than male early career researchers (from the years 2001 to 2017).

We now have three years of application and funding outcome data (2017 to 2019) for our refreshed suite of funding programs. Our current suite of funding programs promote talent development through three mechanisms: 1) salary awards that “buy out” time for research activities, 2) team-based knowledge translation (KT) awards that support researchers and research users to work together to support use of research evidence in policy and practice, and 3) funding for commercialization of promising health technologies.

Over the last three years across all our programs, 51% of applicants were female, 48% were male and 1% preferred not to disclose their gender. Female applicants received more awards and had a higher success rate (36% of applications funded; 183 awards) than male applicants (29% of applications funded; 137 awards). Total funding amounts to male researchers and female researchers were similar ($30.3M and $29.7M respectively).

This led us to examine applications and funding outcomes by gender across our three types of funding awards:

Salary Awards (Scholar, Research Trainee, Health Professional-Investigator)

• There were a similar number of male and female applicants (340 and 354, respectively).
• Male and female applicants had similar success rates and 52% of salary award recipients were female.

KT Awards (Reach, Convening & Collaborating, and Implementation Science Team Awards)

• More female researchers apply for these awards. For our Reach and Convening & Collaborating competitions, 102 of the 138 teams that applied (74%) were led by female principal investigators.
• Success rates were also higher for teams led by female investigators, ranging from 37% to 78% for female-led teams versus 25% to 61% for male-led teams.
• Female-led teams received 76% of KT awards; in the case of our Implementation Science Team competition, all five teams who received project grants are led by female investigators.

Innovation to Commercialization Award

• More male researchers tend to apply for and receive Innovation to Commercialization awards. While 25% of applicants were female, only 14% of award recipients were female.

How does MSFHR compare with other funders?

Research suggests that gender gaps in funding outcomes are prevalent and in fact, analyses of Canadian Institutes of Health Research (CIHR) data suggests that gender gaps are more likely when reviewers focus on the person (as opposed to the project).

It’s difficult to reconcile the CIHR data with our funding outcomes, given that our Research Trainee, Scholar and Health-Professional Investigator awards are personnel awards where track record as a researcher accounts for a larger proportion of the scores than other evaluation criteria. Over the last three years, women have performed as well as or better than men for all our funding competitions with the exception of the Innovation to Commercialization award.

Other analyses of CIHR data showed that women had significantly lower success rates for personnel awards compared with men but that research topic area may influence funding outcomes. In short, funding outcomes are complex and likely reflect an interplay between factors such as the nature of the funding opportunity, gender, career stage, and subject of research to name a few. At MSFHR, we will continue to follow the literature and conduct our own analyses in this area. In addition, all of our programs undergo an annual program learning and improvement cycle where insights from several data sources (information from peer reviewers, applicants, award holders, and other stakeholders) highlight what is working well or where changes may be required.

What we’re doing now (and what we’ve been experimenting with)

Application review

We look to the literature and other funders for ways to improve peer review and research funding allocation processes and mitigate bias in peer review and decision making. Based on our review of the limited evidence that is available, we decided to experiment with two interventions in the application review process; unconscious bias training and gender neutral application review.

We piloted unconscious bias training for our peer reviewers for our 2018 Innovation to Commercialization competition using CIHR’s unconscious bias training module and received positive feedback from reviewers. Since then, we have implemented this training for all of our funding programs and our peer reviewers are required to complete it before starting their review.

As a next step to support and foster gender equity in our funding programs, we piloted a new gender neutral application review process for our 2019 funding competitions. The applications our peer reviewers received listed only the applicant’s last name, and those submitting letters of reference were instructed to use gender neutral language and were provided with guidance to support this.

As a learning organization, we collected feedback about the gender neutral application review from our peer reviewers, in-person at the end of each panel meeting. We also administered an anonymous survey about the peer review process in general which included some questions about the gender neutral application review process and the unconscious bias training.

Reviewers reported that unconscious bias training was generally helpful in preparing them to review applications and reported that they found it to be a useful way to reduce bias in peer review.

When it came to the gender neutral application review there was support for the intention of the process but many reviewers reported that it did not work in practice and that it was not logistically possible to remove gender from the application review process. This is consistent with the available literature suggesting that there are additional cues to an applicant’s gender even when particular information about the applicant is hidden from the reviewer.

Many reviewers suggested there were other more effective ways to mitigate gender bias. For example, reviewers suggested that unconscious bias training, as noted above, and balanced review panels are effective strategies to support the equitable assessment of applications.

We also looked at gender balance in our peer review panels. We learned that from 2017 to 2019, the overall number of female and male peer reviewers was similar (54% female, 46% male). However, gender is only one factor we look at when choosing reviewers – we need to ensure we have reviewers with relevant expertise and that we minimize conflicts of interest. We can see this in action with our KT awards panels where 83 % of the reviewers were female. This is partly due to the fact women make up the majority of experts in the relatively new field. We will continue to monitor the composition of our peer review panels to ensure the equitable assessment of all applications.

After reviewing feedback from the peer reviewers, as well as the emerging literature about ways to mitigate bias in the review of research funding applications, we made the decision to discontinue the pilot of the gender neutral application review process. We will continue with unconscious bias training and we will strive for a diverse make up for our peer review panels, balancing gender and expertise.

Equity and diversity survey

To learn more about who is applying for and receiving MSFHR funding, we recently implemented an equity and diversity survey. It collects information similar to equity and diversity questionnaires used by Canada’s federal research funding agencies and will enable us to understand diversity as it relates to our own work, as well as contributing to the bigger picture understanding of BC’s and Canada’s research system diversity. Data such as these are crucial to support subsequent conversations about actions that may need to be taken to address equity and diversity in the research system. The survey asks applicants, team members, trainee supervisors, and peer reviewers to report their gender, age and whether they identify as LGBTQ2+, Indigenous, racialized and/or disabled. It also includes an open text question where we invite feedback about how we can improve the survey.

Collecting enough data to perform robust analyses may take a few competition cycles (we typically receive ~200-250 applications/year) but over time this survey will allow us to have a more nuanced understanding of who are our applicants, award holders, and peer reviewers.

What we’re planning to do – and hoping to learn – next

In 2018, we committed to developing an equity strategy for MSFHR and to defining what equity means in the context of that strategy. On further reflection, we thought this approach might create the possibility of an unintentionally siloed approach to equity. After surveying the literature and talking with our stakeholders as part of the process of developing MSFHR’s next strategic plan, we decided that it made more sense to take an integrated approach.

That’s why MSFHR’s forthcoming 2020-2025 strategic plan will include a strategic commitment to fostering a more equitable, diverse and inclusive health research system. At MSFHR, we believe diversity in research is important for cultivating talent and promoting inclusive excellence, which in turn drives discovery and helps build a more innovative and impactful health research system.

We acknowledge that all of this work is taking place in a rapidly-shifting context where research funding agencies, universities and other actors are increasingly focused on equity, diversity and inclusion (EDI). For example, the federal government now requires Canadian universities to establish and monitor equity and diversity targets for Canada Research Chair awards. In May 2019, the Government of Canada announced the Dimensions: Equity, Diversity and Inclusion Canada charter and pilot program for Canadian post-secondary institutions. Several BC-based post-secondary institutions were selected for the pilot – Camosun College, the University of British Columbia, Simon Fraser University and Vancouver Island University.

We’re excited about working collaboratively on gender equity and EDI with others in the research funding sector and the health research system, both within and outside the province. We see opportunities to share knowledge and promising practices with each other, including in the context of our work on research on research through the newly established research on research institute (RoRI) where equity, diversity and inclusion is on the research agenda for funders across the globe.

We would love to learn from others who are working towards a more equitable, diverse and inclusive health research system. Leave a comment or get in touch (jlangton@healthresbcdev.wpengine.com) if you have ideas or expertise to share.