Success in grant applications for women and men

Sex-disaggregated data on the success rates of applications to the individual grants at the European Research Council and selected national funding agencies show similar outcomes for women and men. There are large differences in success rates between countries and in all countries with applicants to the European Research Council men are applying disproportionally more (and women less) compared to the demography of the researchers in the higher education sectors in the respective countries. Therefore, the proportion of women funded is even lower than their representation in the fields of Natural Science and Engineering and Technology. Some contributing factors are discussed and the question on how the current and future success rates could be interpreted is raised.

1 Introduction

Although women in a number of countries have been eligible to participate in higher education for more than a century, the proportion of women continuing with an academic career and reaching the higher academic positions is still low. Women are in a minority as researchers in science and technology disciplines and the proportion of women among full professors in disciplines such as physics and chemistry is around 15 % in the EU-28 (European Commission, 2019). A number of explanations have been proposed to explain the “leaky pipeline”, e.g. where a disproportionate number of women receiving PhDs leave academia along the career path starting at the Post Doctoral level, continuing through receiving a permanent position, and ending with a lack of women as full professors. The explanatory factors that have been suggested include lack of role models (Shen, 2013), low access to important networks or less advantages of networking (Forret and Dougherty, 2004), effects of unconscious bias and stereotypic thinking among women and men (Moss-Racusin et al., 2012), and/or unevenly distributed resources for women and men (Ceci and Williams, 2011).

One of the most important factors in achieving an advanced independent academic career is to be successful in grant applications. A number of organizations and societies have become aware of the importance of funding in shaping the scientific community. Therefore, the European Union (EU) and the US National Science Foundation Advance Program have funded research projects on achieving gender equality in academia during the last decades (https://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5383, last access: 29 May 2020; https://ec.europa.eu/programmes/horizon2020/en/h2020-section/science-and-society, last access: 29 May 2020; https://ec.europa.eu/programmes/horizon2020/en/h2020-section/promoting-gender-equality-research-and-innovation, last access: 29 May 2020). In addition, the EU has fostered gender balance in research teams, strived to ensure gender balance in decision-making and integrated the gender dimension in research and innovation (https://ec.europa.eu/info/policies/justice-and-fundamental-rights/gender-equality/gender-balance-decision-making-positions_en, last access: 29 May 2020).

In this paper we present success rates collected from individual grant schemes. These data are available as sex-disaggregated data in some countries and at the European Research Council (ERC). This means that there is sufficient data available to investigate how large a proportion of the research funded by tax money is awarded to women and men. We have focussed on the large amount of data collected on the prestigious individual grants (Starting Grant, Consolidator Grant and Advanced Grant) at the ERC. More specifically we have used the data on the applications submitted to the Physical Sciences and Engineering (PE) panels from 2012 through 2018. Data from selected national funding agencies are also included for comparison.

The two major aims of this paper are to investigate (1) the success rates and compare the outcome for women and men and (2) if the number of applications from women and men are in parity with the proportion of female and male researchers found in the fields of physical sciences and engineering (European Commission, 2019).

2 Material and methods

Success rates were determined by dividing the number of funded applications by the number of submitted applications.

2.1 European Research Council data

There are ten panels within the field of Physical Science and Engineering (PE) at the ERC that fund individual researchers (Mathematics, Fundamental Constituents of Matter, Condensed Matter Physics, Physical & Analytical Chemical Sciences, Synthetic Chemistry and Materials, Computer Science & Informatics, Systems & Communication Engineering, Products & Process Engineering, Universe Sciences, Earth System Sciences). Success rates for women and men applying to these panels were compiled for Starting Grants, Consolidator Grants and Advanced Grants for the period of 2012–2018. Ineligible applications and withdrawn applications were removed and disregarded in the analysis. Country-specific success rates are only presented for countries with at least 50 applications during the period 2012–2018. Data on the proportions of women researchers in Natural Science and in Engineering and Technology per country in 2015 were retrieved from the European Commission (2019).

The grades (A–C) from all the PE panels (sex-disaggregated data) from 2012–2018 were collected. This data include applications from all countries. In the reviews at the ERC a proposal can receive either an A (highest quality research project), a B or a C in the first stage of the evaluation (Step 1 which evaluates the synopsis of the full proposal). Only proposals with grade A get to pass to the second stage of the evaluation (Step 2 which is the evaluation of the full proposal). Researchers getting grade B are eligible to apply after a one-year break and those getting grade C are eligible to apply after a two-years break. In the second stage of the evaluation proposals can either get an A or a B. All researchers reaching the second stage are eligible to apply again without any restrictions, independent of the grade (A or B).

Linear regression was used to analyse the relationship between the proportion of women researchers in the higher education sector and the proportions of applications to the ERC that were submitted by women. Chi-square (X²) tests were used to analyse differences between grades and differences between success rates for women and men.

2.2 National funding agencies

Success rates for grant schemes funding research in science and technology from a selection of national funding agencies were retrieved from webpages and reports from these agencies (Table 2). We used the statistics from the open calls, that is proposal calls with no pre-defined topics. The applications in these calls are reviewed by experts, but the systems vary greatly and are specific for each country. For example, proposals could be reviewed only by a small group of reviewers (n=4), could be reviewed by a panel (n>8), and/or could include expert external reviews.

Chi-square (X²) tests were used to analyse differences in success rates for women and men.

3 Results

3.1 Analysis of success rates at the European Research Council (ERC)

The total number of applications for the years 2012–2018 at the ERC in the combined PE Panels was 24 009, of which 4350 were submitted by women and 19 659 submitted by men. Of these 315 applications are not presented on a country level, since they are submitted from countries with less than 50 applications during the years 2012–2018. The overall success rates for the combined granting schemes (Starting Grant, Consolidator Grant and Advanced Grant) were similar for men (12.1 %) and women (12.7 %). The proportion of successful applications for women and men differ between countries ranging from a low of around 1 % successful applications to approximately 25 % successful applications (Fig. 1, Table A1). Three countries have significantly higher success rates for women than for men, United Kingdom (X² (1, $N=\mathrm{4178})=\mathrm{12.05}$, p<0.001, Ireland (X² (1, $N=\mathrm{316})=\mathrm{5.7}$, p=0.017) and France (X² (1, $N=\mathrm{2807})=\mathrm{4.0}$, p=0.045). Note that the numbers of applications differ between countries, and some proportions are based on a low number of applications (Fig. 2).

Figure 1 Successful applications (% of all applications) for women and men per country for the ERC PE panels 2012–2018 in countries with at least 50 applicants during that period.

Figure 2 Number of applications for women and men per country for the ERC PE panels 2012–2018 in countries with at least 50 applicants during that period.

The proportions of women among the total number of applications by country varied between 6 % and 36 %, with the two-thirds in the interval between 15 % and 20 %. This is a large deviation from the proportion of women among researchers in the higher education sector in the respective countries (European Commission, 2019). Applicants to the PE panel calls are active in either Natural Science (NS) or Engineering and Technology (ET). In Fig. 3 data on the proportion of women among researchers in 2015 in NS and ET are presented together with the proportion of applications submitted by women to these panels at the ERC between 2012 and 2018. All countries had a lower proportion of women applying for ERC grants relative to the proportion of women researchers in the scientific field of NS and ET (Fig. 3). In all countries men are applying disproportionally more than their representation among researchers would suggest. There is a statistically significant positive relationship between the proportion of women researchers in a country and the proportion women applicants (p=0.0019 and p=0.0113, R²=0.39 and 0.28, for NS and ET, respectively).

Figure 3 Proportion of women among researchers in the higher education sector (%) in Natural Sciences (blue diamonds) and Engineering and Technology (red circles) (source: Table 4.2 in European Commission, She Figures 2018) and the proportion of applications submitted by women to the ERC 2012–2018 in the PE panels. Each point represents one country. The black line indicates the 1:1 line with an equal proportion of applications submitted by women and proportion of women in the higher education sector.

The scores for all the applications (24 009) submitted to the PE panels during the period 2012–2018 are presented in Table 1. Women had a significantly higher proportion of their applications scored within the C category (29.9 %) compared to men (26.5 %) (X² (1, $N=\mathrm{24}\mathrm{009})=\mathrm{21.49}$, p<0.00001), which affects their possibility to apply the next two years because of eligibility restrictions described above.

Table 1 Distribution of the final scores on all applications submitted to the ERC 2012–2018 in the PE panels in number of applications and the proportion of applications with that particular grade per sex.

Table 2 National individual grants with reported success rates per sex. There were no statistical differences between success rates for women and men.

Sources: ¹ Australian Research Council (2019); ² Natural Sciences and Engineering Research Council of Canada (2015); ³ Styrelsen for Forskning og Uddannelse (2017), ⁴ Deutsche Forschungsgemeinschaft (2018); ⁵ Swedish Research Council (2019); ⁶ United States Government Accountability Office (2015); ⁷ National Environment Research Council (2018).

3.2 National funding agencies

We searched for success rates at national funding agencies to investigate the representativeness of the ERC PE data. Three levels of ambition could be identified: funding bodies that (1) collect and present sex-disaggregated statistics on success rates (Table 2), (2) collect sex-disaggregated statistics on success rates and mention the outcomes in graphs, but without showing the actual numbers, (3) do not collect (or at least not mention that they collect) sex-disaggregated data. In addition, among the sex-disaggregated data on success rates in the disciplines of Science and Technology (Table 2) national funding organizations showed rather similar success rates for women and men, and no statistical differences between sexes could be found. In the reports from Australia, Sweden and United Kingdom the number of applications per sex were available, and could directly be used in the analysis. For Germany, USA and Denmark the numbers of applications were calculated from the total number of applications and the estimated or presented proportion of women among the applicants. The number of applications could not be retrieved from the Canadian report and therefore no statistical test could be performed. The apparent difference in success rates in Denmark could be due to the low number of applications from women (78 out of 690, of which 5 was funded).

4 Discussion

4.1 No large differences between success rates for women and men

Our analysis shows that success rates for women and men applying for grants are in most cases similar at the ERC (Fig. 1) and at national funding organizations (Table 2), with a few countries (United Kingdom, Ireland and France) having higher success rates for women for the ERC-applications.. The differences in success rates are larger between countries than between women and men within countries (Fig. 1, Table A1). The success rates at selected national funding agencies ranged from around 10 % in Denmark to 60 % in Canada (Table 2). These differences in overall level of success rates at the national agencies could be due to how the national research funds are distributed (directly to universities, in open calls, or in calls aiming at certain research questions) in the different countries, and are not further investigated in this study.

The main difference between the scores for women and men in the ERC calls was the higher proportion of scores in the C category among women applications. Access to data to explain success rates of individual applications is not generally publicly available. In this study we have not evaluated if the success rates are reflecting the objective quality of the applications or earlier performances (CV). There are well-known examples of implicit bias in evaluation of job applicants (Moss-Racusin et al., 2012), promotion (Lyness and Heilman, 2006), invitations to put other scientists' work into perspective (Conley and Stadmark, 2012), and bias in scholarly publications have also been found (van den Besselaar and Sandström, 2017). Regarding research grants, earlier studies (Wennerås and Wold, 1997; Holst and Hägg, 2018) have identified bias in the evaluation process, which resulted in lower competence scores on women's applications than men's, with the objectively measured identical merits, resulting in lower success rates at equal merits. In addition, van der Lee and Ellemers (2015) noted lower scores for women on the quality of researcher, but not on the quality of proposal or knowledge utilization. By contrast, Ceci and Williams (2011) reference several studies that do not show any bias in grant evaluation and Waisbren et al. (2008) found that differences in success rates between women and men could be explained simply by the academic rank of the applicants.

4.2 Fewer women than men apply as compared to those employed

Success rate is only one criteria that should be considered when examining the lower number of women funded. As the success rates for men and women do not seem to differ in most schemes (Fig. 1, Table 2) focus should be on the number of applicants. In some of the countries (Fig. 3) the proportion of women in the higher education sector in NS and especially in ET is low which results in small pools of potential women applicants. Moreover, the proportion of women applying is lower than what could be expected from the proportion of women employed and available in the pool of potential applicants (e.g. Fig. 3). Why women are applying to a much lower degree than men could depend on a variety of reasons including the support from their institutions, such as who are allowed to apply, who gets institutional support, support from national contact points and grant offices to apply, or who has collegial support to apply or someone to polish the application before it is actually submitted. In addition, knowledge (conscious or unconscious) about the bias that repeatedly has been shown in the reviewing process when it comes to evaluation of merits (Wennerås and Wold, 1997; Holst and Hägg, 2018; Tamblyn et al., 2018) could discourage some women from applying.

4.3 Are the current success rates fair?

If women in academia act as women applying for jobs outside an academic setting, e.g. they have met all required qualifications before they apply for a job and are highly qualified (Mohr, 2014; Ignatova and Tockey, 2019) we would expect women to have a lower proportion of their applications graded with a C than men have. Available data on the grades from the ERC applications show that men have a higher percentage of their applications graded as A and B, and a lower percentage graded as C compared to women (Table 1). Does this mean that men applying have higher qualifications and more novel ideas than women or is there bias in the evaluation procedure?

When success rates are presented by funding organizations it is understood that the underlying assumption for equal success rates for women and men is the goal, and that there is no bias in the reviewing process. This assumption needs careful consideration since the proportion of women researchers that apply for grants is lower than the proportion of male researchers that apply (Fig. 3). Success rates are calculated based on the total number of applications and to increase the number of women with successful applications it is maybe not the success rates as such that needs attention, but rather the question how to increase the number of applications from women so that the proportion represents the number of researchers in their scientific fields. Who are the women we should encourage to apply? And, more critically, how? If more women with low or average merits apply and the success rates stay the same, more women would receive grants, but from both the reviewers' perspective and from a researcher with lower merits it would be a waste of time writing proposals that would not be funded. However, practice makes perfect, so from the low merit women's perspective it could be a good investment nevertheless especially if adequate feedback is given to improve the next application.

Since the success rates are equal between women and men today, and if we assume that there is no bias in the evaluation, we need to encourage women from all levels to apply. This raises the question: What makes women not apply? Is it the culture in the research institutions or the wording of the open calls? We need to identify the women that should apply, but currently are not applying. We also need to address and change institutional practices that cause lower application rates for women.

4.4 What would happen if more women applied?

Currently we assume that the pools of women and men in our analysis are equal in quality and society expects equal success rates. Here we speculate on what would happen with the success rates for women if more women applied for grants, and we can identify three different outcomes. The success rates for women could (1) be lower than at present, (2) remain the same, or (3) become higher for women than at present. What would be the implications of these three outcomes? First, the success rates for women could become lower if poorer applications than average were received from the pool of women applicants, or if there is a bias in the current evaluations that is masked by differences in the current pools of women and men applicants. Lower success rates for women than men could discourage women from applying. Secondly, if the success rates for women remained the same we must assume that women with average potential, as seen from the total pool of applicants, are currently not applying, but if they did they would succeed as the average does in the pool. In this case, and with no additional funding added, the total number of men funded would decrease and the number of women funded would increase, since there would be an increased number of women in the pool. Finally, if the success rates for women increased we should assume that would be due to the addition of a larger number of qualified successful applicants, and they are most likely found among women with the highest potential. It is unlikely that unequal success rates would be viewed positively or tolerated for the long-term by the applicants, by the funders or by society.

4.5 Other funding sources

We have focussed on data from the ERC and from national funding agencies where sex-disaggregated data are compiled. Data from private funding organizations are often impossible to analyse because nearly everything from the process of evaluation to the number of people applying is not public knowledge. Some private foundations are progressive in achieving, via funding, more women research leaders. This is for example done by requiring a lowest acceptable ratio of women among the funded applications for prestigious schemes and if that ratio is not met when all available funds are distributed not all funds are used, but a line drawn where the ratio is for example 30 % or 40 % women funded (https://www.kaw.wallenberg.org, last access: 8 January 2020, Knut and Alice Wallenberg's Foundation, Sweden).

4.6 Data collection and future research

Data collection forms the basis needed for the investigations of bias in the granting process, and we strongly recommend that all funding organizations collect, analyse and present data on the composition of the applicants. Detailed data from for example the Swedish and Australian Research Councils are open and available online (https://www.vr.se/analys/svensk-forskning-i-siffror/vetenskapsradets-forskningsfinansiering-i-siffror.html, last access: 2 June 2020 (in Swedish) and https://www.arc.gov.au/policies-strategies/strategy/gender-equality-research/gender-outcomes-ncgp-trend-data, last access: 2 June 2020). There is a need for an in-depth analysis of the entire pool of applications submitted that could justify (or not) the current success rates. Are women and men that apply for open grants equally merited, i.e. do the two groups have the same distribution of qualifications?

The current EU Horizon 2020 project Grant Allocation Disparities from a Gender Perspective (https://www.granted-project.eu/, last access: 29 May 2020) aims at investigating the entire application process with prior performance, the evaluation process, etc. taken into account. We are looking forward to reading the reports from that project and other initiatives investigating this important topic.

Other interesting topics for further research are: what are the differences between the countries where the percentage of women that apply are closer to parity with their presence in academia compared to the countries with more skewed proportions (Fig. 3)? Are the academic cultures different? Are there practices that can be adopted by countries where women are not applying to the same extent? As shown above, even if there is a large variation across countries, all countries had an underrepresentation of women applicants to the ERC.

5 Concluding remarks

A more thorough understanding of the behaviour regarding applying for research grants globally is needed. Applicants need support from their institutions and knowledge about how subtle biases throughout careers can impact the culture, including the numbers of women in the pools of potential applicants and application behaviours, is needed for change to happen. Presentations of sex-disaggregated data on current success rates are important, but more knowledge is needed to answer the question “Why are men allocated disproportionally more research grants than women in the disciplines of science and technology?”.

Appendix A

Table A1 Country-specific numbers of applications from women and men to the PE panel at the ERC (2012–2018), the success rates and the proportion of women among all applicants (%). Only countries with at least 50 applications during this time period are included in the table. Source: European Research Council, https://erc.europa.eu/sites/default/files/publication/files/ERC_funding_activities_2007_2013.pdf (last access: 4 June 2020).