Introduction

Effective Research Development and Grant Submission (RDGS) proposal evaluation is essential for ensuring that resources are allocated efficiently to research projects. RDGS programs are designed to foster innovation, promote scientific progress, and address societal challenges. A robust evaluation process maximizes the impact of RDGS investments by identifying research projects that have the potential for significant outcomes. Designing and implementing an effective evaluation process is complex and requires adherence to best practices that ensure transparency, fairness, and objectivity. This article outlines five best practices for evaluating RDGS proposals, including establishing clear and specific evaluation criteria, ensuring evaluator expertise and diversity, implementing a transparent and accountable evaluation process, providing constructive feedback to applicants, and focusing on impact and feasibility.

5 Best Practices

1. Establishing Clear and Specific Evaluation Criteria

A critical element of effective RDGS proposal evaluation is the establishment of clear, specific, and measurable evaluation criteria. Clear criteria ensure that evaluators can assess proposals objectively and consistently, reducing the risk of bias and enhancing the credibility of the evaluation process.^[1] The evaluation criteria must be tailored to the goals of the RDGS program, the target outcomes, and the broader funding strategy. Moreover, the criteria should be comprehensive and holistic, covering various aspects of the proposal, including scientific merit, innovation, alignment with RDGS goals, the qualifications of the research team, and the expected societal or economic impact.^[2] This approach allows evaluators to assess proposals in a balanced manner. Establishing these criteria in advance provides a roadmap for both evaluators and applicants, ensuring that everyone understands the expectations and requirements for successful proposals. Additionally, it is important to ensure that the criteria are flexible enough to accommodate the diversity of research projects that may be submitted.^[3]

2. Ensuring Evaluator Expertise and Diversity

Another best practice for effective RDGS proposal evaluation is ensuring that evaluators possess the requisite expertise and diversity to assess proposals accurately and comprehensively. Evaluators must be experts in the relevant fields of research to ensure that they can provide an informed assessment of the scientific quality and technical feasibility of proposals. However, the importance of diversity in the evaluation panel cannot be overstated. The composition of the evaluation panel should reflect a diversity of expertise, backgrounds, and perspectives to ensure that the evaluation process considers all aspects of the proposals.^[4] As research becomes increasingly interdisciplinary, evaluation panels must include experts from different fields who can evaluate various dimensions of the research. In addition to expertise, diversity in evaluators’ perspectives is essential for mitigating biases that can influence the evaluation process. Human biases, such as confirmation bias or overconfidence bias, can influence decision-making.^[5] By including a diverse group of evaluators with different backgrounds, funding bodies can reduce the likelihood of such biases influencing the assessment of proposals.

3. Implementing a Transparent and Accountable Evaluation Process

Transparency and accountability are cornerstones of an effective RDGS proposal evaluation process. Transparency in the evaluation process fosters trust among stakeholders and ensures that all parties understand how decisions are made.^[6] It also ensures that evaluators remain accountable for their decisions. To achieve transparency, it is essential that the evaluation process is clearly documented and communicated to all participants, including applicants, evaluators, and funders. This documentation should include the evaluation criteria, the process by which proposals will be reviewed, and the timeline for decision-making. Accountability in the evaluation process can be reinforced through the establishment of oversight mechanisms.^[7] These mechanisms help ensure that the evaluation process is rigorous and fair.

4. Providing Constructive Feedback to Applicants

Another key best practice in RDGS proposal evaluation is providing constructive feedback to applicants. Effective feedback not only benefits applicants by helping them understand why their proposal was successful or unsuccessful, but it also contributes to the overall improvement of the research community. Constructive feedback helps applicants understand the strengths and weaknesses of their proposals, providing valuable insights for improving future submissions.^[8] Providing feedback also enhances the fairness of the evaluation process, ensuring that decisions are based on objective criteria rather than subjective preferences or unacknowledged biases. When evaluators provide detailed, actionable feedback, applicants can learn from their mistakes and refine their proposals for future submission, increasing the overall quality of research proposals submitted to RDGS programs. Feedback should be specific and focused on the key aspects of the proposal, including scientific rigor, innovation, relevance to the RDGS goals, and potential for impact. Providing feedback in a constructive and supportive tone ensures that applicants view the evaluation process as an opportunity for improvement rather than a judgment of their abilities.^[9]

5. Focusing on Impact and Feasibility

Another best practice for effective RDGS proposal evaluation is focusing on the impact and feasibility of the proposed research. As RDGS programs often aim to address societal challenges or promote innovation, it is crucial to assess how the proposed research will contribute to these goals. Impact refers to the broader consequences of the research, including its potential to advance knowledge, improve practices, or address pressing societal issues.^[10] Evaluators should assess whether the proposed research has the potential to make a meaningful contribution to its field or to society. The feasibility of the research must also be carefully considered. A highly innovative proposal may have great potential, but if the research is not feasible within the proposed timeline or budget, it may not be worth funding. Evaluators should assess whether the research team has the necessary resources, expertise, and capacity to successfully complete the proposed work. This assessment involves evaluating factors such as the qualifications of the research team, the adequacy of the proposed budget, and the likelihood that the proposed methodology will lead to successful outcomes.^[11]

Conclusion

By following best practices such as establishing clear evaluation criteria, ensuring evaluator expertise and diversity, implementing a transparent and accountable process, providing constructive feedback, and focusing on impact and feasibility, funding bodies can ensure that their RDGS programs achieve their intended outcomes. These best practices help to identify high-quality research proposals and foster a culture of fairness, accountability, and continuous improvement in the research community.

Take Away

This article outlines 5 best practices to consider when conducting a RDGS evaluation. By adhering to these practices, evaluators can maximize the impact of RDGS funding and contribute to the advancement of knowledge and the resolution of critical challenges.

[1] Patton, M. Q. (2008). Utilization-focused evaluation. Sage publications.

[2] MacLean, M., Davies, C., Lewison, G., & Anderson, J. (1998). Evaluating the research activity and impact of funding agencies. Research Evaluation, 7(1), 7-16.

[3] Chubb, J., & Reed, M. S. (2018). The politics of research impact: academic perceptions of the implications for research funding, motivation and quality. British Politics, 13, 295-311.

[4] Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2010). The program evaluation standards: A guide for evaluators and evaluation users. Sage Publications.

[5] MacLean, M., Davies, C., Lewison, G., & Anderson, J. (1998). Evaluating the research activity and impact of funding agencies. Research Evaluation, 7(1), 7-16.

[6] Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2010). The program evaluation standards: A guide for evaluators and evaluation users. Sage Publications.

[7] Chen, H. T., & Rossi, P. H. (1987). The theory-driven approach to validity. Evaluation and program planning, 10(1), 95-103.

[8] Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research. Sage publications.

[9] Kaplan, B., & Shaw, N. T. (2004). Future directions in evaluation research: people, organizational, and social issues. Methods of information in medicine, 43(03), 215-231.

[10] Vinkenburg, C. J., Ossenkop, C., & Schiffbaenker, H. (2022). Selling science: optimizing the research funding evaluation and decision process. Equality, Diversity and Inclusion: An International Journal, 41(9), 1-14.

[11] Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2010). The program evaluation standards: A guide for evaluators and evaluation users. Sage Publications.