What is peer review?
Simply put, peer review is the assessment of someone's work by fellow experts. It is used in science publishing to decide whether a journal should publish a set of research findings. And it's used by reputable research funders to help decide whether a research proposal is worthy of investment. Essentially, it's a quality control step.
Some of the things we ask peer reviewers to comment on are: are the proposed experiments realistic and feasible; is it original; do the applicants have the right expertise and collaborative networks; what could go wrong and do the researchers have good back-up plans; are they in the right place; is the budget appropriate; is the work relevant to the charity? It’s also a chance for experts to comment on whether the proposed study has all the right ethics and regulatory approvals. This includes whether the application properly takes into account the ‘3Rs’ of using animals in research – reduce numbers to a minimum where possible, replace with an non-animal alternative where appropriate, and refine procedures to minimise harm and improve welfare – and, for clinical trials, whether the design adequately considers safety and patient numbers.
For each application, we gather a number of peer review reports, and all the applications and peer reviews are then considered by the relevant expert committee.
Why does it matter?
AMRC’s five principles of peer reviews are:
- Accountability: This says that charities should be open and transparent about their peer review procedures – we publish ours here and also list the members of our expert committees.
- Balance: Advisory panels should reflect a fair balance of experience and scientific disciplines. This is so we can properly consider all the different research proposals that come our way. We consult a number of different peer reviewers for each application and have lots of varied expertise on our committees.
- Independent decision making: Our expert committees are independent of the charity's administrative staff and trustees, so we know the advice is an honest and open scientific evaluation.
- Rotation of scientific advisers: It is important that expert advisors serve for a fixed time period before being replaced by another expert. Our committee members usually serve 3-6 years, and rotation is staggered so that at any one time there is a mix of new and more experienced members. This makes sure that decisions don’t just sit with the same group of people and fresh viewpoints can be heard.
- Impartiality: There are number of ways to ensure advice is fair and free from undue bias – our committees are a mix of researchers who receive Bloodwise funding and those who don’t, we have international as well as UK representatives, and we apply a rigorous conflicts of interest policy that, for example, means committee members who are applicants or have a connection with an applicant don’t participate in that decision.
We know that peer review isn’t perfect – research is inherently risky so predictions can be difficult and the balance of how much pilot data is needed is a fine one. But it’s a tool we can use to help guide decisions and these principles mean it’s the best it can be by being fair, objective and expertly interrogated.
Peer review also provides critical feedback to applicants. Those who are unsuccessful can use the feedback to modify and improve their proposal, giving it a better chance of it eventually being supported. And those we do fund can still use the peer reviewers’ comments to refine the project once it’s underway.
By following high-quality processes in making research funding decisions, every pound we invest goes to research of the highest standard. Our investments in health and medical research ideas then have the greatest chance of having an impact.
Policy makers also recognise this indicator of quality, meaning our research benefits from the Charity Research Support Fund awarded by the government to universities to support costs like maintenance and running costs of labs, as well as NHS infrastructure for research.
What is a peer review audit?
Every five years, all members of AMRC complete a survey to explain how its peer review processes and policies ensure these five principles are upheld. It’s a good chance for members to review their own practices to ensure they are gold-standard. And it’s also an opportunity for AMRC to hear about some of the innovations in how charities review research proposals and for members to learn from one another.
Ultimately, it provides a kitemark of quality, so that supporters and the public can be reassured that the charity’s investments in research goes to the best, original research for patient benefit.
We were pleased to hear Bloodwise were awarded this badge of quality in the 2015 audit!