MRC e-Val – Researcher feedback 2009
Following completion of the 2009 MRC e-Val survey, we invited researchers to give us feedback on the entire e-Val process, commenting on both the technical aspects and the user experience.
More than 150 researchers responded, giving marks out of ten for four different aspects of the system as well as for their overall ‘satisfaction’ with the process. These scores have been aggregated and the results are shown below:
% Scoring 0-5 |
% Scoring 6-10 |
Average Score Overall | |
Generally, it was easy and intuitive to navigate MRC e-Val |
15% |
85% |
7.9 |
For each question, I was able to understand what it was the MRC wanted to know, either with or without the support of guidance text in the survey |
15% |
85% |
7.4 |
The performance of the system was good (that is, when I clicked links in the form, pages refreshed and/or updated at an acceptable speed.) |
15% |
85% |
8.4 |
The number and detail of questions in the form seemed appropriate for the MRC’s objectives. |
15% |
85% |
7.5 |
Overall, I was satisfied with MRC e-Val. |
15% |
85% |
7.5 |
Reaction to the general navigation and use of the system was positive, notably the ability to return to each section as and when required. The increased use of drop-down menus and look-ups was well-received by researchers, who found this an easy and consistent way to provide their data. Researchers also told us that the PubMed look-up function was very useful and in reduced the time it took to provide details of their publications. Feedback also indicated that being able to delegate the form to administrators and other users was a useful way to help achieve the deadline.
While overall we are extremely pleased with this result, we acknowledge that some users were dissatisfied with the process and remain unclear as to the objectives of the exercise. We would like to respond to these specific points straight away.
Some researchers questioned the purpose of collecting these data. The aim of the MRC e-Val project is to provide information that can be used by the MRC to better understand the progress, productivity and quality of output from MRC-funded research. We will publish results from MRC e-Val in the next 2-3 months, and expect that further reports will be published throughout 2010. Data collected via e-Val will also contribute to the MRC’s regular publications (the Annual Review, for example) and wherever e-Val data have been used, documents will display the MRC e-Val logo. We are currently focusing our effort on extracting the most value from the data.
MRC e-Val effectively replaces the output/narrative content of the final grant report. We plan to repeat the MRC e-Val exercise annually. This does not mean, however, that researchers will be required to re-enter any data. Each survey will present the results for previous years and researchers will simply be asked to add, amend and, if appropriate, confirm the data we hold on outputs from their research. We recognise that research outputs may not be realised until some time after a grant finishes and one of the key objectives of the e-Val system is to maintain an ongoing dialogue with researchers whose work has been funded by the MRC.
We have already implemented some immediate changes to the form based on the feedback received. The first change that researchers will notice is in the publications section, where they can now see the full reference on the screen. We hope to make more improvements to the form later in the year.
Researchers provided a lot of feedback about the way in which questions and guidance text were worded. This is extremely useful to us and will help us shape the way in which the questions are presented to users in future surveys. We are committed to limiting the number of changes to the structure of the survey so that researchers know what to expect each year, but we are keen to expand on the help text and FAQ provided.
In the meantime, we would welcome any further feedback. You can contact us at evaluation@headoffice.mrc.ac.uk.