If you are happy with your Pure profile (including up-to-date activities and impact), and have proposed any outputs that you wish to, then you have completed the RRE. There is now no need to rank your outputs.  You will be asked to choose the order that your items will appear in (First to Fifth Output, followed by ‘Other RRE output(s)’) however this order will not be used for reviewing and will have no bearing on the grade the output is given. On the morning after the deadline, all new outputs in the ‘Proposed for RRE’ list, will be put forward for review. There is no 'submit' button.

Outputs that have already been graded may also be re-reviewed as part of the RRE process.

If you do not see the RRE tab when you log into your Pure account, it may be because you are not eligible for the exercise. Only REF-eligible staff currently on the University payroll are required to take part in the RRE. The list of teaching and research eligible staff in Pure is updated roughly monthly, and for eligible research-only staff it is updated twice a year. So if you are a new member of staff you may not have access until after an update.

If you think you should be RRE eligible because you have research in your contract, are employed more than 0.2FTE, and are not a research assistant or research associate, please contact your relevant RRE contact in the first instance.

No extensions to the deadline (once announced) are possible. In order to prepare outputs for the reviewers, a snapshot of the data in Pure will be taken on the morning after the deadline. You can continue to propose outputs, activities and impacts in Pure and they will be captured in the next RRE instead.

Your RRE proposed outputs and their scores will never be publicly visible. In Pure, individuals can see their own scores, and a small number of staff with administrative privileges are able to look at all RRE scores within a Unit of Assessment - UOA(s).

If you want to update your external profile, create a CV , or any other Pure related functions, please see the Pure Support pages.

Heads of School use RRE grades as one indicator to inform their assessment of the quality of the aggregate research output profile (i.e. the School and UOA profiles) and provides key background data to support their decision making for the REF submission in a fair and transparent manner. They are aware that RRE grades for specific outputs may be revised and recalibrated by the UOA coordinating team as part of the preparation of the REF 2021 submission, and in light of new information about the output, such as subsequent citation rates or prizes.

When assessing an individual's research profile for probation or promotion other indicators of output quality pertinent to the discipline or field are used as the primary source (peer review, referee comments, article citations, book reviews, etc.). The assessment of output quality is one part of the assessment of overall research performance, along with the record of PGR supervision, research funding, etc. as detailed in the probation and promotion guidelines. This is in line with the implementation of the statement of research expectations.

Hence, RRE grades should not be presented in the paperwork for probation or promotion by either the applicant or in accompanying supporting statements; or introduced in discussions in probation and promotion committees.

We would like to know about all your potential 3* and 4* outputs, so please propose any outputs that you believe are likely to be judged internationally excellent (3*) or world-leading (4*)

All graded outputs from previous years’ RREs will stay in your list of proposed outputs.

This may be because it is not eligible - for example it has been discovered that it was first published before 1 January 2014. In addition, if it wasn’t published when the deadline closed, grading is at the review panel's discretion.

Some review panels have also been disrupted in RRE2020 by industrial action and the developing Covid-19 situation, preventing review of outputs.

Any ungraded outputs will roll forward into the next Research Review Exercise - there is no need to propose them again.

Previously, outputs that were not Open Access compliant may not have been scored, but from 2019 they have been considered as eligible and reviewed - provided they meet the conditions above.

There is no right of appeal against the academic judgement of reviewers. Staff may request re-review of an output assessed by a previous exercise at the discretion of their School’s Research Director. This would only be appropriate where there has been a significant change in the profile of the output, for example a conference paper now published in a journal, or a major increase in citations etc.

Formative evaluation (giving individuals feedback on their outputs) is not part of RRE objectives, but your School Director of Research or Output champion can give you advice and support about improving the quality of outputs. Please refer to your local contacts for more information.

Firstly, check if the research output is in your 'My research' tab. If the item is not in Pure, please follow these instructions for adding a new output.

If the item is there, but not showing as RRE eligible, it might be because it was published before 1 January 2014. If that's not the case, please contact REF2021@manchester.ac.uk.

Note, when you open the pop-up window to propose outputs, you must view 'All' rather than 'Proposed' to see all eligible outputs: click on these headings at the top of the pop-up window to toggle between the two views. 

Use this step-by-step guide. Remember that all journal articles and conference proceedings accepted for publication since 1 April 2016 should be deposited via the Open Access Gateway, rather than directly into Pure. This means that the Library can ensure that your paper meets the open access requirements to be eligible for the next REF by verifying the manuscript version, checking applicable journal and funder policies and setting embargo periods on publication.

The Library will aim to create the record in Pure within one working day of your deposit via the OA Gateway. 

If your output has been proposed in RRE, it cannot be removed from your list as we wish to keep a historical record of all proposed and scored outputs. However if you have proposed an output in error or, for example, you believe that it is not eligible, you should raise this with your local RRE contact in the first instance.

First check if the output is eligible: Does it contain original research, where you have made a distinct and substantial contribution to the output? Has it been published since 1 January 2014?

Of your eligible outputs, we advise you to select the best, most important pieces of work. Reviewers will assess originality, significance and rigour of the piece, and these factors should be the main drivers when choosing your output, before you consider other indicators such as citations. As long as your contribution was significant, it does not matter whether you were a lead author. This means that your best outputs for RRE may not be the ones that you would choose for other purposes, e.g. making a promotion case, or demonstrating the breadth or your work.

If you need help in selecting your best outputs for RRE purposes, please contact your School Research Director, Head of Division, or Institute Director in the first instance for advice.

We use the REF2021 definition of original research: 

Definition of research for the REF2021

1. For the purposes of the REF, research is defined as a process of investigation leading to new insights, effectively shared.

2. It includes work of direct relevance to the needs of commerce, industry, and to the public and voluntary sectors; scholarships; the invention and generation of ideas, images, performances, artefacts including design, where these lead to new or substantially improved insights; and the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes, including design and construction. It excludes routine testing and routine analysis of materials, components and processes such as for the maintenance of national standards, as distinct from the development of new analytical techniques. It also excludes the development of teaching materials that do not embody original research.

3. It includes research that is published, disseminated or made publicly available in the form of assessable research outputs, and confidential reports.

This means that review articles will not usually be RRE eligible (an exception might be a Cochrane Systematic Review for example). If you need help in selecting your best outputs for RRE purposes, please contact your School Research Director, Head of Division, or Institute Director in the first instance for advice.

Unlike previous RREs, we are not asking you to rank your outputs. You will be asked to choose the order that your items will appear in (First to Fifth Output, followed by ‘Other RRE output(s)’). This order will not be used for reviewing and will have no bearing on the grade it is given.

Output order is not an indication of likely grade. It is not related to the REF quality levels of world leading (4*), internationally excellent (3*) etc. 

All new outputs will be reviewed, although we would not expect individuals to have more than six new outputs and in exceptional cases we may need to limit the number of outputs that are reviewed from an individual.

You will be asked to mark each output with one of the following:

• First output
• Second output
• Third output
• Fourth output
• Fifth output
  
• Other RRE output(s)

This indicates the order that your outputs will appear in only, it will not be used for reviewing and will have no bearing on the grade the output is given.  Outputs marked ‘Other RRE output(s)’ will be given the same priority as those marked first to fifth.

If your output was added to Pure whilst still at 'accepted' stage, it may not yet have been updated by the Library. You can amend this by opening the output item, and clicking "Add publication status and date" under Publication State.

Choose "Published" and add the date of publication.

Items that are "E-pub ahead of print" may also show as not yet published, but these are eligible for RRE because they are available in the public domain.

Please do not amend any details about public access to the output, as this may affect REF eligibility. If in doubt, contact the Library at uml.scholarlycommunication@manchester.ac.uk

You can now propose all your likely 3*/4* outputs, although we would not expect individuals to have more than six new outputs and in exceptional cases we may need to limit the number of outputs that are reviewed from an individual.

During the migration of data to Pure, the dummy text 'RRE 2015' was added to any outputs proposed in previous RREs that did not have any output supporting text. This is because the field for output supporting text is mandatory. You can edit this, please refer to the guidance from your Faculty. 

All RRE outputs will be assessed in line with the REF2021 Assessment Criteria and graded according to the REF grade levels, as follows: 

Where you see ‘no grade available’, this means that your output has not been graded. This may be because it wasn’t published when the deadline closed for example. It will remain proposed and reviewed in a future RRE.

All selected outputs will be examined in detail by reviewers: journal rankings will not be used to rate outputs.  The University of Manchester is a signatory of the San Francisco Declaration on Research Assessment which recognises that journal-based metrics are a poor surrogate measure of the quality of individual research articles.

See the RRE Criteria and Working Methods for further information on how outputs will be assessed. 

Unclassified (or “u/c”) means quality that falls below the standard of nationally recognised work, or work which does not meet the published definition of research for the purposes of this assessment, for example a review article that does not contain any original research.

Where you see ‘no grade available’, this means that your output has not been graded. This may be, for example, that it wasn’t published when the deadline closed.