#### Inputs required to estimate research value

#### Step 1: Primary outcome

**Type of primary endpoint**
The primary outcome measure or endpoint captures the most important aspects of health outcome in the research.

**Express results in natural outcomes (e.g. heart attacks avoided) or in QALYs?**
The benefits of research can be expressed in either natural outcomes or in Quality Adjusted Life Years (QALYs).
Using QALYs requires more inputs but enables a comparison of the health benefits of further research and implementation efforts across diverse clinical areas.

**Is the outcome a benefit (e.g. cure) or a harm (e.g. heart attack)?**
For natural outcomes: the value of additional research is expressed in terms of ‘benefits gained’ or ‘harms avoided’ depending on whether this outcome is a benefit or harm.

**Name of outcome e.g. heart attack**
For natural outcomes: this will be used in reporting results.

**Do the treatment costs depend on the primary outcome?**
For binary outcomes: in some cases treatment costs will depend on whether the primary outcome occurs or not.
For example, if a treatment is used to prevent disease progression then it will cease to be used (and its cost will no longer be incurred) if the individual progresses.

**Number of possible states if the primary outcome does / does not occur (4 maximum)**
For binary outcomes: the primary endpoint may be a scale or a composite outcome which is composed of a number of health states.
If there are different levels of health and costs associated with these health states then these can be considered explicitly here.
For example, if primary outcome is a composite of heart attack and stroke then there are two possible states if the primary outcome occurs and the costs and health consequences associated with these states must be explicitly considered.

**Conditional on the primary outcome occurring, what is the probability of being in this state?**
For binary outcomes: if the primary outcome is composed of distinct health states, different proportions of individuals may be expected to enter these health states.
For example, if primary outcome is a composite of heart attack and stroke then conditional on the primary outcome occurring 40% and 60% of these individuals may be expected to experience heart attack and stroke respectively.

**Patient time horizon / time in this state (years)?**
For binary outcomes: if differential survival is considered then represents the expected survival time associated with each state.
Otherwise this represents the patient time horizon considered for the decision i.e. how far into the future individual patient outcomes are modelled.

**What is the health utility associated with this state?**
For binary outcomes: this is a number which represents the health related quality of life associated with a state.

**What are the disease related costs associated with this state?**
For binary outcomes: these are the costs associated with a particular disease state, they do not include the costs of the treatment under consideration.

**What is the health utility associated with the pre-transition health state?**
For survival outcomes: this is a number which represents the health related quality of life associated with the pre-transition state.

**What are the expected monthly disease related costs associated with the pre-transition health state?**
For survival outcomes: in the same manner as for health utility changes in the expected survival must be linked to changes in disease related costs.
These are the costs associated with a particular disease state, they do not include the costs of the treatment under consideration.

**By how much is a one unit increase in the primary outcome expected to increase/decrease the health state utility?**
For continuous outcomes: the effect of changes from baseline on changes in health related quality of life (utility) will depend on the severity of the disease and range of the outcome measure.
'Mapping' studies which use statistical methods to estimate the effect of a unit change in a natural outcome on utility provide this link.

**By how much is a one unit increase in the primary outcome expected to increase/decrease monthly disease related costs?**
For continuous outcomes: changes in the primary outcome may also be expected to result in changes in disease related costs.
These are the costs associated with a particular disease state, they do not include the costs of the treatment under consideration.

**How long is the treatment effect expected to last? (months)**
For continuous outcomes: the scale of the health gains and disease related costs associated with changes in the primary outcome will depend on the expected treatment effect duration.
Estimates of treatment effect duration exist for few outcomes so in practice with will require expert opinion to inform this.

#### Step 2: Interventions

**How many treatment options are under consideration? (Maximum of 4)**
There may be a number of relevant treatment options for a given indication.
This app currently allows for up to 4 options to be considered.

**Current level of utilisation (%)**
Some estimate of the current level of utilisation of the interventions in clinical practice is required to establish the value of changing practice if the results of new research suggest a change.
It can also be used to establish whether there is greater value from encouraging the implementation of what existing evidence suggests is the most effective intervention rather than conducting new research.
The utilisation of all treatments must sum to 100%.

**Choose method of entering baseline probability of outcome**
An estimate of event rate with the baseline treatment is required.
This is used to obtain an estimate of the absolute effect of the interventions on the primary outcome by applying the relative measure of effect to the baseline risk.
There are two options for entering this data. 1) Upper and lower 95% range: this may come from discussion with an expert and/or from a confidence interval reported in the literature.
2) Number of events and number at risk: this may come from an observational study or control arm of an RCT

**Scale for relative effect**
An estimate of the relative effectiveness of the intervention is required for the primary outcome, along with an estimate of its uncertainty.
This can be expressed with a 95% confidence interval in terms of an odds ratio (binary), relative risk (binary), risk difference (binary), hazard ratio (survival) or mean difference (continuous).

**Minimum clinical difference (MCD)**
Specifying a MCD required to change clinical practice is one way to incorporate differences in costs, adverse events or other considerations which are not captured in the primary endpoint.
The MCD for a particular treatment is always defined relative to a current standard of care (baseline treatment).
The MCD is defined as the 'the minimum improvement in the primary endpoint which is required for the new treatment to be considered worthwhile'.
For example, a larger MCD may be required if the new treatment is more expensive than the current standard of care.
How the MCD is expressed differs between binary, continuous and survival endpoints.

- Binary: Absolute change in probability of primary endpoint e.g. an MCD of 0.02 implies that the probability of death must decrease by at least 2% for the new treatment to be worthwhile relative to the current standard of care.
- Continuous: Natural units of primary endpoint e.g. an MCD of 2 implies that the Mini-Mental State Examination (MMSE) must be 2 points higher for patients given the new treatment for it to be considered worthwhile relative to the current standard of care.
- Survival: Months of survival in the origin state e.g. an MCD of 1 implies that the new treatment must be associated with an expected increase in progression free survival relative to the current standard of care for it to be considered worthwhile.

**Treatment costs over patient time horizon**
For binary outcomes: here Treatment costs are assumed to be the same for all individuals treated; regardless of health outcomes (see below).
If treatment costs accrue over multiple years they should be discounted to present value.

**Treatment costs over patient time horizon if the primary outcome occurs / does not occur **
For binary outcomes: in some cases treatment costs will depend on the primary outcome, for example; intensive preventative treatment may be administered continuously until an event occurs (e.g. heart attack). Once the event has occurred the preventative treatment is halted and these treatment costs are no longer incurred.

**Treatment costs per month**
For continuous or survival outcomes: treatment costs incurred each month.

**Are individuals always treated until progression under this treatment?**
For survival outcomes individuals may be treated until progression or there may be a maximum duration of treatment.

#### Step 3: Proposed research

**Type of research**
The value of research can be calculated for either full research (e.g a randomised controlled trial (RCT)) or feasibility studies.
The inputs required for the analysis will depend on the type of study chosen.

- Full research: In contrast to feasibility studies which facilitate full research this type of research aims to address clinical questions directly. For example RCTs which aim to reduce uncertainty about relative effects.
- Feasibility study: If there are uncertainties about whether a full trial is possible, a short feasibility study can be carried out to assess the possibility of future full research.
If the feasibility study is successful, researchers have the option to carry out the follow up trial.

**Probability of feasibility research leading to follow-up study**
For feasibility studies: the motivation is that there is uncertainty about whether a full trial is possible. If the feasibility study shows that the full trial is not possible, the research budget spent on the feasibility study will have no impact on health outcomes. For this reason, the likelihood of a feasibility study leading to the full trial is an important determinant of its value.

**Expected duration of research**
Some assessment of the duration of time for the proposed research to be conducted and for the results of the research to report is required since the value of research declines the longer it takes to report.
This might be informed by an assessment of sample size, recruitment rates, or historical experience from conducting similar types of studies.
For feasibility studies: estimates of duration for both the feasibility study and the full trial are required.

**Costs of the research to funder**
These costs are the costs of research which are directly borne by the research funder.
For feasibility studies: estimates of research funder costs for both the feasibility study and the full trial are required.

**Costs of the research imposed on health system**
These costs are the costs of research which fall on the general health system.
Carrying out research consumes valuable resources from the general health care budget e.g. additional treatment costs and health professionals whose time commitments are moved away from general patient care and reallocated to research projects.
For feasibility studies: estimates of research health system costs for both the feasibility study and the full trial are required.

**Length of time for which the new evidence would be valuable**
The information generated by new research will not be valuable indefinitely because other changes occur over time. For example, over time new and more effective interventions become available which will eventually make those currently available obsolete.
This means that new information about effectiveness is only relevant for a specific amount of time. A judgement about the length of time that the evidence from the proposed research might be valuable is required to estimate the expected benefits over an appropriate time horizon.

**Discount rate**
Discounting should be used to reflect the fact that resources committed today could be invested at a real rate of return to free up more resources in the future.

**Incidence per annum**
An estimate of the number of individuals facing the uncertain choice between alternative interventions is required in order to establish the size of the benefits to the target population.

**Opportunity cost of health system expenditure**
Increasing treatment costs will be associated with health opportunity costs.
These are the improvement in health that would have been possible if any additional resources required had, instead, been made available for other health care activities.