Embedding stakeholder preferences in setting priorities for health research: Using a discrete choice experiment to develop a multi-criteria tool for evaluating research proposals

We determined weights for a multi-criteria tool for assessing the relative merits of clinicaltrial research proposals, and investigated whether the weights vary across relevant stakeholder groups. A cross-sectional, adaptive discrete choice experiment using 1000minds online software was administered to consumers, researchers and funders affiliated with the Australian Clinical Trials Alliance (ACTA). We identified weights for four criteria - Appropriateness, Significance, Relevance, Feasibility - and their levels, representing their relative importance, so that research proposals can be scored between 0% (nil or very low merit) and 100% (very high merit). From 220 complete survey responses, the most important criterion was Appropriateness (adjusted for differences between stakeholder groups, mean weight 28.9%) and the least important was Feasibility (adjusted mean weight 19.5%). Consumers tended to weight Relevance more highly (2.7% points difference) and Feasibility less highly (3.1% points difference) than researchers. The research or grant writing experience of researchers or consumers was not associated with the weights. A multi-criteria tool for evaluating research proposals that reflects stakeholders' preferences was created. The tool can be used to assess the relative merits of clinical trial research proposals and rank them, to help identify the best proposals for funding.