PROmoting integrity in the use of RESearch results

Good, effective policymaking—that produces outcomes that benefit, protect and sustain society, communities, groups and individuals—depends upon robust, rigorous, and interpretable research. If research is flawed by lack of integrity and by being unethically conducted, it is of no use to policymakers. Decision takers and policymakers seek evidence to support their work from the range of expertise on offer. Any errors, fraud or corrupt practices by researchers in these fields can lead to serious damage to society’s social, economic and cultural structure, as well as having deleterious impacts on the physical environment.

As a corollary, sound, reliable and transparent research, divorced from political ideology and undeclared vested interests, produces robust evidence to benefit social wellbeing and societal progress. Researchers, reviewers, regulators and policymakers can be tested by the diverse and sometimes conflicting codes of ethical practice and regulations, by the complexities of data protection legislation, by inconsistencies in the applications of regulations and by the professional and practical pressures of conducting research in the diversity of non-medical fields.

Guidance framework

The PRO-RES project has produced a guidance framework that encourages policymakers and their advisers to seek evidence for their decisions from research that has been conducted ethically, responsibly and with integrity.

The framework contains advice and guidance on how one ‘ought’ to behave in producing ethical evidence. The framework constructed by the PRO- RES project is based upon principles of normative ethics which refers to the grounds of meaning or acceptance of decisions about why certain behaviour is right or wrong. This is about the way one ought to behave as a researcher. Normative principles have been established over time from considering moral choices at an abstract level in ethical theorising, from the codes of behaviour established by professional institutions and from the observation in practice of what happens when research is done for the ‘wrong’ reasons or in ‘incorrect’ ways as well as observing the benefits from doing research the ‘right’ way. Thus theorising about ethics can never be divorced from the application of principles in practice.

To answer the “Who is this for?” question: it is for policymakers keen to use ethical evidence; it is for policy advisers seeking to offer advice based on responsible sources; it is for researchers and their funders wishing to make sure that policies will be based upon their ethically produced evidence; it is for think tanks wishing to enhance their legitimacy by demonstrating that their reports have been produced with integrity and it is also available for citizens to make their own assessment about the evidential sources of the policies that directly affect them.

The first pillar of the framework is the STEP (Scientific, Trustworthy, and Ethical evidence for Policy) ACCORD, a statement that lays out the principles for ethical research, which we hope all stakeholders can sign up to and endorse. The statement is for all who are concerned to ensure policies are based upon ethical evidence.

As signatories to the accord:

We recognise that an underpinning by high-quality research, analysis and evidence, including policy appraisals and evaluations, is a pre-condition for evidence-based policy-/decision-making, and hence rational policy actions and effective outcomes.
As individuals and institutions involved in commissioning, funding, sponsoring or conducting research, collecting or using evidence for policymaking, we aim to be as transparent as possible on how the high quality of that evidence is assured and will flag up any potential conflicts of interest.
We agree that to a reasonable degree, the independence and integrity of individuals responsible for the conducting and/or gathering of research evidence and its use in policymaking must be respected and supported in ways that ensure the evidence they produce is neither biased nor misleading.
We will communicate, employ and/or apply only high-quality evidence, research or enquiry, in other words, evidence that has been undertaken, gathered, collated and analysed using sound, robust and ethical methods appropriate to the task.
We will ensure that the commissioning, funding, management, conduct, dissemination and governance of research meet high standards of ethics and integrity.

Our rationale in constructing this accord is that most of the codes and guidelines for research ethics and integrity are constructed on the basis of a normative prescription or a ‘duty-based’ as opposed to a ‘rights-based’ morality. Although one could conceive of an alternate guidance structure based on rights, it would be much harder to apply since rights are more difficult to define and operationalise and, in practice, are more inclined to conflict with each other. It would certainly be confusing (as it is with the current mix of rights and duties in European law) to try to reconcile a ‘rights’ approach with ‘duties’ under the law, to ethics and to the research profession. The comparison of rights-based and duty-based moralities brings out the problem of all ethical principles being in tension. The writers of codes and guidelines are constantly trying to reconcile such tensions—any right to be informed will always be contradicted by a right for data not to be disclosed. It would only take one respondent in many datasets to seek anonymity for the rest to have to be anonymised—thus restricting its availability—even perhaps for tests of validity and reliability to be conducted by other researchers. The PRO-RES approach was always based on resourcing reflective practice rather than on formal bureaucratic compliance.

And this approach was vindicated in all the work done with stakeholders. The framework was constructed by drawing upon the views and ideas of the full range of key stakeholders in an iterative process. It has built upon previous foundational research ethics codes, guidelines, and frameworks to assess what elements of these foundations have ‘worked’ in influencing and informing policymaking in the past.

Experience suggests that the more regulatory a code, the more malpractice is encouraged if the institutional and/or infrastructural pressures not to behave ‘well’ remain unaddressed. This represents a pragmatically-oriented ‘virtue ethics’ approach; one that encourages and rewards responsible conduct in researchers and their employing and/or funding institutions. This, too, was endorsed in our interactions with the full range of stakeholders. It is always possible to construct prescriptive codes if the power to apply sanctions, such as restricting access to funds or delegitimising the ability to offer research exists.

Assessing the ethical quality of research

The PRO-RES project has also created a toolbox for assessing the ethical quality of research evidence that can be used to check that an ‘evidence generating organisation’ has acted with integrity. This includes a set of key ethical questions to be asked about any research output, scientific finding, evidence-based policy advice or similar and by seeking the answers to these questions, it is possible to make a judgement about how ethically the research/analysis was conducted and if the researchers/analysts/advisers behaved with integrity.

Anyone wishing to ‘test’ a researcher and their work for its integrity should be able to ask these questions of them. The questions are applicable to all forms of enquiry seeking to gather data and analyse it for evidential purposes. The first tool in the PRO-RES toolbox (https://prores-project.eu/tool-no-1/) thus concerns the following questions:

WHO did the research/conducted the enquiry/provided analysis or advice?
HOW did they do the research, or what did they base their advice and analysis on?
WHOM/WHAT was being studied?
WHY was the research/analysis conducted?
WHEN/WHERE was the research/ analysis conducted?
Was the research REVIEWED in advance for quality considerations?
What were the OUTCOMES of the research/analysis?

Each question above is broken down into several questions that need to be answered to assess the ethical quality of research evidence.

It is important to note that, given the range of evidence employed in policymaking, we are adopting a very broaddefinitionof‘research’ thatincludes all forms of data gathering intended to supply evidence for policymaking. As a result, the agencies gathering the data might include academic researchers, think tanks, lobbying agencies, PR consultants, advocacy agencies, civil society organisations, early adopters/ influencers (bloggers, etc.); these criteria do not ‘rule out’ novice researchers, citizen scientists, members of the public, journalists etc. All these ‘agencies’ could be regarded as ‘evidence generating organisations’ (EGOs).

Important conclusions regarding evidence-based policy advice

There is no explicit requirement for only experienced researchers to be treated as ‘legitimate’. The key is to be transparent about exactly who the researcher/agency is and who they are working for, even if it is for themselves. It is to be expected that researcher CVs/résumés would be supplied together with any agency track records, details on the background of the RPO/EGO/agency and its main funding sources, which could be large corporations with heavily vested commercial interests or crowd funding schemes in which the interests might be more diverse. Mission statements or adherence to codes, guidelines and/or professional association memberships would be appropriate here. A key question for the evidence-gathering agency would be how does it fund itself? Does it have a diversity of funding, or is it dependent on a particular stakeholder and with what contractual commitments?

There is no implicit judgement of the ‘ethical quality’ of the variety of methods that can be employed. What matters is, again, the transparency of those conducting the research and their

offering of clear justifications/rationale for any methods used. Thus covert research, deception, community/ societal engagement, social engineering etc. are not to be regarded as inherently unethical. The judgement of whether they are or not might depend upon the context in which they are used and whether a policymaker/adviser considers evidence derived from a particular method is justifiable.

Neither is there any implication that only primary research is of evidential value. All forms of secondary data analysis can be subjected to these questions: from meta-analyses of controlled experimental studies to simple frequency counts of questionnaire responses. The ‘validity’ of primary research data depends upon the rigour of the research design and its accurate execution; the validity of most forms of secondary data analysis depends upon access to/availability of raw source data. Even documentary or archival analyses are valid to be tested against accurate use of source materials.

Motive and intent are key ethical issues. They go to why the research was conducted in the first place and what outcomes were hoped for, and by whom. Impacts could be environmental, social, psychological, political etc. Hence the question of who commissioned and funded the research/enquiry is doubly important—details on the funding agency is key to full transparency. Most ethical judgements rely upon a full understanding of the context in which the action under consideration occurred—the place and the time. This requires a comprehensive understanding of place and time: geographical, institutional, organisational etc. and diurnal, annual, chronological, historical and so on. Thus there are wide variations between a laboratory site, urban settings entailing risk and threats, libraries, and high- and low-resource countries. Laboratories can vary in licensing levels, while field sites vary in the kinds of permissions required. Historical archival research varies considerably in terms of ethical risk from the study of more contemporary documentation but engaging in historical enquiry may still entail risks to the present in terms of societal or communal stigmatisation and/or reputation. For example, knowledge of how and why a particular organisation was established may ‘taint’ its current reputation.

Article summary

PROJECT NAME

PRO-RES

PROJECT SUMMARY

The overall goal of the PRO-RES project is to build a research ethics and integrity framework devised cooperatively with, and seen as acceptable by, the full range of relevant stakeholders and similar to Oviedo/ Helsinki. This will be a normative framework for evidence-based policy originating from cutting edge research responses to ethical challenges.

PROJECT LEAD

Project PI – Ron Iphofen (AcSS) has been working in this field since the late 1990s and has national and international recognition as an expert on social research ethics and governance and in maintaining professional standards in research. He has published widely on the topic— with book chapters, guidance documents, training packages, textbooks and articles in the professional press. He founded and edited a journal in social gerontology for fifteen years.

Project COO – Emmanuel Detsis (ESF) is a certified project manager and has been responsible for several of ESF’s European framework projects and many European and international studies of the foundation.

PROJECT PARTNERS

European Science Foundation, France Academy of Social Sciences, United Kingdom European Alliance For Social Sciences and Humanities, Belgium.

Tartu Ulikool, Estonia.

Hrvatsko Katolicko Sveuciliste, Croatia Innovation In Research and Engineering Solutions, Belgium.

Institute Des Hautes Etudes Economiques et Commerciales Association, France Conoscenza E Innovazione Societa Aresponsabilita Limitata Semplificata, Italy Steinbeis 2i Gmbh, Germany.

Hellenic Centre For Marine Research, Greece Consiglio Nazionale Delle Ricerche, Italy European Policy Centre, Belgium.