With this article, we offer you a sneak preview of our major findings from a systematic review on education for a responsible conduct of research (RCR). We have just submitted this article to a scientific journal for peer review, so this blog post constitutes a pre-publication, and we ask you to wait for the published study before making references to this work.

From 2019 to 2020, we reviewed scientific studies that investigated the effectiveness of different approaches to educate students, citizens, or researchers in a responsible conduct of research. Overall, we screened 1.548 abstracts, selecting 84 articles for full review. In the end, 30 studies met our standards for inclusion: They investigated the extent to which learners’ knowledge, attitudes, or competences related to RCR improved due to an educational intervention, whereby pretests and/or a control group were utilized to measure the course outcome. We excluded studies that only assessed the degree to which learners liked a course or believed it was effective, because enjoyment does not necessarily equate to learning and learners may err about the impact that a course has had on their development (Mayer, 2014). In order for us to be able to apply statistical methods that are typical of meta-analyses, we also excluded studies that did not allow us to calculate an effect size for the effectiveness of an intervention (the main problem was that some studies only reported mean values but no standard deviations).

Unlike prior meta-analyses on “ethics training in the sciences” (Antes, Murphy, Waples, Mumford, Brown, Connelly, & Devenport, 2009; Watts, Medeiros, Mulhearn, Steele, Connelly, & Mumford, 2017), an explicit assumption that drove our investigation was that different teaching approaches may be needed to achieve different kinds of learning outcomes related to RCR. For example, we expected that individual learning activities would be most effective when it comes to understanding, remembering, and recalling concepts, facts, and procedures related to RCR. Contrarily, we assumed that the abilities to notice and solve ethical problems would benefit from a combination of individual and group-based learning. Overall, we were able to test 10 hypotheses about effective RCR courses. To do so, we drew on 75 effect sizes that we calculated for five different types of learning outcomes (knowledge of RCR, moral judgment, moral sensitivity, moral attitudes, and moral behavior).

The effect sizes between the studies in our sample varied a lot. The “smallest” effect we found had a Hedges’ g of –0.56, meaning that learners’ measured ability was significantly worse after the course than beforehand. The largest effect we found was exceptionally large at g = 1.7 (following Cohen, 1988, standardized effect sizes of 0.8 and higher demark large effects). Average effect sizes were smallest for moral judgment and sensitivity, suggesting that these outcomes were most difficult to achieve.

So far, meta-analyses on ethics training have mainly reported the results of so-called moderator analyses. Moderator analyses look at the average effect sizes of all studies that reported a certain characteristic, e.g., which used blended learning (the combination of face-to-face and remote learning activities). Larger effect sizes for one course characteristic than for others imply that this characteristic may be beneficial. In our study, we went beyond the use of moderator analyses and worked with so-called meta-regression analyses. Meta-regressions draw on the same information as moderator analyses. However, they calculate the degree to which different variables (e.g., course characteristics) tend to lead to larger or smaller effect sizes. Unlike moderator analyses, which only look at the impact of a single variable, meta-regressions take multiple variables into account simultaneously. Since variables are often correlated, e.g., courses that use case discussions usually also employ group-based learning, the simultaneous analysis of several variables allows us to detect the most probable source of larger/smaller effect sizes.

Using meta-regression, we investigated which course characteristics had the greatest impact on the development of moral judgment and the development of RCR-related knowledge, as well as on the achievement of all five learning outcomes overall. Distinct analyses on the development of moral attitudes, sensitivity, and behavior were not possible due to the small number of studies that had assessed these learning outcomes.

In line with our expectations, different goals of RCR education benefited from different teaching approaches. This was most obvious when looking at the coverage of codes of conduct as part of RCR education. Applying codes of conduct to ethical cases – compared to not covering them at all, or only covering them superficially – had a significant positive impact on learners’ knowledge acquisition. However, not covering codes of tended to be a feature of the most effective courses overall. An explanation of this seemingly paradoxical finding is that attitudinal and behavioral learning is hampered through reactance when people are expected to adopt evaluations that they have not concluded themselves (Worchel & Brehm, 1971). Telling people what is good and bad, instead of engaging them in figuring it out on their own, often leads to a defensive “I don’t care” attitude, which ultimately aims at preserving learners’ sense of autonomy. At the same time, it is logical that people, whose knowledge of ethical codes is tested, will have learned more if they actually applied these codes in the analysis of cases, instead of merely reading them.

Our second main finding was so strong that we were surprised by its intensity. In coding all of the 30 studies in terms of their teaching approaches, we classified each course as ‘theoretical’, ‘deliberative’ or ‘experiential’. Theoretical courses were the ones that avoided the discussion of concrete cases. Deliberative courses used a classical case-discussion approach, which focused on understanding the ethical dimensions of a problem and weighing the goodness of different alternatives. Experiential courses worked with concrete cases, as well, but engaged learners in imagining how they would deal with such a situation personally, e.g., through role play, or through a discussion of the emotional dimensions of ethical decision-making. Speaking metaphorically, we could say that the main difference between the deliberative and experiential courses was that the deliberative courses emphasize “cool” and “distanced” reasoning, while the experiential courses emphasize a “hot”, “hands-on” reflection on action.

We had expected that deliberative courses would yield larger effects than theoretical courses, on average, and that experiential courses would be even more effective – at least for attitudinal learning, the development of moral judgment, sensitivity, and behavior (we did not expect an advantage for the development of knowledge). We did not, however, expect that this ‘course emphasis’ variable would be the strongest predictor of courses’ effectiveness though. Experiential learning was by far more effective in promoting all types of learning outcomes than the theoretical and deliberative approaches were. Moreover, the deliberative approaches – often referred to as the “gold standard” of ethics education (e.g., in the medical domain) – were not significantly more effective than the theoretical approaches, overall. Theoretical approaches only appeared to be significantly less effective than deliberative approaches in the development of knowledge related to RCR.

When the course emphasis was part of the regression equation, hardly anything else contributed to a course’s effectiveness. For instance, none of more than ten teaching methods that we considered (e.g., lectures, role play) had a significant effect, and whether a course combined individual and group-based learning only mattered in the case of knowledge acquisition (where pure individual learning was best). One factor that played a role though, was the number of teaching methods employed: The more methods were employed, the lower a course’s effectiveness, on average.

Based on these findings, a couple of recommendations seem clear: First, RCR education should make use of experiential learning, and engage learners in an active reflection of how they would personally deal with challenging situations pertaining to research integrity. Useful approaches could include the discussion of engaging stories (e.g., the interactive novel The Brewsters by Rozmus and colleagues 2015), the use of role play and simulations (e.g., the digital game Academical by Melcer and colleagues, 2020), or techniques like coping-modeling/problem-solving, where learners watch a video of a difficult situation and discuss different courses of action (cf. Simola, 2010). Secondly, teachers should be careful about the way in which they introduce codes of conduct for RCR in their courses. Fruitful approaches could be to first win learners’ support for rules and regulations pertaining to RCR (e.g., through experiential learning) before concrete codes are presented. Moreover, engaging learners in a critical discussion of codes of conduct may help them to make sense of them and learn to appreciate their value. Third, teachers should not rely on the belief that learners’ acquisition of RCR-related knowledge will automatically equip them with higher-order competences like the abilities to notice or solve ethical problems. Our findings support the contention that we are dealing with distinct, independent learning outcomes. This brings us to our final recommendation, namely, to choose few methods of instruction, and to do so wisely.