Abstracts of Three Meta-Analysis Studies of Serious Games

A meta-analysis is a study of studies. It is a way of aggregating research data to make informed conclusions rather than relying on the data of one study. Here are three abstracts from three different studies.Games for learning do have merit but they are not a blanket panacea. Like any good instructional method, they have to be used intelligently and in conjunction with other instructional elements to be effective for learning.

Abstract One from
Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013, February
4). A Meta-Analysis of the Cognitive and Motivational Effects of Serious Games. Journal of
Educational Psychology. Advance online publication. doi: 10.1037/a0031311

It is assumed that serious games influences learning in 2 ways, by changing cognitive processes and
by affecting motivation. However, until now research has shown little evidence for these assumptions.
We used meta-analytic techniques to investigate whether serious games are more effective in
terms of learning and more motivating than conventional instruction methods (learning: k = 77, N
= 5,547; motivation: k = 31, N = 2,216). Consistent with our hypotheses, serious games were found
to be more effective in terms of learning (d = 0.29, p < .01) and retention (d = 0.36, p < .01), but they were not more motivating (d = 0.26, p <.05) than conventional instruction methods. Additional moderator analyses on the learning effects revealed that learners in serious games learned more, relative to those taught with conventional instruction methods, when the game was supplemented with other instruction methods, when multiple training sessions were involved, and when players worked in groups.

Abstract Two from
Stizmann, T. (2011) A Meta-Analytic Examination of the Instructional Effectiveness of Computer-based Simulation Games. Personnel Pyschology, 64, pp. 489-528.

Interactive cognitive complexity theory suggests that simulation games are more effective than other instructional methods because they simultaneously engage trainees’ affective and cognitive processes (Tennyson & Jorczak, 2008). Meta-analytic techniques were used to examine the instructional effectiveness of computer-based simulation games relative to a comparison group (k=65, N=6,476). Consistent with theory, post training self- efficacy was 20% higher, declarative knowledge was 11% higher, procedural knowledge was 14% higher, and retention was 9% higher for trainees taught with simulation games, relative to a comparison group. However, the results provide strong evidence of publication bias in simulation games research. Characteristics of simulation games and the instructional context also moderated the effectiveness of simulation games. Trainees learned more, relative to a comparison group, when simulation games conveyed course material actively rather than passively, trainees could access the simulation game as many times as desired, and the simulation game was a supplement to other instructional methods rather than stand-alone instruction. However, trainees learned less from simulation games than comparison instructional methods when the instruction the comparison group received as a substitute for the simulation game actively engaged them in the learning experience.

Tennyson RD, Jorczak RL. (2008). A conceptual framework for the empirical study of instructional games. In O’Neil HF, Perez RS (Eds.), Computer games and team and individual learning (pp. 39–54). Oxford, UK: Elsevier.

Abstract Three from
Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., and Boyle, J. M. (2012)A systematic literature review of empirical evidence on computer games and serious games. Computers & Education 59, pp. 661–686.

This paper examines the literature on computer games and serious games in regard to the potential positive impacts of gaming on users aged 14 years or above, especially with respect to learning, skill enhancement and engagement. Search terms identified 129 papers reporting empirical evidence about the impacts and outcomes of computer games and serious games with respect to learning and engagement and a multidimensional approach to categorizing games was developed. The findings revealed that playing computer games is linked to a range of perceptual, cognitive, behavioural, affective and motivational impacts and outcomes. The most frequently occurring outcomes and impacts were knowledge acquisition/content understanding and affective and motivational outcomes. The range of indicators and measures used in the included papers are discussed, together with methodological limitations and recommendations for further work in this area.

Results
So what do we know from these three abstracts?

• Serious games were found to be more effective in terms of learning and retention than conventional instruction methods.
  
• Mixed results concerning if learning games are more motivating or not than traditional instruction.
  
• Games should be supplemented with other instruction methods.
  
• Games should be played in multiple training sessions.
  
• Games should be played in group .
  
• Games appear to increase learner confidence (self-efficacy).
  
• Games help increase declarative knowledge, procedural knowledge and retention over traditional non-interactive training.
  
• There is strong evidence of publication bias in games research.
  
• Trainees learned more, relative to a comparison group, when simulation games conveyed course material actively rather than passively.
  
• Learning occurred when learners could access the game as many times as desired.
  
• As above, the game was more effective when it was a supplement to other instructional methods rather than stand-alone instruction.
  
• Learners learned less from simulation games than comparison instructional methods when the instruction the comparison group received as a substitute for the game actively engaged them in the learning experience.(so activity, not game elements seems to increase the learning).
  
• The most frequently occurring outcomes and impacts of games for learning were knowledge acquisition/content understanding and affective and motivational outcomes.