Games Teach!

As the The Gamification of Learning and Instruction Blog Book Tour has been underway, another interesting discussion has been occurring in the learning blogosphere. It was sparked by Ruth Clark’s article “Why Games Don’t Teach.” The article kicked off a firestorm of debate, demands for proof and even name calling. Christy Tucker addresses the question in her post. There is a healthy discussion on the eLearning Guild’s LinkedIn group. In Guy Wallace’s post on the subject, he quotes Dr. Richard Clark talking about the “ideologically committed gamester.” (ok, so its mild name calling, but still….)

I wanted to look at the argument to see what has merit, what is supported by research and just what the overall state of the literature happens to be. Also, please know that I respect the work of Dr. Ruth Clark and actually think that aside from the title most of the article is on target, we don’t have a universal definition of “game”, research is sort of fragmented and a paper prototyping is always the way to go. Same with Dr. Richard Clark, he’s done and continues to do great work. So my issues are with the ideas that were put forth not with the individuals putting forth the ideas. So please, let’s not get into name calling…even mild name calling.

The Argument
So, So looking at the argument, Why Games Don’t Teach, we have several interesting positions:

• Games Don’t Teach
  
• All current reviews of adequately designed and peer-reviewed research have found NO learning or motivational benefits from games
  
• When learning benefits are found for games it stems from instructional methods that can be found in non-game contexts.

Dr. Richard Clark also argues on Guy Wallace’s posting that whenever a study comes out refuting the positive impact of games, the ideologically committed gamester’s claim “ that ‘the research did not study a game, I define ‘game’ differently’ or ‘I’d need to see the game that was tested, it was probably poorly designed’ or … it goes on and on.” Basically he is arguing that the game proponents are not arguing from a research base, instead they are arguing from perceived or imagined flaws in the game. The game was poorly designed so that’s why the result showed the game was no better than traditional learning; if the game was better designed the results would be different.

Let’s examine the three statements and try to argue from evidence and not conjecture.

The first argument is the easiest to refute. So we’ll start there.

One of the studies cited by Dr. Richard Clark against the use of games for learning is also cited by myself in my book is the meta-analysis (study of studies) by Robert Hays of the Naval Air Warfare center Training System Division who conducted a review of the literature in 2005. He examined 274 documents related to the design, use and evaluation of games. In his review of the literature, Hays eliminated 169 of the documents due to structural flaws, the use of opinion instead of statistics, and other research related flaws. Yet, he still concluded, and I quote directly from his research paper:

… games can provide effective learning for a variety of learners for several different tasks (e.g., math, attitudes, electronics, and economics)…(p.6) [and]… The second “claim,” that games enhance cognitive learning, continues to be supported. The research shows that people can learn from games.

So there are clearly evidence-based support for using games for instruction. Now, in the quote above, I have taken the statements out of context and that was on purpose, to prove my point. I will now include the entire context because one cannot evaluate research without looking at all the findings not just the ones that you happen to agree with.

Although research has shown that some games can provide effective learning for a variety of learners for several different tasks (e.g., math, attitudes, electronics, and economics), this does not tell us whether to use a game for our specific instructional task. We should not generalize from research on the effectiveness of one game in one learning area for one group of learners to all games in all learning areas for all learners.(P.6)

Of course nor should we generalize from one study about the ineffectiveness of one game in one learning area for one group of learners to all games.

The second “claim,” that games enhance cognitive learning, continues to be supported. The research shows that people can learn from games. However, the research does not indicate that games are superior to other instructional methods in all cases. Like any instructional activity, games should be chosen because they provide learners with interactive experiences that help them meet instructional objectives. (P. 46).

Key words here are “interactive experiences” and “chosen carefully.” He doesn’t say don’t use games, he says, choose wisely grasshopper (ok, I added the grasshopper part).

Additionally, there are several meta-analysis studies (studies of studies) that show that individuals can and do learn from games. Note, at this point, I am not saying games are better than traditional instruction, I am providing empirical, vetted research that strongly supports the claim that individuals can and do learn from instructional games.

• Ke (2009) conducted a meta-analysis study that found the effects of learning within games was positive in 52% of the studies. (examined 256 studies only 89 empirical studies were used in analysis)
  
• Stizmann (2011) found learning of declarative knowledge, procedural knowledge from simulation-games.(55 research reports including 39 published reports, 12 dissertations, 4 unpublished reports.)
  
• Wolfe (1997) found a game-based approach produced significant knowledge level increases. (7 studies)
  
• Vogel (2006) found cognitive gains observed in subjects utilizing interactive simulations or games. (examined 248 studies, only 32 of high enough quality to use in analysis)

So, the conclusion from the literature with a preponderance of evidence is that yes, games can teach! It is safe to say that evidence-based practice in the field of learning supports the use of games as one method of helping people to learn.

We Need Instructional Designers
Let’s look at one other thing Hays mentions before we move on.

The most poignant statements Hays makes in his meta-analysis, in my opinion, is his statement that we need instructional designers involved in the educational game design process. Again, I quote:

Although each company used interdisciplinary design teams to create their instructional games, none of the teams included instructional developers. The teams usually consisted of: 1) graphic artists, 2) program managers, and 3) programmers. Commenting on the avoidance of instructional designers, Squire stated, “Most game-based learning approaches do not employ that particular category of expert whatsoever” (p. 35). In most cases, the game designers fulfilled the role of instructional developer. It appears that the “instructional gaming” industry does not value the skills of instructional developers.

So let’s focus on getting instructional designers involved in designing games rather dismissing games saying they don’t teach. Again as Hays states “It appears that games can be of instructional value if they are well designed and targeted to meet specific instructional objectives.”

Second Argument
The second argument is that:

• All current reviews of adequately designed and peer-reviewed research have found NO learning or motivational benefits from games

What Does the Research Indicate?
Now since Hay’s paper was published in 2005, a number of other meta-analysis papers have been studied, I mentioned many of them earlier and discuss them more in-depth in my book but here are some interesting findings.

On was written by FengFeng Ke, an Assistant Professor, Department of Educational Psychology and Learning Systems at Florida State University. Ke chose to conduct a qualitative meta-analysis. So, instead of a statistical analysis of the outcomes of aggregated studies, Ke followed the replicable process of analyzing textual reports through a qualitative method to develop new interpretations. Ke analyzed 256 reports and eliminated 167 for various reasons. Ultimately, Ke aggregated results from eight-nine empirical studies. Ke classified the studies into five different areas to examine separate research questions. However, not all eighty-nine studies were applicable to all the research questions, so for some research questions the pool of studies is fewer than eighty-nine as is described below.

Ke’s findings (2008):

• The effects of computer-based games on learning are positive. When analyzing research results of the sixty-five out of the eighty-nine studies that specifically examined the effectives of computer-based games on learning, Ke found significant positive effects for computer-based games as compared with conventional instruction. Ke found a positive impact 52 percent of the time (34 studies). Found mixed results reported 26 percent of the time (17 studies). Mixed results were when an instructional game facilitated certain learning outcomes but not others. And found no difference between the games or conventional instruction reported 18 percent of the time (12 studies). In only one study was conventional instruction more effective than computer games. (p. 20).
• Instructional games seem to foster higher-order thinking such as planning and reasoning more than factual or verbal knowledge. This conclusion is drawn from studies that looked into cognitive learning outcomes in the areas of basic motor skills, descriptive knowledge, conceptual knowledge, problem solving, and general cognitive strategies. Ke indicates that this finding sustains the finding of Dempsey et al. (1996) from a previous meta-analysis. (p. 22)
• Affective learning outcomes, involving self efficacy, value (or attitudes toward subject content learning), affective feedback toward game use, and continuing motivation (or persistence), are present in many game studies. Generally, instructional computer games seem to facilitate motivation across different learner groups and learning situations. This finding is in agreement with Vogel et al.’s (2006) quantitative meta-analysis conclusion that the effect size of games vs. traditional teaching methods is highly reliable for attitude. Meaning that games facilitate motivation.

Now I can just hear it. Hey, that was a qualitative study so was it adequately designed and peer reviewed? A careful inspection of the chapter does reveal that it was well designed and that the process was rigorous and well documented. And, just to keep the record straight, the Hays meta-analysis was a technical paper and was not published in a peer reviewed journal. So we need to be careful when we arbitrarily declare that some technical papers are sound while a book chapter or a study in a non-peer reviewed journal may not be. So if we include Hays technical paper, we include Ke’s chapter.

But, ok, let’s now look at a peer-reviewed study that appears to be adequately designed and was published in a peer review article. Dr. Richard Clark mentions it in his rebuttal concerning the effectiveness of games so it must be high quality.

Dr. Traci Stizmann’s, who worked for ADL and is at the University of Colorado Denver and mentioned above conducted a meta-analysis with a review of 65 studies with over 6,000 participants. She came to interesting conclusions that are not in sync with the statement “All current reviews of adequately designed and peer-reviewed research have found NO learning or motivational benefits from games.” It might have to be revised to “All but one..” or if we include Ke’s study, “All but two…” The chink is forming. Here is the conclusion word-for-word from Sitzmann’s study, not a paraphrase, not an interpretation, not half the conclusion, the entire conclusion:

Simulation games have the potential to enhance the learning of work related knowledge and skills. Overall, declarative knowledge was 11% higher for trainees taught with simulation games than a comparison group; procedural knowledge was 14% higher; retention was 9% higher; and self-efficacy was 20% higher. Characteristics of simulation games and the instructional context were instrumental in determining the amount that trainees learned from simulation games relative to a comparison group. Specifically, learning from simulation games was maximized when trainees actively rather than passively learned work-related competencies during game play, trainees could choose to play as many times as desired, and simulation games were embedded in an instructional program rather than serving as stand-alone instruction.The ultimate goal for simulation game design teams is to exploit the motivational capacity of simulation games to enhance employees’ work-related skills. Thus, additional research is needed to examine the dynamic interplay of affective and cognitive processes during game play and, ultimately, their effect on training
transfer.

Now Stizmanns’s meta-analysis clearly supports the claim that simulation-games were more effective than traditional learning for conveying certain types of knowledge and lead to higher confidence levels (self-efficacy). Now, what is the argument from the anti-game establishment? Here is at least one peer-reviewed, reputable study that discounts the argument that games have no advantage over traditional instruction.

They say:(although, I did not read that in the statement above)

“Sitzmann’s analysis concludes that when learning benefits are found for games (ah, so reviews do show benefits from games) it stems from instructional methods that can be presented in non-game contexts.” In other words, if the traditional instruction that was used to do the comparison in the studies was not so poorly designed, the results would be different. It was just poorly designed traditional instruction or what they are really saying is that the results would be the same or show no statistical difference if the traditional instruction was designed more like (wait for it) a game.

Poor design is the same argument Dr. Richard Clark claims the game ideologues use against him when he reports “All current reviews of adequately designed and peer reviewed research have found NO learning or motivational benefits from games.” The pro-gamer faction say—the game was poorly designed and now the anti-gamer faction say–the traditional instruction was poorly designed.

The reality is the statement about “All the “good” research indicates and NO learning and no motivation” simply is not supported by Sitzmann’s (and Ke’s) research results. A review of dozens of studies with the elimination of dozens more because they are not adequately designed or peer reviewed clearly (and the inclusion of dissertations and non-published articles by Sitzmann) clearly indicate that games can have benefits over traditional instruction. Again, do they always have benefits, NO but can they in terms of learning and confidence, YES.

Can’t We All Just Get Along
So, theoretically when classroom instruction or e-learning is designed to be interactive, provide continuous feedback and reinforce learner behaviors it can be as effective as games but how many classrooms have you been in where that specific design is not implemented or how many e-learning modules employ those elements? Too few, whereas games naturally have interactivity, challenge and feedback. These are elements of almost every game.

These are not typically elements of instruction. Yes, when learning benefits are found for games it stems from instructional methods that can be found in non-game contexts. But the key phrase is “can be found” but typically, routinely and as a rule are NOT found in non-game contexts. Somehow instructional design has come to mean: a list of objectives, slides with bullets and a multiple choice test at the end. That is one of the differences between theoretical findings and what practitioners do on a daily basis.

So where does all this leave us?

Gamification(you knew this was coming)

First of all, making salacious claims like “Games Don’t Teach” or “All current reviews of adequately designed and peer-reviewed research have found NO learning or motivational benefits from games” isn’t only not supported by the research (which clearly shows learning occurs from games and games are motivational, oh and increase self-efficacy), it doesn’t help practitioners one bit. These types of statements stifle serious efforts to create engaging learning events.

Second, this is where gamification fits into the picture. Now before I go too much further. Gamification does not equal technology. The concepts of gamification can and frequently are applied to instruction without technology. These are concepts that make games exciting and engaging, such as challenge, interactivity and continuous feedback (not an arbitrary addition of points and rewards.)

Challenge, interactivity and continual feedback can be applied to a classroom exercises, a paper and pencil activity or used online. The design is universal while the delivery vehicle can change. It is not technology that makes a game a game—it’s the design, the inclusion of a challenge and interactivity that make a game a game.

Gamification, as a design sensibility, forces the designer to think about interactivity, to think about challenge and feedback. Instructional design as it is practiced in most places and taught by most institutions simply does not foster the ideas of creating challenging, interactive learning events that continually provide feedback. It just doesn’t happen. It is not standard practice, it could happen, but it doesn’t.

Most instruction is not interactive—it’s lists of bullets. Most instruction doesn’t start with a challenge—it starts with a list of learning objectives. Most instruction doesn’t have continual feedback—only a score on a multiple choice test at the end.

Gamification, on the other hand, starts with the ideas of interactivity, challenge, and continual feedback. It fosters thoughts of engagement. The “myth” of gamification is that is encourages all the elements of instruction required for learning. So rather than pit games vs. non-games as a paradigm, let’s be smarter than that and look at the elements of learning that are effective and apply those to our instruction whether its a game, simulation or a standard classroom lecture.

And a good place to see how those elements are expertly applied is to look at games which have the wonderfully educational elements of challenge, interactivity and continual feedback down to a science. Elements we desperately need to add to the learning events we design and delivery every day.

You know, if you take an online game and remove the challenge, interactivity and continual feedback all you have left is a typical e-learning module.

Conclusion
Games, in all forms, electronic, paper and pencil and even just person to person have the potential to be powerful teachers as is clearly indicated by the preponderance of evidence not just the results of one study. There are peer-reviewed articles in reputable journals (Sitzmann) that clearly support the conclusion that games can have higher knowledge and motivational levels than traditional instruction. When non-gamers read those results they say “the traditional instruction was poorly designed”. It seems the shoe is now on the other foot.

What do you think?

Read more about this subject in my latest book.