DESIGN AND EVALUATION FRAMEWORK FOR RELEVANT CHEMISTRY-RELATED EDUCATIONAL CARD AND BOARD GAMES

During the 21st century, new generations of both commercial board games and digital games have appeared, and in their wake, game-based learning has been extensively studied in recent years. There has also been some research on and development of card and board games for learning chemistry. Most of this research has been conducted in the field of regular and educational digital games. Many different classification, evaluation and assessment frameworks and tools are available for digital games. Few have been developed for card or board games, but many general rules for good educational games have been offered in research articles. Based on a literature review, a novel design and evaluation framework for card and board games for chemistry education on the lower secondary level has been developed. The aim of this framework is to help designers and teachers to design new educational card and board games, to support them in evaluating the viability of already existing chemistry-related educational games and instructing them in supporting student learning with a game.


Introduction
Game-based learning is one of the topical teaching approaches studied around the world.
There is a need for designing more relevant learning environments to engage students, especially in chemistry.Game-based learning has been extensively studied in recent years.
The research has focused especially on digital games, but there has also been some research on and development of educational card and board games related to chemistry (e.g., Tüysüz, 2009;Kavak, 2012;Rastegarpour & Poopak, 2012;Bayir, 2014).During the 21 st century, new generations of both commercial board games and digital games have appeared (Keskitalo, 2010).

Definition of Educational Game
The term game can be defined as "a system in which players engage in an artificial conflict, defined by rules that results in a quantifiable outcome" (Salen & Zimmerman, 2003).An educational game is a game with a certain didactical meaning that aims to support, improve and advance the learning process (Dondi & Moretti, 2007).

Educational Games as a Teaching Method in General and in Chemistry in Particular
As a teaching method, educational games employ the idea of a student participating actively in his or her own learning.Active participation in the learning process enables deeper conceptual understanding.(Lujan & DiCarlo, 2006;Tüysüz, 2009) Educational games are mentioned as one motivating teaching method for group work on the lower secondary level in the new Finnish National Curriculum Framework 2016 (OPS 2016(OPS , 2015)).
There are many opportunities and challenges in using educational games in chemistry education.In general, pre-, in-and post-game guidance and instructions given by a teacher have a positive effect on students' motivation and learning performance, ensuring a positive attitude towards the topic of learning (Casbergue & Kieff, 1998;Ke, 2009).Using educational games has been found to support especially students who are lower achievers, but not without a teacher's guidance (Ke, 2009, 21-22).If a teacher is not involved, the students just learn how to play and are not immersed in the learning topics (Ke, 2009).
Challenges and conflicts as well as rewards and feedback during play give alternating feelings of frustration and pleasure to players.These alternating feelings are one of the engaging elements of games and playing.(e.g., Tüysüz, 2009;Annetta, 2010) A co-operative goal structure, as opposed to competitive or individualistic goals, has been found to enhance positive attitudes towards the learning content.Boys have been found to engage better in co-operational and problem-based playing than girls.(Ke, 2008) According to Ke (2009), competition is the difference between game and simulation, but in-game competition should not necessarily be against other players, but towards the goal.Playing a computer game based on uncertainty improves the factual memorisation and emotional engagement of students aged 10-16.Purportedly, this is also effective in non-digital games and other areas of learning.(Howard-Jones & Demetriou, 2009) With adults, it has been suggested that uncertainty and changes in game difficulty engage the player in both the learning and the game (Chanel, Rebetez, Bétrancourt, & Pun, 2008;Tüysüz, 2009).The effects of uncertainty or different difficulty levels in chemistry-related educational card or board games have not yet been studied.
Both computer and card games have been demonstrated to have a positive effect on chemistry learning and attitudes towards chemistry (Sherman & Sherman, 1980;Kavak, 2012;Rastegarpour & Poopak, 2012).To support the idea of chemistry learning with computer-based games, statistically significant positive effects (n=176, p > 0.05) were found in chemistry achievement and attitudes towards chemistry among pre-service primary-level teachers.However, no significant difference in their metacognitive levels was observed.(Tüysüz, 2009) Bayir (2014) has studied three different chemistry-related card and board games with 250 students (grades 9-12) and 30 in-service and pre-service teachers.The themes in the games were compounds, the elements and the periodic table.The results of Bayir's research indicate that using these card and board games was beneficial to both the teachers and the students.According to the teachers, the three main benefits of using the games were facilitating the learning of the main concepts, teaching the concepts in an interesting and enjoyable way and enabling the students to understand the relationships between the concepts.The research suggests that games spark students´ interest in chemistry and facilitate the learning of the chemistry concepts in question.
There is also evidence against games as effective learning tools (e.g., Randel, Morris, Wetzel, & Whitehill, 1992;Emes, 1997).The conclusions of these studies propose that games are effective only for certain learning content and in situations where the objectives are clearly defined.

Evaluation Criteria for Educational Games
Different criteria for evaluating digital games exist, including, for example, the flow of the Design research in science education is an effective research method that is connected to real teaching situations and real problems of science education.Design research has three main parts: problem analysis, design procedure and design solution.There might be some variation and the parts may be repeated during the research process (Edelson, 2002), but the process always starts with problem analysis to ensure that the design approach is based on a real life problem and that it encompasses a theoretical framework (Pernaa, 2013).

Methods
The main aim is to develop a design and evaluation framework for educational card and board games related to chemistry learning on the lower secondary level.This framework is intended to support both teaching and learning and help to develop new, high-quality educational games.The objective of this research project is to answer the main research question: What kind of framework for designing and evaluating games for chemistry education would support the creation of games for better and more relevant teaching and learning?

Theoretical Problem Analysis
The problem analysis for this design research project was theoretical.It was executed as a literature review.This systematic literature review was conducted in accordance with the criteria and models for integrating and systematic data collection (Salminen, 2011;Koskinen, Kangas, & Krokfors, 2014).
The phases of the literature review were: After the searching phase, data extraction was executed with a directed content analysis.
Directed content analysis is one of the three different approaches to qualitative content analysis in which a theoretical framework serves as a basis for initial codes (Hsieh & Shannon, 2005).
The last phase of the literature review was data synthesis, where codes from the data extraction phase were clustered an evaluated.During the evaluation, the codes which were found to be irrelevant or impossible to take into account in card and board games were discarded.

Design Procedure
After clustering, a design and evaluation framework for chemistry-related educational card and board games on the lower secondary level was developed by forming design and evaluation criteria to give the framework a structure that teachers could easily understand.
The section on lower secondary level chemistry in the Finnish National Curriculum Framework 2016 (OPS 2016(OPS , 2015) was used as an example for incorporating curricular chemistry aspects into the framework.The main topics from the curriculum were gathered and included into the design and evaluation framework.The structure of the framework was modified and additional subclasses and details were added in order to make the structure and content of the framework as good as possible.
In accordance with the rules for qualitative design research presented by the Design-Based Research Collective (2003), the design solution produced in this study is a guiding model which is transferable to different fields of teaching.

Results of the Literature Review
The literature review was conducted to broaden the theoretical framework and determine the theory-driven central concepts in the assessment and evaluation of both games and educational games.These central concepts are listed in Table 1.After the synthesis, some of the central concepts were excluded because they were mainly connected to digital games and much easier to include in the context of video games, for example, than in card or board games.The excluded concepts were: navigation, number of attempts, identity or customisable characters and narrativity or richness of storyline.Selfesteem was also excluded, even though it could probably be evaluated with a post-game questionnaire or some sort of self-assessment tool.

Design Solution
Based on the results of both the theoretical problem analysis (Table 1) and the design process (Chapter 2.2), a novel design and evaluation framework was developed for card and board games related to chemistry learning on the lower secondary level (Figure 1).This framework includes classes and subclasses which should be included in a good chemistry related educational game.The tool also helps teachers to assess and support a playing session.The detailed subclasses may be used to guide game design and evaluation.
The best research-based alternatives are in bold.

Discussion and conclusions
Results of the literature review (Table 1) clearly demonstrate a consensus among developers of digital educational games about what the central and important concepts and contents in games are.These results support and add to both the theoretical framework about games as a teaching method and the main lower secondary level chemistry content of the Finnish National Curriculum Framework 2016.Both the curriculum and the developers of educational games are focused on sociality and co-operation; evaluation, assessment and feedback; problem solving and challenges; and real-life connections.All of these concepts are also connected to the socio-constructivist concept of learning and to supporting guidance and formative assessment in helping students learn better.
The design solution (Figure 1) is a framework which makes it easy for a teacher to evaluate the educational quality of a certain card or board game.The framework can also be used for games in other formats.The curriculum content in the framework can be adjusted to support curricula in different countries.
The next point of study will be how new games can be created by using the developed design and evaluation framework.


Defining the objective: To review the relevant literature relating to the classification and evaluation of educational games. Defining criteria for articles: Article includes a tool, framework or other relevant, research-based information for classifying or evaluating games or educational games; article does not include only simulations or commercial games; article presents general information and does not focus only on one game; exceptions to the previous rule are articles about chemistry-related games; entire article is available without extra cost; sources are from 2000 to 2014. Defining key words for articles: first search with games and classification, second search with games and evaluation, third search with games and quality assessment, fourth search with educational games and quality assessment. Defining data sources: Nelli Search Portal (including databases, journals and ejournal sources in the Helsinki University Library and the National Library of Finland) Literature search: (see below)  Data Extraction: a directed content analysis with coding (see below and Chapter 3)  Data Synthesis: a directed content analysis with clustering (see below and Chapter 3) A total of 11 articles from the four searches were accepted into the data extraction phase of this literature review.The first search with the key words games and classification yielded 132219 results.When sorted by relevance, none of the first ten articles were considered relevant for this research.These articles only classified different games by violence or cheating, for example, or their content concerned something else than playable games (e.g., weakly acyclic games).The second search with the key words games and evaluation yielded 339307 results, a number which was absolutely too high to wade through.The decision was made to only include the top 90 articles when sorted by relevance: 6 of these 90 were considered to comply with the criteria set for articles and accepted.This search was then adjusted to only include articles from the year 2014, and three more articles were accepted.The third search with the key words games and quality assessment resulted in 126075 articles, the relevant top 30 of which were waded through and two more articles were accepted.The fourth and last search with the key words educational games and quality assessment yielded 51080 results.Two more articles were accepted from the top 30 of articles sorted by relevance.Additional 15 articles were chosen to complement the data search described in the previous paragraph.These articles were either referenced in the searched articles or they DESIGN AND EVALUATION FRAMEWORK FOR EDUCATIONAL GAMES addressed developing, evaluating or researching chemistry-related educational card and board games.

Figure 1 .
Figure 1.The design and evaluation framework for chemistry-related educational card and board games on the lower secondary level.

Table 1 .
Coding and clustering of central concepts in the field of game and educational game assessment and evaluation based on literature review (n=26).