Initial Publication Date: July 12, 2018

Grand Challenge 3:

How do we assess the influence of teaching with societal problems in terms of student motivation and learning about the Earth?

Rationale

Teaching about the Earth through the use of societal issues or problems can theoretically increase student motivation, engagement, and learning. The NRC (2012) advocates for the use of societal problems in the K-12 classroom in multiple disciplines, but this can be especially useful in the geosciences at K-12 and at the undergraduate levels because our field focuses around the surface of the Earth where humans live:

"studying and engaging in the practices of science and engineering during their K–12 schooling should help students see how science and engineering are instrumental in addressing major challenges that confront society today, such as . . . solving the problems of global environmental change'' (NRC, 2012, p. 9).

Societal issues may serve as the vehicle to increase cognitive and affective skills like problem solving, as a student may be more motivated or engaged during problem solving that has personal significance (Gilbert, 2006; Sawyer, 2006; McConnell & Van Der Hoeven Kraft, 2011). Furthermore, in today's society, students must be able to distinguish between "fake news" and scientific facts, especially when there is an issue that impacts their local community. By teaching about these types of situations early and often during students' academic careers, we can prepare them to be informed citizens that can vote accordingly:

"Scientists must make critical judgments about their own work and that of their peers, and the scientist and the citizen alike must make evaluative judgments about the validity of science-related media reports and their implications for people's own lives and society. (NRC, 2012, p. 71)"

In order to know if teaching through the use of societal problems is valid, we as a community should produce research to substantiate the claims that we make about increases in engagement, motivation, and problem solving and learning. We should also investigate how student-centered course activities like flipped courses or service-learning could help to increase engagement and motivation:

"... the geosciences... offer fertile ground for service-learning programs that address intersections between science and society" (National Academies, 2017, p. 6).

All of the calls for integration of societal-relevant approaches to teaching and learning, however, require that quality assessment techniques are used to measure changes in both the cognitive (e.g., problem solving and learning) and affective domains (e.g., motivation, engagement, self-efficacy). In the future, we will need to conduct multi-institutional longitudinal studies that robustly measure the impact of teaching with societal issues.

Research on the efficacy of teaching about the Earth through the use of societal problems should include student data, but should also explicitly link defined student learning outcomes to validated assessment techniques. To do this, we must first fully explain student learning outcomes and the numerous variables related to these, such as defining "geoscientific literacy" as this phrase may have different definitions. In general, GER will need to define the best ways to measure the effect of using societal problems on student learning and on resulting motivations to act (e.g. solve problems/change behaviors). To do so, we will need to determine what instruments currently exist or need to be developed to assess the use of societal problems that allows for future meta-analysis. We suggest that although there are generalized problem solving, argumentation, engagement, and motivation surveys, it may be useful to tailor these specifically for the geosciences.

Recommended Research Strategies

In the cognitive domain, we should assess general problem solving skills as well as how students approach a problem, make decisions, argumentation, and solution generation. To do this, we can use validated assessment techniques like the Social Problem-Solving Inventory-Revised (SPSI-R; D'Zurilla et al., 2004). This inventory examines the ways in which students orient themselves towards the problem, rational problem solving, impulsivity, and avoidance, and self-efficacy. Instructors can also use open-ended responses to further examine problem solving-skills from a quantitative view. In some instances, new instruments may need to be developed to measure problem solving skills when societal problems are integrated into curriculum.
Argumentation may also be an effective way to engage students in problem solving and learning (Driver et al., 2000; Osborne et al., 2004), but needs further research. While assessment of argumentation is difficult, there are methods such as Toulmin's (1958) argumentation model, and revisions of this model, based upon warrants and claims; however, this data is much more qualitative in nature, which merits consideration of review of existing quantitative instruments (or the development of new instruments) that measure argumentation learning strategies.
General learning in the geosciences as a result of teaching using societal issues could be assessed using the Geoscience Concept Inventory (GCI; Libarkin and Anderson, 2005), a validated bank of questions that assess learning, or through the use of the Learning and Study Skills Inventory (LASSI; Cano, 2006). General learning can also be assessed using open ended response questions; however, these questions often take much longer to assess and rubrics are typically subjective depending upon the nature of the question.
Student affective domain is of equal importance when considering societal issues because of the claim that teaching with these issues may lead to increases in engagement and motivation. To measure engagement, instructors and researchers can use a variety of instruments, but one of the most popular of these is the National Survey of Student Engagement (NSSE; Kuh, 2003). However, this instrument is expensive, and fairly generalized and so it may be useful to develop additional engagement surveys that relates more directly to the geosciences. Additionally, we should investigate changes in engagement over the course of one semester, but also examine changes in students' affective domain in geoscience departments that teach primarily in the context of societal issues.
Examine the relationship between students' motivation and attitude and teaching with societal problem. In terms of motivation and attitude, there are several validated options including: Attitudes toward Science Survey (ATSS; Bickmore et al., 2009), Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich et al., 1991), Intrinsic Motivation Inventory (IMI; Ryan and Deci, 2000), and the Academic Motivation Scale (AMS; Vallerand et al., 1992). In addition to these instruments, there are quite a few instruments listed on the NAGT GER Toolbox (GER Toolbox, 2017). Student engagement, motivation, and attitudes can also be linked to the teaching style of the instructor (instructor centered or student centered), and so using a observation protocol like the Reformed Teaching Observation Protocol (RTOP) could be useful to gauge the impact of the instructor (Piburn and Daiyo, 2000)

Show References

References

Bickmore, B. R., Thompson, K. R., Grandy, D. A., and Tomlin, T. (2009). On teaching the nature of science and the science-religion interface. Journal of Geoscience Education, 57(3), 168–177.

Cano, F. (2006). An in-depth analysis of the learning and study strategies inventory (LASSI), Educational and Psychological Measurement, 66, 1023–1038.

Driver, R., Newton, P., & Osborne, J. (2000). Establishing the norms of scientific argumentation in classrooms. Science Education, 84, 287-312.

D'Zurilla, .TJ., Nezu, A.M., & Maydeu-Olivares, A. (2004). Social Problem Solving, Theory and Assessment.

GER Toolbox. (2017). Instruments and Surveys Collection. NAGT. Retrieved from:
https://nagt.org/nagt/geoedresearch/toolbox/instruments/collection_1.html

Gilbert, J. K. (2006). On the nature of 'context' in chemical education. International Journal of Science Education, 28 957–976.

Libarkin, J.C., & Anderson, S.W. (2005). Assessment of Learning in Entry-Level Geoscience Courses: Results from the Geoscience Concept Inventory. Journal of Geoscience Education, 53(4), 394-401.

Kuh, G.D. (2003). What we're learning about student engagement from NSSE: Benchmarks for effective educational practices, Change: The Magazine of Higher Learning, 35(2), 24-32.

McConnell, D. A., & van Der Hoeven Kraft, K. J. (2011). Affective domain and student learning in the geosciences. Journal of Geoscience Education, 59(3), 106–110.

National Research Council (NRC). (2012). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core ideas. National Research Council, Washington, D.C.: National Academies Press.

National Academies of Sciences, Engineering, and Medicine. (2017). Service-Learning in Undergraduate Geosciences: Proceedings of a Workshop. Washington, DC: The National Academies Press.

Osborne, J., Erduan, S., & Simon, S. (2004). Enhancing the quality of argumentation in school science. Journal of Research in Science Teaching, 4(10), 994-1020.

Piburn, M., & Sawada, D. (2000). Reformed Teaching Observation Protocol (RTOP) Reference Manual. Technical Report. Arizona Collaborative for Excellence in the Preparation of Teachers

Pintrich, P.R., & DeGroot, E.V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33–40.

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78.

Sawyer, R. K. (2006). Educating for innovation. Thinking skills and creativity, 1(1):41–48.

Toulmin, S. (1958). The Uses of Argument. Cambridge: Cambridge University Press.

Vallerand, R. J., Pelletier, L. G., Blais, M. R., Briere, N. M., Senecal, C., & Vallieres, E. F. (1992). The academic motivation scale: A measure of intrinsic, extrinsic, and a motivation in education. Educational and Psychological Measurement, 52, 1003–1017.

« Previous Page