Initial Publication Date: December 8, 2017

Research on Cognitive Domain in Geoscience Courses: Quantitative Reasoning, Problem Solving, and Use of Models (Cog B)

Post a Comment »

Authors: Kim Kastens, Lamont-Doherty Earth Observatory; Ashlee Dere, University of Nebraska at Omaha; Deana Pennington, The University of Texas at El Paso; and Vic Ricchezza, University of South Florida

Introduction:

Quantitative reasoning, problem-solving, and use of models and modeling present many daunting challenges to both students and instructors. All would benefit from more education research. In choosing which challenges and strategies to highlight, we had to be quite selective and put aside many other intriguing candidates. We favored challenges and strategies that are: (a) high impact, (b) under-researched, (c) addressable on a ten-year time scale, and/or (d) central to how geoscientists think about the Earth and about Earth/human interactions.

As an example of a "high impact" strategy, we chose to focus strategy 1-E on students with poor math preparation or attitudes--even though we know that virtually all students need help with quantitative reasoning--because we thought that research-based strategies could make the most difference there. As an example of "under-researched," we foregrounded "problem-finding" in strategy 2-A, in part because it was so much less researched than problem-solving. As an example of "addressable on ten-year timescale," we narrowed down our systems thinking strategy 3-C to positive and negative feedback loops, in part because we thought that all of systems thinking was too big to tackle in the allotted timeframe. As an example of "central to geoscientific thinking," in strategy 3-B we sought to get a handle on how the human mind manages to "run" mental models of events in the past or future.

Addressing each of these challenges will require innovative, creative thinking, along research pathways that are not yet clear, along with vast amounts of hard work. But we are confident that each of them is ripe for new discoveries, and we look forward to both the intellectual and practical outcomes of these efforts.

Jump Down To: Grand Challenge 2 | Grand Challenge 3

Grand Challenge 1: Quantitative Thinking

How does quantitative thinking help geoscientists and citizens better understand the Earth, and how can geoscience education move students toward these competencies?

Rationale: Ability to think quantitatively is an important part of what transforms an intro student into a major and then into a professional geoscientist. Employers value quantitative thinking. Quantitative thinking may be a sweet spot for GER research, in that there is rich trove of math education research to build upon. Please note that this set of strategies is not meant to comprehensively cover the entirety of geoscience quantitative thinking; we have prioritized strategies that we think offer the highest leverage and that will produce a strong foundation upon which future efforts can build.

Strategies

  • 1-A: Collaborate with mathematics education researchers and quantitative literacy experts.
    • Rationale: There is already a large community who has thought about these issues, and we want to be able to build on their efforts rather than start from scratch. Some things that we think would could gain from such collaborations:
      • vocabulary and constructs with which to talk about how experts and novice participants in our studies are thinking and learning.
      • insights about mathematical habits of mind, and a partner in researching how these habits of mind come into play in thinking about the Earth.
    • Contact points:
  • 1-B: Research how novices and experts take an Earth phenomenon that they understand holistically or experientially and transform it into a mathematical representation (e.g., word equation, mathematical equation, mathematical or computational model)
    • Rationale: We feel that this is a skill that most of our students lack, and it is generally not being taught in math classes. For majors, this is an essential skill for doing original research. For non-majors, this is a valuable life skill. There is very little research on this, and also not much guidance for educators.
    • Models:
      • Roth, W. M., & Bowen, G. M. (1994). Mathematization of experience in a Grade 8 open-inquiry environment: An introduction to the representational practices of science. Journal of Research in Science Teaching, 31, 293-318.
      • Jasper Woodbury series (1990's vintage). Vanderbilt, (1992). The Jasper Experiment: An exploration of issues in learning and instructional design. ETR&D, 40(1), 65-80.
  • 1-C: Research what quantitative habits of mind expert geoscientists use in understanding the Earth.
    • Rationale: Research suggests that habits of mind are more enduring and transferable than specific skills. We do not know what the geoscience careers of the future will entail, or what specific skills might be needed. Habits of mind should prepare students for whatever specific tasks are required. We and our math colleagues have put a lot of effort into teaching math skills; we now want to move beyond teaching quantitative skills to teaching quantitative habits of mind. This topic is seriously under-researched.
  • 1-D: Work towards a community consensus on what quantitative skills and habits of mind are needed to function effectively as a citizen of the planet.
    • Rationale: Many of the critical Earth-related problems facing humanity can be broadly understood at either a qualitative or quantitative level; for example climate change, resource depletion, and resilience in the face of natural disasters. However, to move beyond merely understanding the problems, so as to be able to weigh the costs and benefits of conflicting paths forward, requires quantitative thinking. There is not a consensus on what the elements of such thinking should be, but the traditional algebra-calculus sequence is almost certainly not an optimal match. Deciding what needs to be learned is a necessary pre-cursor to designing a comprehensive research program in this area.
    • Approaches: This could be approached as a community discussion. Or it could be approached as a research question, looking out in the world at what kinds of tasks and decisions citizens face in the context of Earth/human interactions, and what quantitative capacities are needed to succeed at these tasks and make wise decisions.
  • 1-E: Research what learning experiences can help students with poor math preparation or attitudes have an experience where they can feel the power of math to answer questions or solve problems they care about concerning the Earth.
    • Rationale: Extensive literature in and out of the geosciences and uncounted personal experiences as educators tell us that many of our students enter our classes or our major(s) with a negative attitude about math (e.g., math anxiety, math phobia) combined with a lack of proper math preparation, that leads to math avoidance. This shuts them off to the rich possibilities of the power of math to solve problems and open entire career opportunities they had not considered before. Our working group recognizes that improving quantitative thinking about the Earth is important for all students, but we have prioritized this population for research attention because the problems here are so gigantic and so important. We prioritized "have an experience where they can feel the power of math to answer questions or solve problems they care about" because we think that this can be a pathway to transform math from "something I hate" into "something I want and need."
  • 1-F: Collaborate with Assessment experts to develop and validate assessments for the learning goals articulated in strategies 1-B and 1-E, and to begin to shape the findings of Strategies 1-C and 1-D into assessable constructs.
    • Rationale: There are few to no tested, validated, research-grade assessment instruments that tackle quantitative reasoning in the context of Earth education. The building of such assessments requires both deep knowledge of the Earth and serious expertise in assessment; collaboration will be helpful.
    • Approaches: It might be possible to build Earth content into existing quantitative reasoning assessments. Or to increase the quantitative component of existing Earth literacy assessments. Or to formalize and validate assessments that have been developed as summative or formative assessments for coursework. Any of these pathways would need to begin with a clear articulation of learning goals and of what student behavior and/or product would demonstrate that each learning goal had been met. This is a long path; all the more reason to start sooner rather than later.

State of current knowledge:

The literature outside of geoscience is extremely rich (e.g., mathematics, mathematics education, and educational psychology), and there is some depth to the research done within the geosciences as well. Ashcraft is a good source for mathematicians. Maloney and Beilock (2012) was a good source on this as well, and several sources have indicated that modest gains in student attitudes can be achieved with some effort (Wismath and Worrell 2015; Lipka and Hess 2016; Follett et al. 2017; Ricchezza and Vacher 2017). However, results are mixed and not all interventions have produced desired results (Sundre et al. 2012; Mayfield and Dunham 2015).

Grand Challenge 2: Problem-finding & Problem-solving

How can we help students find and solve problems they care about concerning the Earth, in an information-rich society (big data, emerging technologies, access to a wide-variety of tools, rich multimedia)?

Rationale: Historically the problems that students tackle in science classes, including geoscience classes, have been assigned by the teacher and rather constrained in scope. But many of the problems geoscience students will confront in the future are complex, messy, ill-defined, and require working across disparate knowledge, methods, and data sources. Such work has been coined "convergent" science, as solutions for problems must be converged on from different directions. We are at a time where technology can leverage the power of undergraduates so that they can make real contributions to solving authentic, messy problems, rather than being constrained to well-bounded classroom problems. Information technology has changed, and will continue to change, the kinds and quantities of resources that are available for problem solving. Students need to learn to navigate this rapidly changing space, identifying and harnessing resources (e.g. tools, data, models, experts, collaborators) that can be brought to bear on their problem. We anticipate that young people who learn to identify and solve convergent science problems as students will carry that skill-set and habit of mind into their personal, civic and professional adult lives.

State of current knowledge:

  • There is existing research on the process of diffusions of innovation and on technology adoption. Both of these identify awareness, perceived usefulness, and initial training as key early phases in the process of technology adoption. However, there is little research on how to enable these early phases in the sciences in general and the geosciences in specific.
  • There is existing research on computational thinking and data analysis skills, mostly within computer science education. There is very little research on this topic in geoscience, beyond identification of general categories of skills needed.
  • The Geoscience Employer's Workshop Document identifies a set of existing technologies with which students need to be familiar; this list will change continually in the future.
  • There is a body of literature on problem-based learning, including in medicine, business, engineering, and to a lesser extent in geosciences. Much of this literature comprises "curriculum & instruction" style papers rather than discipline-based educational research. Given the messy and heterogeneous nature of problems and problem-solving, it is hard for researchers to produce generalizable knowledge on problem-based learning, findings that can be extended beyond the immediate context of a study site.
  • There is a body of literature on the science of team science and cognition in groups. This has mostly been developed through case studies of teams in different contexts – mostly within large organizations, medical teams, and community organizations. There is some emerging research on how learning occurs in teams, and how activities can be designed in geoscience classrooms to develop these capabilities.

Strategies:

  • 2-A: Research the problem-finding process -- techniques by which vague, open-ended problems are turned into solvable problems, and how these can be taught:
    • Rationale: Problem identification in convergent science requires the ability to co-create a shared conceptualization of the problem to be solved based on what each participant can contribute. There are an infinite number of ways to frame research on ill-defined problems; solutions depend on the expertise at hand. The challenge is to learn enough about the different contributing perspectives to determine how they can be collectively leveraged. Moreover, to make serious headway on a substantial problem, the problem and proposed solution has to be one that is of high importance to the solver or solving team; otherwise, they won't have the motivation to push onward through the inevitable challenges and setbacks. Finding a problem that is both solvable and of passionate personal interest is doubly hard. We need evidence on how skilled problem-solvers do this, models for how learning occurs in these situations, and pedagogical approaches to help students learn to do the same. Employers, including those involved in the Future of Geoscience Education Employers Workshop, articulate the importance of learning to work on problems with no clear answers and manage the uncertainty associated with solving these types of problems
  • 2-B: Research the process by which geoscience students learn and adopt new methods and technologies.
    • Rationale:As technology advances, new tools are available that generate ever larger datasets. Such datasets are potentially valuable to help solve complex problems, but the most effective strategies for learning how to manage and extract solutions from large datasets are not clear. Skills are needed to: 1) skillfully collect, integrate and analyze data that are increasingly generated automatically by advanced sensors and/or simulation models; 2) understand advanced methods and technologies for conducting data-intensive science; and 3) timely identify and learn technologies that are relevant to the problem and are emerging at an increasingly rapid pace. Likewise, new technologies could be used to process data in new ways or to advance learning, but more research is needed on how to most effectively use such technologies, especially when technological developments constantly evolve. In addition, employers, including those involved in the Future of Geoscience Education Employers Workshop, articulate the importance of the ability to use data to solve problems.
  • 2-C: Collaborate with experts on team science (from cognitive or learning science) to research effective strategies to teach collaboration and teamwork in undergraduate geoscience education.
    • Rationale:Convergent science requires the ability to collaborate effectively across disciplines and/or with external stakeholders, especially with experts from social sciences, engineering, and computer science. Employers, including those involved in the Future of Geoscience Education Employers Workshop, consistently emphasize the importance of ability to work in teams, including interdisciplinary teams. Although many classes incorporate team projects, most provide little training to students on how to work effectively in a team. There are few relevant models of teamwork training for geoscience faculty to follow, and most do not have the knowledge and expertise to construct their own models. Although there exists decades of research on teamwork in other contexts, there is little GER research on how what is known about teamwork can be applied in geoscience contexts.

Grand Challenge 3: Models and Modeling

How can we help students understand the process by which geoscientists create and validate physical, computational, mental, systems, and feedback models and use those models to generate new knowledge about the Earth?

Rationale: We have prioritized this grand challenge because we think that many or most citizens do not understand how modern scientific models are developed and tested, and how they are used to make predictions. Geoscientists use an ambitious and iterative process of building models, starting with mental working models and working up to computational models, testing their models against empirical data at every iteration. Only after many such cycles is the model considered robust enough to make predictions about the earth where we have no data--including the past or the future. Lack of understanding of how modern scientific modeling works allows skeptics and deniers to dismiss evidence that comes from modeling, for example evidence that climate change is anthropogenic.

Current state of knowledge:

  • There is some good literature on how scientists create and validate models, including external runnable models (Nersessian is a good source for this).
  • There is active good work on how students and teachers understand the scientific practice of modeling, but that mostly deals with conceptual models (Christina Schwartz is good source for this).
  • There does not seem to be as much work on how students (and teachers and the public) understand how scientists develop and validate modern computational models (such as global climate models), or on how such models are used to develop new knowledge, or on the affordances and limitations of such models.
  • There is a lack of good assessments of students' ability to create and use geoscience computational models.
  • There is a particular shortage of educational research at the interface between models and data: how to help students learn to use data to test models, and how to help students learn to use models to interpret data.

Strategies:

  • 3-A: Research what students at various levels understand the process by which geoscientists create and validate models and use those models to generate new knowledge about the Earth.
  • Rationale: It has been asserted (e.g. Kastens, et al, 2013 ), that students and the general public have little understanding of the process by which the computational models of modern science are created, validated, and used to make predictions, but the breadth, depth, distribution and nature of this ignorance needs to be probed, to lay the groundwork for a comprehensive research agenda.
  • 3-B: Collaborate with cognitive/learning scientists to understand how the human mind runs mental models of the future and/or the past, and then use this understanding to research how geoscience education can improve and leverage that ability.
  • Rationale: The first step towards generating a scientific computational model of a part of the Earth system is to develop a conceptual model that can be "run" in the mind (i.e. one can envision processes that produce observable products or behaviors, and can think through how those products or behaviors would differ as circumstances or inputs change.) The ability to run mental models is thought to be unique to the human brain and is therefore a powerful cognitive tool we have to understand the world around us. Even without formal training, our brains have this inherent ability (for example, anticipating where one will and will not be able to find parking on campus), but it is unclear how this ability is applied to understanding earth systems and how we can leverage this power of the mind to inform education practices.
  • 3-C: Research how does the human mind understand positive and negative feedback loops, how can geoscience education foster that ability, and how can we assess this?
  • Rationale: Many, and maybe even most, environmental problems are underlain by reinforcing (aka positive) feedback loops–for example the albedo feedback loop that strengthens the impact of climate change in the Arctic as the polar sea ice melts. Many of the potential solutions to environmental problems work by strengthening balancing (aka negative) feedback loops, or by weakening positive feedback loops. To understand environmental problems or contribute to environmental solutions in a deep and impactful way, students need to understand such processes. Practitioners find that these topics can be taught, but are challenging to teach and to assess. Feedback systems can be taught at a qualitative level or a quantitative level, and both are challenging.
  • Relevance to workforce skills: The Geoscience Employers' Workshop Outcomes list "systems thinking" as an "Earth Science Habit of Mind" under "Geoscientific Thinking." We thought that all of Systems Thinking was too big for our 10 year time line, and that feedback loops might be tractable on that time scale.
  • Approaches: This is a challenge that could benefit from collaboration with other DBER's, perhaps through the DBER-A alliance, as feedback loops are very important in life sciences (ecology, physiology) and engineering.




Cognitive - Problem Solving, Quantitative Reasoning, Models -- Discussion  

I particularly like Grand Challenge 2 and Strategy 2-C. My dissertation research, in the Learning Design & Technology department of Education, will investigate diffusion of innovation of smartphones used in fieldwork for collaboration/teamwork/communication purposes. I'm also going to look at how field instructors encourage and assess collaboration/teamwork/communication (as a process rather than grading products) and how adoption of the innovation may help.

12420:34655

Share edittextuser=37906 post_id=34655 initial_post_id=0 thread_id=12420

Originally Posted by Trina Kilty


I particularly like Grand Challenge 2 and Strategy 2-C. My dissertation research, in the Learning Design & Technology department of Education, will investigate diffusion of innovation of smartphones used in fieldwork for collaboration/teamwork/communication purposes. I'm also going to look at how field instructors encourage and assess collaboration/teamwork/communication (as a process rather than grading products) and how adoption of the innovation may help.



Hi Trina, Good luck with you dissertation research. You are working on an interesting topic. There is good work coming out of engineering education reform and education engineering research on teamwork and collaboration.

12420:34698

Share edittextuser=75 post_id=34698 initial_post_id=0 thread_id=12420

This post was edited by Kristen St. John on Jan, 2018
1. I like that the intro articulates the approach your WG took to narrow your focus (“We favored challenges and strategies that are: (a) high impact, (b) under-researched, (c) addressable on a ten-year time scale, and/or (d) central to how geoscientists think about the Earth and about Earth/human interactions.”). Could you preface this with stating the charge of your WG? WG#3 did this and I think that set the stage well.

2. Please see WG#2 because there are important connections between their GC#3 [What approaches are effective for students to understand various models (numerical and analytical) that are used for prediction and research in atmospheric, oceanic and climate sciences, including model limitations?] and your theme [Quantitative Reasoning, problem solving, and Use of Models]. I will work to make sure these connections come out in the synthesis but there may be citations and connections WG3 and WG7 want to directly address in your own theme’s rationales and strategies too.

3. The proposed strategies are thorough and I like that there are contact points included too. However they are writing in more of an outline format. For consistency sake I would like to see each strategy simply with a letter (e.g. A, B) but not with sub-bullets. I don’t think we need the #-letter (e.g., 1-A) because it is already under GC#1 so that is already clear. Anything you can do to also migrate more to paragraph format would be helpful to the flow of it too.

4. Can the state of current knowledge sections each be folded into their corresponding GC rationales? I would like to see each rationale section be better grounded in the literature. Sometimes the state of current knowledge is listed after the strategies and sometimes before, but I’d like to see it embedded within the rationale and cited. For example, in GC#2 there is a bullet on “The Geoscience Employer's Workshop Document identifies a set of existing technologies with which students need to be familiar; this list will change continually in the future.” But what is this employer’s document? Is that the Summit report? Please cite these statements so researchers then can go and read them and get off the ground on their own research questions that branch off from these resources you highlight.

5. I mentioned this for WG#7 too: Each GC should have its own set of references [which you have] and those should only include citations that are made in that section. What I’m seeing though are references that are listed but not cited in the paragraphs of the GC rationale, strategies etc. I think that makes it harder for the reader to know hw the references fit/how they are useful. That said, I also like the idea of making all of the references we collected for a theme available publically. Perhaps that could be as an added Recommended Reading list or as Additional Resources list? But that isn’t a substitute for integrating specific key references into the intro, rational and recommended strategies for your theme. Please look at other WGs to see how they approached it. A good example is WG#2.

6. WG#3 included some suggestions for important researchable questions under each of their Grand Challenges, in addition to their recommended strategies. That wasn’t a requirement in the format, but I think it is very effective. Please look at WG3’s draft chapter and see if you think that is a good way to go to help give more concrete examples of important research directions in quantitative thinking, problem solving, and use of models in the geosciences.

7. Small thing in formatting but impt for consistency with other WGs: please edit the GCs so that the question itself (e.g., How can we help students find and solve problems they care about concerning the Earth, in an information-rich society (big data, emerging technologies, access to a wide-variety of tools, rich multimedia)?) is actually listed as part of the GC header – not as subset of it. So it could be Grand Challenge 2: Problem-finding and Problem solving - How can we help students find and solve problems they care about concerning the Earth, in an information-rich society (big data, emerging technologies, access to a wide-variety of tools, rich multimedia)?

12420:34863

Share edittextuser=1143 post_id=34863 initial_post_id=0 thread_id=12420

I really appreciated the "preamble" that helped to highlight and describe the tough decisions that you had to make in prioritizing, and how you went about that process! In passing this on to NSF and recognizing that some future readers there might not be geoscientists, I wonder if it might be helpful to add a little more intro material on how/why these specific skills are critical to success in our field, and what more research (presumably enabling greater student success in these areas) would likely enable us to do? (You do that a bit in the individual challenges, I see- but perhaps foreshadow those ideas a bit earlier to pique the reader's interest?)

1-B: This is really interesting, and made me wonder how much of "temporal and spatial reasoning" might be connected to mathematical literacy. Could you unpack a little further what it means to "understand holistically or experientially," and maybe summarize a bit the literature you reference below to help folks new to this idea process and invest?

1-E: I wonder if a recommended approach for this GC might involve collaboration/overlap with the "Societal Problems" group? There's a body of literature on the conceptual teaching of math in order to facilitate transfer- I also wonder if this body of literature also includes any affective analysis? (e.g. Schwartz et al., 2011 and Richland et al., 2012)

In GC1, "State of current knowledge" comes after your individual challenges- perhaps moving this up and expanding it a bit might help the reader understand your entry point into the conversation a bit more easily?

I enjoyed reading this! Good luck with the revisions!

12420:35370

Share edittextuser=18919 post_id=35370 initial_post_id=0 thread_id=12420

This is a really rich and detailed exposition of the state of the multitude of challenges facing our community in this complex area, and I really appreciate the way this summary breaks this down into manageable units of analysis.

Echoing many of the comments above, and especially those of Kelsey Bitting, I am intrigued by the intersections of all of these manageable units with all of the other grand challenge areas and working groups. For instance, there is a small but rich literature on perspective taking and relationship with meaning units as a function of language, culture and experience in indigenous communities (e.g. the work of Sara Unsworth, Megan Bang, Doug Medin and his group). Clearly this has some relevance on understanding problem solving and identification, thinking about models and modeling, and also the nature of expertise and quantitative thinking. It also is an exemplar of a connection to student-centered issues in access and success (WG5). It's really just a general comment rather than an actionable item of review or revision, but the richness of many of applications and real-world situations we encounter as educators, trainers, and researchers happens in the presence of the convolution of these variables, which complicates studies and their generalizability in fields as small as ours. Your summary does a nice job of illuminating which variables are getting convolved, however, and is valuable.

Excellent stuff - thanks for the strong effort.

12420:35376

Share edittextuser=429 post_id=35376 initial_post_id=0 thread_id=12420

Join the Discussion


Log in to reply