Research on Institutional Change and Faculty Professional Development, TA Training

Post a Comment »

Authors: Kelsey Bitting, Northeastern University; Leilani Arthurs, University of Colorado at Boulder; LeeAnna Chapman, North Carolina State University; Heather Macdonald, College of William and Mary; and Cathy Manduca, SERC-Carleton College

Jump Down To: Grand Challenge 1 | Grand Challenge 2 | Grand Challenge 3

Introduction

Over the past 20 years, numerous institutions and groups have repeatedly called for changes in university-level STEM education in the United States (e.g., NSF, 1996; NAS, 2005; PCAST, 2012; AAAS, 2017). Continued and growing deficits in the U.S. STEM workforce can be tied to evidence that students leave science majors because of instructional experiences, rather than because they lack the ability to succeed (Griffith, 2010; Seymour & Hewitt, 1997). The persistent underrepresentation of geoscience majors from a wide variety of backgrounds (AGI, 2014; 2016) contributes to this workforce deficit, and also restricts the perspectives and approaches that are brought to bear upon pressing societal challenges such as climate change, environmental degradation, and equity of access to Earth resources. These same societal challenges also require more widespread and effective geoscience learning experiences for non-majors that can produce a geoscience-literate public that supports science and understands its value to society (Feinstein et al., 2013). The active, engaged pedagogies that have been demonstrated to impact learning and retention for all students (NRC, 2012; Freeman et al., 2014; Kuh, 2008) represent a dramatic shift from the traditional lecture style of past STEM and geoscience courses (especially at the introductory level). In a 2012 survey of geoscience instructors, 57% of respondents used active pedagogies for more than 20% of time in a given course (Manduca et al., 2017).

Beyond a basic paradigm shift from instructor-centered (lecture-based, "transmissionist" or behaviorist) to student-centered (active learning, constructivist) pedagogies, two additional factors contribute to the need for instructional change: the increasing recognition of the need to teach more than just disciplinary content to prepare students to enter the workforce, and the potential transformative value of geoscience education research (GER) to inform continuous improvement in teaching specific geoscience concepts.

In a rapidly-evolving and highly-technical society that faces global challenges like those above, college graduates require robust skills and habits of mind that transcend disciplinary boundaries (AAAS, 2017). Outcomes of the 2017 Geoscience Employers Workshop coincide with broader employer surveys (Hart Research Associates, 2015) in identifying skills such as critical thinking, problem solving, communication, and collaboration as essential for college graduates. While it may be tempting to suggest that these general skills should be fostered in general education courses, integrating these skills into the disciplinary curriculum is likely to achieve substantially better outcomes (Abrami et al., 2008). The need to integrate non-disciplinary skills explicitly and intentionally into geoscience courses challenges the traditional narrative that geoscience content knowledge alone qualifies an instructor to teach geoscience courses. As our society and the professional contexts in which geoscientists work continue to evolve, the specific skills that must be integrated into geoscience courses will also need to continuously evolve and adapt to those needs.

Geoscience education research (GER) investigates questions related to how people reason about and understand geoscience concepts, how students learn and evolve their understanding to be more expert-like, and how instruction can best foster that learning and development. This field of study has experienced dramatic growth in recent years, exemplified by the founding of the NAGT GER division in 2014 and the current existence of 15 GER graduate programs in the U.S. (Libarkin, 2017). This body of research can and should inform the approaches that geoscience instructors use in curriculum and course design, instruction, assessment, and mentoring to result in more efficient practices and better outcomes for students and instructors alike.

To foster constructivist approaches to geoscience learning, effectively integrate trans-disciplinary skills learning into the geoscience curriculum, and continuously move GER findings into practice will require a substantial investment in research on institutional change and instructional change. Bodies of research on institutional change and teaching-related professional development exist already that can be drawn upon to help direct our research in this area. Interested readers are encouraged to refer to recent reviews of the topic in the geosciences and beyond (Bitting et al., 2017; AAAS, 2017; Henderson et al., 2011; Manduca, 2017). Additional background specific to our grand challenges will be provided in the sections below. Ultimately, this set of challenges and preexisting scholarship led our working group to identify the following three grand challenges to guide research on institutional change and professional development in the geosciences:

  1. How can we best support the continual growth of geoscience instructors' ability to teach effectively and to progress in their practice? How does the individual's cumulative experience, position type, institutional context, and the nature of the desired learning impact the type of learning opportunities that are most effective?
  2. What supports the health of the programmatic environments in which geoscience learning occurs?
  3. What are the respective roles of on-campus interdisciplinary opportunities, national disciplinary opportunities, and other forms of community in informing and enabling geoscience instructor learning and change in undergraduate geoscience teaching practice? How do instructors negotiate learning and change across interaction with multiple forms of learning opportunities and communities across the arc of their careers?

The integration of these three challenges, and the ways in which they come together to support undergraduate geoscience teaching and learning, is represented in figure 1, to the right.

Grand Challenge #1: How can we best support the continual growth of geoscience instructors' ability to teach effectively and to progress in their practice? How does the individual's cumulative experience, position type, institutional context, and the nature of the desired learning impact the type of learning opportunities that are most effective?

Rationale:

Our community's ability to support instructors' continuous progress in their teaching practice will determine to the degree to which we are able to produce citizens and geoscientists who are able to understand today's global challenges and contribute to possible solutions. Studies analyzing the overall impact of single professional development programs have demonstrated a number of valuable considerations to inform program development: for example, longer-term interventions, programs that facilitate growth in participants' beliefs and conceptions about teaching (rather than only changing their practice), involvement in a community of practice or community of transformation, and systems-design approaches that recognize the roles of and feedbacks among various components are often more successful (Condon et al., 2016; Chapman & McConnell, 2017; Henderson et al., 2011; Gehrke & Kezar, 2016; Kastens & Manduca, 2017; Kastens & Manduca, in press).

Further work is needed to determine the ways in which an individual instructor's personal history and identity (including gender, race, type of employment, undergraduate learning experience, graduate training, self-concept, and perceived support structures) may interact with departmental or institutional variables (such as departmental and institutional culture, institution type, and competing priorities or initiatives) to result in widely-variable outcomes, needs, motivations, professional goals, and even freedom to engage in professional learning about teaching. Preliminary evidence also suggests that different types of learning may require different types of engagement: for example, beliefs about teaching may be relatively difficult to change (Yerrick et al., 1997) but can be effectively targeted through collaborative and authentic long-term engagement (Pelch & McConnell, 2016), while changes in practice that are consistent with the beliefs already held by a participant might be easier to achieve (e.g., Glackin, 2016). Further work is necessary to clarify the different types of learning required of geoscience instructors, and how best to tailor interventions to those goals.

Proposed Strategies:

  • Conduct descriptive case studies of individual instructors representing a variety of identities and institutions, with special attention to how they make decisions about potential instructional learning and change. Highlight identity and institution characteristics that seem most impactful, and recommend that future teaching professional development programs collect demographic data that captures those variables after participation.
  • Conduct interviews with community leaders in teaching professional development for the geoscience community and review existing literature to identify common learning objectives for geoscience instructors participating in teaching professional development programs. Convene a small working group to sort those objectives according to the cognitive processes, level of challenge, and type of change required to provide a typology of learning objectives.
  • Based on the typology of learning objectives, identify or design appropriate and rigorous assessment measures for each category. Recommend the use of those assessment instruments across future teaching professional development programs, to allow consistent comparisons and future meta-analyses.
  • Set up a community database of teaching professional development program assessment data that includes 1) demographic information (as identified by case studies above) and 2) assessment data on specific learning objective types using recommended measures to analyze the correlations among these variables.

Grand Challenge #2: What supports the health of the programmatic environments in which geoscience learning occurs?

Rationale:

Geoscience departments and programs can be conceptualized as complex systems comprised of instructors, students, and administrators, as well as curricula, courses, and assessment mechanisms, as well as physical structures such as classrooms and labs (Condon et al., 2016; Manduca, 2017). Viewed from this systems perspective, healthy programs are those in which new and potentially valuable ideas about teaching and learning enter the system continuously, are discussed freely among individuals, and enable experimentation with and implementation of corresponding teaching practices, curriculum design, and assessment (Manduca, 2017). While our task is to consider program health with respect to teaching and learning, we hypothesize other measures of programmatic health (such as research productivity, faculty and student diversity, and student retention and graduation rates) are likely to co-vary due to underlying factors such as relational trust or collegiality.

Previous work on departmental climate is congruent with this systems conceptualization. Instruments designed to quantify and describe departmental climate for instructional change consider the impact of factors such as teaching-related rewards structures, resources and opportunities for professional development, collegiality, leadership, and other factors (Walter et al., 2014). The role of collegiality and leadership are supported by recent social network analyses that have identified DBERs (such as geoscience education research faculty) as influential agents for change (Andrews et al, 2016). Beyond this, little is known about departmental climate in geoscience programs of various types, and if, how, and why climate characteristics impact the health of the overall system. How characteristics that support healthy systems are fostered, negotiated, and maintained over time is also unknown.

While each geoscience program is a system unto itself, it is embedded within larger systems such as the college, institution, discipline and its component professional organizations, and national and global societies, all of which exert various types of influences on the health of a program. Furthermore, individuals from a program may participate in communities of transformation that transcend individual disciplines (Gehrke & Kezar, 2016), and these ties may also contribute to systems health within a program. Thus, further work will need to clarify factors contributing to program health from both within (departmental climate) and beyond the program itself.

Proposed Strategies:

  • Conduct mixed-methods case studies describing the health of a variety of geoscience programs, including measurements of departmental climate and interviews with students, faculty, and administrators that seek to determine their perceptions of internal and external influences on teaching and learning information flow and changes in practice.
  • Based on those case studies, develop a categorization or rubric for programmatic health.
    • In departments in which health has been maintained over time, interview faculty and administrators to determine ways in which they perceive that health to be sustained or threatened, and what actions they and others have taken to support it.
    • In departments in which health has been improving, interview faculty and administrators to determine actions and influences that have allowed that change.
    • In departments that are less healthy, assess internal and external influences that maintain the status quo. Test and evaluate potential interventions to foster and improve health.
  • Within the case studies above, examine the hypothesis that other types of health co-vary by collecting data on student and faculty diversity, retention, satisfaction, and research productivity.
  • Based on those case studies, formulate hypotheses about internal and external variables that appear to have the greatest impact on program health, and design quantitative survey instruments designed to test those hypotheses across a representative subsection of geoscience programs in the U.S.
  • Conduct social network analyses at a variety of scales within the geoscience education community, including departments and disciplinary societies, to identify the characteristics of change agents and to understand of how those change agents support program health.

Grand Challenge #3: What are the respective roles of on-campus interdisciplinary opportunities, national disciplinary opportunities, and other forms of community in informing and enabling geoscience instructor learning and change in undergraduate geoscience teaching practice? How do instructors negotiate learning and change across interaction with multiple forms of learning opportunities and communities across the arc of their careers?

Rationale:

The 2017 AAAS report, "Policies and Practices to Support Undergraduate Teaching Improvement" identified three types of knowledge necessary for effective instruction: content knowledge of the discipline, general teaching knowledge, and pedagogical content knowledge (AAAS, 2017). On-campus instructor learning opportunities frequently take place in centers for teaching and learning, involve participants from a wide range of academic disciplines, and typically focus on fostering general teaching knowledge such as how to foster metacognition or encourage classroom interaction (AAAS, 2017). In contrast, discipline-specific learning opportunities that are most frequently held at the national level by disciplinary societies (e.g., NAGT, GSA, AGU) are well-positioned to focus on pedagogical content knowledge, which includes a knowledge of specific misconceptions novices commonly hold about specific geoscience concepts such as plate tectonics or deep time, knowledge of the pathways students typically follow in becoming more expert-like in their geoscience knowledge and skills, and effective approaches to guiding that specific knowledge and skill development (AAAS, 2017). Discipline-specific learning opportunities such as in-department graduate teaching assistant training (Bitting et al., 2017), on-campus interdisciplinary STEM centers (NSEC, 2017), and national interdisciplinary communities of transformation such as SENCER and PKAL (Gherke & Kezar, 2016) likely represent hybrid models that may support growth in both general teaching knowledge and disciplinary pedagogical content knowledge.

Over the course of an instructor's career, they are likely to participate in multiple different types of learning opportunities, and gain different types of benefits from these different forms of community. For example, graduate students who engage in meaningful learning about teaching in their departments from the start of their careers may understand the importance of continuous learning about teaching and be more comfortable experimenting with new techniques throughout their careers (Bitting et al., 2017). Participants in national communities of transformation frequently felt that they gained credibility in their home communities via their association with the national group (Gherke & Kezar, 2016). Further investigation into the mosaic of differential benefits for learning and changes in practice will clarify the roles these different types of opportunities can play.

Beyond the individual learning opportunity, little is known about how instructors teaching knowledge and practices evolve and change over time in a non-linear way (Manduca, 2017). Theories of individual change specify that pathways through the change process (much like the rock cycle) are non-linear and multi-directional (DiClemente & Velasquez, 2002). An instructor may attend many different learning opportunities before deciding to experiment with a new practice, while another may incorporate small incremental changes as their thinking about teaching evolves. Roughly a third of physics instructors who do experiment with active pedagogies abandon them (Henderson et al., 2012).

Peer support and collaborative learning is a powerful influence in the learning process: Teams that attend community of transformation meetings are more successful at instituting and sustaining change (Gherke & Kezar, 2016). Co-teaching helps new instructors adopt established non-traditional course models more effectively (Henderson et al., 2009). Department networks and opinion leaders influence the flow of information about changes in teaching practice (Andrews et al., 2016). Finally, communities of practice provide value to both the individual and to the community to sustain the learning of both (Kastens & Manduca, in press). These various sources of peer exchange combine and interact with learning from the various types of learning opportunities described above in ways that are currently unknown.

Proposed Strategies:

  • Conduct a longitudinal study that follows early-career geoscience instructors (graduate teaching assistants and early-postgraduates) for a 10-year period or longer. Develop semi-structured interview protocols, administered at two-year intervals designed to understand participants growth and evolution in both conceptions about teaching and learning and teaching practices (including classroom practice, assessment, and mentoring), as well as to understand how they make decisions to pursue learning opportunities and consider, adopt, and abandon or sustain changes in their practice.
  • Collaborate across institutions and disciplinary societies to develop and deploy common end-of-opportunity surveys or assessments to identify different learning outcomes and benefit types. Iteratively redesign these instruments at three- to five-year intervals, as hypotheses about relationships are formulated and reformulated with progressive analyses of the combined datasets.
  • Encourage, support, and provide guidelines and protocols for follow-up interviews and classroom observations with opportunity attendees both before and three, six, or 12 months after their participation. This process would seek to elucidate how participants connect what they learned during the opportunity to their prior thinking and practice, whether they have been able to implement changes based on their learning, the specific characteristics of the changes that have been implemented, and the features of the learning opportunity or participant's situation (peer support, leadership, etc.) that the participant believes have informed the learning and implementation process.




Institutional Change and Professional Development -- Discussion  

1. The intro brings together many different components that help set the stage for this theme and is well cited. However I am a little concerned about the length of the intro, compared to other WG themes and wonder how it could get to the geoscience context faster? One way might be to have the last paragraph come first? Or maybe that needs stay where it is but have a short intro paragraph that defines the WG charge (WG#3 did this but here I think it needs to be done earlier in the intro here if you do it) and state that to tackle that charge it requires looking both outward at big ideas in undergraduate [instructional?] and institutional change and inward at our discipline? Then you can get into the bulk of your intro?

2. I’m a little confused about instructional change as discussed in the intro because I keep thinking about how we also have a WG on Instructional Strategies. How are these the same or different? Can you clarify? Is it in the role that programs have in determining instructional goals and instructional strategies are more determined at the individual instructor level? And/or is it that in this WG theme we are looking at how institutions can influence instructional approaches through PD and TA training? I’m not sure how the PD and TA training evolve out of the intro.

3. Intro: Can you provide a citation for the Outcomes of the 2017 Geoscience Employers Workshop?

4. Figure: I like the figure symbolism and the ideas it includes. I wonder though if it would be better if the base was two layers rather than two side-by-side blocks? Maybe National is the base and the Campus is the next layer then the pillars comes out of base?

5. Can you explain the figure a bit more in the intro? Just a few sentences.

6. I really like the three GCs. These are dense but I think will resonate well among geoscience faculty from different stages in their careers, different roles, and from different types of institutions. Your GC#1 in particular, makes me think about comments I made in WG5 on access and success – it isn’t just students access and success that need research attention, but also how to support a diverse faculty so we see broad success.

7. I also really like that your 1st recommended strategy for GC#1 is qualitative study. I think qualitative analyses are sometimes pushed aside for quantitative studies but we need both because we get different types of information/insight from each.

8. I also really like how GC#2 looks at geoscience departments/programs as complex systems. That opens up doors for how to study programs with the goal of identifying essential components, positive and negative feedback and cause and effect outcomes. And developing a rubric for program health is a great idea.

9. GC#3 is well done too. A dense question (or pair of questions), but meaningful and important.

10. Can you highlight strategies that are short-term/small scale? Most look long-term/large scale, which may limit who can do that research. But I think there are opportunities for research at different scales.

11. WG#3 included some suggestions for important researchable questions under each of their Grand Challenges, in addition to their recommended strategies. That wasn’t a requirement in the format, but I think it is very effective. Please look at WG3’s draft chapter and see if you think that is a good way to go to help give more concrete examples of important research directions in growth of the instructor, programmatic env health?

12. Lastly, I wonder if the WG theme is misnamed? Should Growth of the instructor and Programmatic health be in the theme title instead of what we currently have? These seem more descriptive and better reflect the GCs.

12429:34872

Share edittextuser=1143 post_id=34872 initial_post_id=0 thread_id=12429

RE: Grand Challenge 1: This feels like a ground-zero kind of list of priorities, when we're not at ground zero anymore - the Early Career Faculty workshops which have been running for many years have generated several academic "generations" of young faculty who were instilled with the kind of appreciation of geoscience teaching and understanding of its value that we deem desirable. Why not interrogate this cohort specifically, to see how they are pursuing their professional development as educators, now in some cases many years post-event? This should provide a lot of baseline information to work from.

RE: all the Grand Challenges listed here: Nothing seems to be aimed at addressing the "great unwashed" in our field, who haven't attended the Early Career workshops (or predate them...), and who don't peruse the literature of geoscience or STEM education (but need to!). Some attention to practices/strategies to bring the uninitiated geoscience faculty in our field into a place where they are comfortable with and desire to engage with best practices in geoscience education would seem appropriate here. Are there such strategies out there? How well do they work? How scaleable are they?

RE: Grand Challenge 2: I'm not sure the term "departmental health" is accurate here. This is really about how friendly units are to the scholarship of education and how able/willing they are to put it into practice in their courses. That's a question of departmental/institutional culture. From an institutional standpoint, it might be very "healthy" for a department to be hostile to disinclined to pursue changes in instructional practices provided other measures (reseearch, graduate education, endowments) are strong. The question is more in how to identify and change departmental drivers to make a more scholarly approach to geoscience education more desirable/accessible.

12429:35151

Share edittextuser=936 post_id=35151 initial_post_id=0 thread_id=12429

I agree with many of Kristen's and Jeff's comments. Some specifics:

Jeff mentions the "great unwashed" - I was thinking about this in terms of motivation. GC #1 is about supporting continued growth, but a completely unmentioned component of this is understanding motivation to continued growth, or motivation to grow at all. There is a lot of research about barriers and motivations to faculty change that could be built on here, including:

Sunal, D. W., Hodges, J., Sunal, C. S., Whitaker, K. W., Freeman, L. M., Edwards, L., Johnston, R. A. and Odell, M. (2001), Teaching Science in Higher Education: Faculty Professional Development and Barriers to Change. School Science and Mathematics, 101: 246–257. doi:10.1111/j.1949-8594.2001.tb18027.x

Plus research from Ellen Iverson at SERC about the impacts of PD experiences that should be incorporated here.

In addition, NAGT has been running Traveling Workshop Programs for many years, and there is a lot of evaluation data from this program that could be useful.

GC #2: Minor change, but "comprised" is used incorrectly here in the first sentence. "Comprised of" should be replaced by "comprising" to be grammatically correct.

GC #3: This is an interesting question that I think has substantial variability across different types of institutions. Private R-1 institutions have people dedicated to these efforts (I suggest contacting Catherine Riihimaki at Princeton and Robyn Wright Dunbar at Stanford for two perspectives) while other types of institutions are more general. Again - Ellen Iverson has significant insight into this question.

12429:35226

Share edittextuser=888 post_id=35226 initial_post_id=0 thread_id=12429

I will try to add to rather than repeat comments by Kristen, Jeff, and Anne.

GC #1: This is definitely an important issue for the community. The Early Career Workshops have influenced many of us, but I wonder how many new geoscience faculty do not participate in those workshops or receive any significant teacher training prior to beginning their careers. What can be/is being done to encourage geoscience Ph.D.-granting geoscience departments to mandate that their students who plan to pursue an academic career must take some form of pre-career teacher training?

I like the proposed strategy of conducting case studies as they can help build a case for showing that pre- and early-career faculty would benefit from more training and support. Those studies can also shed important light on how faculty actually do and do not adopt instructional approaches that are novel to them (see Dolphin, 2016).

Future studies that intend to provide recommendations on promoting changes in how departments and individual faculty perceive their role as facilitators of student learning (especially those recommendations that are intended for faculty who might not feel a need to shift away from a lecture-centered approach) will likely benefit from collaborations with scholars in fields such as psychology, communication, and marketing.

Dolphin, G.R., 2016, Toward an understanding of “teaching in the making”: Explaining instructional decision making by analyzing a geology instructor’s use of metaphors: Geosphere, v. 12, no. 6, p. 1725–1743, doi:10.1130/GES01202.1.

12429:35358

Share edittextuser=5226 post_id=35358 initial_post_id=0 thread_id=12429

I like the connection (in GC1) that supporting “instructors' continuous progress in their teaching practice will determine to the degree to which we are able to produce citizens and geoscientists.” I like that this connection is made, and wonder if there are references that can help make this connection (or maybe this is another strategy?).
In the strategies for this GC, I’m sure the authors are thinking of faculty PD in other disciplines as models or as resources, but I think that could be explicit in GC1.
Proposed strategies for GC3 are great- I like the longitudinal studies (is even 10 years enough to go from GTA to tenured faculty?). I would like to see more here on GTA preparation for teaching as that may help set the stage for one’s approach to teaching as a career-long area of PD.
I like the suggestion that the Early Career workshops may provide a good opportunity to study the long term impact of those workshops and I wonder if Department Workshops may also be a place to look for the extent of departmental change. I appreciate the need to understand departmental culture and think that could be another strategy.

12429:35391

Share edittextuser=749 post_id=35391 initial_post_id=0 thread_id=12429

Hi all,

Anne Egger invited me to review and comment on this page. Since I have not been involved in all the work that led to the page, use my comments as appropriate.

Grand challenge #1: I used the the proposed methods/strategies of both case studies of instructors and interviews with geo ed community leaders to study adoption of active learning strategies within a departmental context for my dissertation (Iverson, 2016). So, my findings and conclusions may be useful as you embark on the studies related to both #1 and #2.

Grand challenge #2: some additional studies related to departmental culture and the role of collegiality that will be useful to your next steps:
Massy, W., Wilger, A., & Colbeck, C. L. (1994). Departmental cultures and teaching quality: Overcoming "hollowed" collegiality. Change: The Magazine of Higher Learning, 11-19.

Knight, P. T., & Trowler, P. R. (2000). Department-level cultures and the improvement of learning and teaching. Studies in Higher Education, 25(1), 69-83.

Hilborn, R. C., & Howes, R. H. (2003). Why many undergraduate physics programs are good but few are great. Retrieved 6/1/04, 2004

Hilborn, R. C., Howes, R. H., & Krane, K. S. (2003). Strategic Programs for Innovations in undergraduate Physics: Project Report. College Park, MD: American Association of Physics Teachers.

Grand challenge #3: In conjunction with the Workshop for Early Career Geoscience faculty and the past On the Cutting Edge program, SERC will be conducting a retrospective study of early career workshop participants. It might be useful to collaborate as you think about next steps for this area of study.

12429:35418

Share edittextuser=69 post_id=35418 initial_post_id=0 thread_id=12429

Join the Discussion


Log in to reply


« Previous Page