Assessment is often treated as an afterthought, Dr. Meltem Alemdar remarks, tacked on to the end of a project to validate the results. For the NSF-IGE grant-funded project “Integrating Team Science into the STEM Graduate Training Experience,” this is, thankfully, not the case. As Alemdar emphasizes, “we’re part of the leadership team, which is unique to this project. Assessment is being recognized as integral to the process.” Mr. Christopher Cappelli adds, “We’re part of every meeting. It’s a collaborative process.”
Alemdar and Cappelli enjoy doing assessment work with the “Integrating Team Science” project, which seeks to train STEM graduate students in that key 21st-century skill, collaboration. The need for the project was clear, they explain, because there is a clear gap in graduate training in effective team dynamics. And “putting the grant together was a smooth process,” Alemdar says, “because we’re working with people who are thinking about how to work well in teams.” Best of all, her co-PIs recognize the enormous value of assessment throughout the entire process of researching, strategizing, designing, developing, implementing, and revising curricular materials.
Assessment is crucial to research in effective team dynamics because it determines which aspects of the curriculum are producing long-term change, and which aspects need improvement. Drawing on their background in educational policy, Alemdar and Cappelli use qualitative and quantitative research methods (surveys, focus groups, and interviews, for example) in order to gather “formative and summative” feedback from workshop participants. The assessment process involves developing and adapting program-specific research instruments that will accurately measure graduate students’ reactions to the training modules. Balancing breadth and depth, Alemdar and Cappelli look for data that reveals patterns in participants’ knowledge-retention and readiness to be involved in team science. They want to find out what’s confusing, how the sequencing could be refigured, and whether there could be better examples of the concepts.
Because the “Integrating Team Science” team incorporates assessment at every stage of the project, the curricular design process is iterative rather than linear, moving from feedback, to revision, to implementation, and back to feedback. Even before the team started developing curricular materials, Alemdar and Cappelli arranged focus groups with graduate students. Understanding graduate students and the team challenges they were facing helped ensure that the first team science workshops would immediately be relevant to grad students’ actual needs and contexts. For instance, they discovered that reactions to conflict-management training differed depending on the field, because of the diversity of grad student experience.
As Cappelli puts it, “assessment for this project is purposefully cyclical – every rendition of the Team Science curriculum is impacted by direct feedback, so that the workshop becomes more and more applicable to grad students.” This innovative cyclical process continuously strengthens the materials. And because academics appreciate research-based practice, Alemdar and Cappelli can also fold the new assessment data back into the workshops by providing it for participants.
Always team players, they have some helpful advice for others who want to do this type of assessment work:
“Do an extensive literature review with a strong grounding in theories and contexts,” Alemdar suggests. “You don’t know what you’ll find.”
“Be adaptable,” Cappelli adds. “It doesn’t always go as planned – expect surprises.”
This is the second article in a two-part series featuring Meltem Alemdar and Christopher Cappelli. Learn more about their collaborative interdisciplinary research methods here.