How Do Classroom Reforms Improve Learning?

Dr. Nicole James Shares Her Research on Evidence-Based Course Reform

Last week’s Chemistry department seminar, on September 9, was entitled "Assessing the impact of an evidence-based course reform: What type of learning is improved?" The seminar was presented by Dr. Nicole James, a Visiting Assistant Professor of Chemistry here at Reed. The seminar was a presentation of Dr. James’ work in studying how introductory chemistry courses can be reformed for greater student success and passion. 

The study was done in Fall 2019, during Dr. James’ Postdoctoral Fellowship at Northern Illinois University. The ultimate objective of the study was to determine how a deliberate practice-driven pedagogical reform of a large introductory chemistry lecture course changes how students learn, and particularly what kinds of learning are most improved. Determining how students learn interdisciplinary skills, as well as how such a mindset can be better promoted within the discipline of chemistry was also key to Dr. James’ research.

To conduct the study, there was one “reformed” class and two “unreformed” classes, which acted as controls against the reformed class. The two unreformed classes were taught by different professors with different styles in order to maximize the diversity of student instruction. The reformed class was called Section R, with the unreformed classes being called Sections UA and UB. 

The class itself was titled CHEM 110 for the purposes of the presentation. It holds around 700 students per year, with lecture sections comprising around 100-200 students each. It is a preparatory course for General Chemistry, satisfies requirements for tracks such as nursing, kinesthesiology, engineering, etc, and does not have a mandatory lab section. The study was conducted exclusively on the lecture component of the course. CHEM 110 was also identified as having among the highest DWF rates, or fail rate, of any class within the institution: about 50%.

Professor James’ primary framework for the reformed course involved the use of what she called “deliberate practice,” in which the student is given assignments that attempt to create a flow from motivation, to practice, to feedback. The example Dr. James gave first involved “why does this matter?” intros which create both an explicit link to real life and academic goals as well as provide pedagogical transparency. In other words, the student understands why they are being asked to learn a particular concept or complete a particular assignment. This functioned as the “motivation” aspect of the framework. The “practice” portion involved pre-class assignments, active practice in pairs during class time, and regular short homework assignments due before the next class. Finally, the “feedback” portion attempted to give students immediate satisfaction and transparent, easily accessible, goal-oriented feedback through things like group work, auto-graded online quizzes, and frequent office hours as well as prompt email responses on the professor’s part. Ultimately, Dr. James’ approach emphasized learning through practice, and then through feedback. Many of these ideas were implemented into the reformed class section. 

The three sections were compared within three categories of criteria: course design, class facilitation, and student outcomes. Course design involved homework and exams, with a specific focus on exam type. Each of the three sections had four exams plus a final. The reformed section had a schedule of small, frequent assignments three times per week, with the unreformed sections having larger assignments once per week. The types of exam questions varied slightly as well between the reformed and unreformed sections. 

Course facilitation was measured by recording each section four times, and then assessing what both the students and professors were doing during said classes in two-minute intervals. Section UB consisted of the students only listening to the professor, as the class was entirely oral lecturing, with the professor occasionally writing on the board for visual aid. Section UA was slightly more involved, as students were recorded asking questions to the professor and vice-versa, creating a noticeably more active classroom environment. Section R showed the greatest diversity in class facilitation, with the professor continuously asking students to remain engaged by posing questions and guiding group activities.  

Student outcomes were measured in several ways. Firstly, students were surveyed on their attitudes towards chemistry as an academic discipline in the first and last week of classes. In both unreformed sections, student attitudes dropped, signifying an overall unsatisfactory experience. Some students suggested that their experience was so poor, they reconsidered their major. The reformed section, conversely, showed a definite improvement in student attitudes, as many in section R showed greater enthusiasm for, and understanding of, chemistry. All three sections had nearly identical first week attitudes. Specific opinions from students were collected via focus groups. 

Students in the reformed section also had higher academic performance, averaging higher grades than their peers in the unreformed sections. Section UA performed the worst grade-wise. However, none of the sections showed noticeable improvement on Chemistry Concept Inventory (CCI) assessments. Dr. James suggested a change in the curriculum could be needed to account for this. 

Dr. James summarized her findings as such: the reformed class section improved student attitudes towards chemistry, improved scores on shared item exams, and improved course grades, but it did not change interaction with CCI assessments, which are largely skills not emphasized in the regular course assessments. Ultimately, Dr. James concluded that while a pedagogical reform can and does improve student learning, it can only affect how much students learn and how they learn it.What students learn is affected by the curriculum, which was unchanged between all three sections. It will be fascinating to see how Dr. James’ work in this area will affect not just chemistry courses here at Reed, but the learning philosophy as a whole.