red Supported Site Cal Poly-San Luis Obispo: Assessment

First quarter introductory physics, calculus-based: mechanics
FMCE Data by Quarter and Instructor
Year Quarter Instructor Number Pre Post Gain
2005 Winter Instructor 1* 28 28.0 60.8 0.46
2005 Winter Instructor 1 28 29.0 68.3 0.55
2005 Winter Instructor 2 30 23.9 58.2 0.45
2005 Winter Instructor 2 31 25.1 62.4 0.50
2005 Winter Instructor 3 22 26.9 61.5 0.47
             
2005 Spring Instructor 3 30 31.0 68.6 0.55
2005 Spring Instructor 3 23 25.0 65.9 0.55
2005 Spring Instructor 4 24 27.4 58.8 0.43
2005 Spring Instructor 4 14 21.0 57.2 0.46
2005 Spring Instructor 4 20 25.7 54.7 0.39
             
2006 Winter Instructor 5 26 23.9 50.6 0.35
2006 Winter Instructor 5 22 32.0 64.7 0.48
2006 Winter Instructor 5 27 26.0 57.5 0.43
2006 Winter TIR 31 30.5 69.6 0.56
2006 Winter Instructor 3 27 35.4 66.2 0.48

* This particular class has unmatched pre and post-tests. There were no names on the post tests. All other data is matched.

We have taught this course and given the FMCE in three different quarters. The Pretest is administered the first day of class and the Posttest is administered the last week of the course (after all topics in the FMCE have been covered). The hake gain on the FMCE for all of the courses was 0.43. The individual sections ranged from 0.35 to 0.55. The FMCE gains for each section vary within the range for a given quarter and for a given instructor. For example, Instructor 5’s sections had gains of 0.35, 0.48, and 0.43, which spans almost the entire range of gain scores. Thus, the gains are independent of the instructor and reflective of the underlying structure that we have developed. Each section has gains significantly higher than the gains of 0.2 reported by Hake for traditionally taught courses.

We can compare these FMCE results with our Studio Physics course (another reformed course we taught in the past) and our traditional sections.

Comparison of three approaches to introductory mechanics—
FCME scores
Year Course Pre Post Gain Number
2005-2006 Active learning PhysTEC course (described in Part II) 26.0 57.5 0.43 383
1998-2001 Studio Physics 31.9 72.1 0.60 966
1998-2001 Traditional Physics 35.5 53.3 0.30 300

The Hake gain is 0.30 for the traditional physics courses, 0.43 for the PhysTEC Physics Course, and 0.60 for the Studio Physics course. The PhysTEC reformed course falls somewhere between traditionally taught courses and our Studio Physics course. This isn’t that surprising, as the Studio Physics course had students doing more reformed style labs that are only demonstrations in the corresponding PhysTEC course. It does give us something to shoot for though. Probably the most important thing to address is student engagement. When students were in the Studio Physics course they were in a group of three students engaged in doing the activities. In the PhysTEC course the students are making predictions about activities in the large group setting. Nominally they are working with their neighbors, but because they are not doing the activity it is easier for them to be disengaged. We need to develop some structures that keep students engaged.

This raises the question of why don’t we teach all of our students in the Studio format? First, we don’t have the space to teach all of our students in this format. Second, there isn’t support in the department to teach in this format.

Third quarter introductory physics, calculus-based: electricity & magnetism
CESM Data by Quarter and Instructor
Year quarter Instructor Number   Pre Post Gain  
2005 Fall Instructor 1 34   27.4 56.3 0.40  
2005 Fall Instructor 2 29   29.6 61.5 0.45  
                 
2005 Spring Instructor 2 54   33.0 57.5 0.36  
2005 Spring Instructor 3 59   31.8 55.6 0.35  
                 
2006 Fall *** 92   28.6 61.6 0.46  

***This data is unmatched at this point, but it appears that this quarter is at least in line with the other quarters.

We have taught this course and given the CSEM in three different quarters. The Pretest is administered the first day of class and the Posttest is administered in the last lab day of the course, usually the last week of the course. Unfortunately, we are covering magnetism the last week, which is material tested by the CSEM. These are matched pretest and posttest scores.

There is not as much reported about the CSEM as the FMCE. However, Maloney et al in their paper “Surveying students’ conceptual knowledge of electricity and magnetism” reported that for the CESM the average pretest score was 31% and the average posttest score was 47%. For our PhysTEC introductory electricity & magnetism course, the average pretest score was 31% (on par with the reported average) and the average posttest score was 58% (10% higher than the reported average). The PhysTEC introductory electricity & magnetism course score on the posttest was 10% higher than the reported average. This is certainly a step in the right direction.

Optics Survey: Physics for Elementary Teachers
  Pre (%) Post (%) Gain Number
Fall 2005 38.4 56.6 0.30 124
Winter 2006 40.0 61.5 0.36 30
Spring 2006 36.3 62.0 0.40 25
Fall 2006 38.7 66.0 0.44 98

 

Heat and Temperature Concept Evaluation (HTCE) by quarter: Physics for Elementary Teachers
  Pre (%) Post (%) Gain Number
Fall 2005 39.1 46.4 0.12 114
Winter 2006 45.7 52.7 0.13 30
Spring 2006 Not enough data      
Fall 2006 44.0 52.3 0.15 93

To evaluate the course we gave our an Optics survey to evaluate the light and color portion of the course and the Heat and Temperature Concept Evaluation to evaluate the heat and temperature portion of the course. The Optics Survey was given the first day of the class and the posttest was given one or two weeks after the optics exam, usually in the seventh week. All of the data are matched. Students moved from 40% on the pretest score to 60% in the posttest.

The HTCE pretest was taken the week prior to starting the Heat & Temperature unit. The students took the test outside of class online. The posttest was taken the last day in class, except in the spring quarter, where students took it online; however, only 7 students took it, so there isn’t enough data to analyze. We will continue to take the posttest on the last day of class. The results are not as good as for the Optics survey. Some of this is expected because six of the twenty-seven questions deal with rates, which we don’t cover in class. However, this isn’t the whole story. Students don’t do that well on the Energy & Temperature question and the mixing questions either. This gives us some topics to work on for next year.