Supported Site Florida International University: Assessment

Content Assessment

The FIU physics department offers introductory physics I in lecture classes with an optional laboratory section. Students enrolling in labs may take either calculus-based (PHY2048) and algebra-based (PHY2053) lecture courses. In Spring 2008, the FIU PhysTEC team implemented reformed labs using the University of Maryland (Scherr and Elby) epistemologically-based tutorials and interactive lecture demonstrations, also known as the Open Source Tutorials. Lab sections lasted 3 hours, so both the tutorial and interactive lecture demonstrations were combined to form the class activities. TAs were responsible for the labs, and were assisted by one or two LAs. Data results from the Force Concept Inventory taken as pre/post surveys in the lecture sections. No reforms were implemented in the lectures. In Spring 2009, the FIU PhysTEC model for lab reform in the introductory physics course were adopted for all Physics I labs (calculus and non-calculus based physics courses share the same labs). In Fall 2009 and Spring 2010 the lab reform continued to be implemented in all Physics I labs.

Spring 2008 Reformed Lab FCI Results - Reform labs were implemented in 6 of 14 lab sections (cap of 24).

Semester

Number of matched students

Pre-test

average (%)

Post-test average (%)

Average normalized gain g*

Standard error of g**

PHY 2048L Reform

38

25.8

50.0

0.33

.030

PHY 2048L Traditional

63

24.1

37.0

0.18

.020

Fall 2008 Reformed Lab FCI Results - Reform labs were implemented in 10 of 24 lab sections (cap of 24).

Semester

Number of matched students

Pre-test

average (%)

Post-test average (%)

Average normalized gain g*

Standard error of g**

PHY 2048L Reform

83

27.1

50.6

0.33

.025

PHY 2048L

Traditional

133

29.8

49.2

0.28

.024

Spring 2009 Lab FCI Results - Reform labs were implemented in 14 of 14 lab sections (cap of 30).

Semester

Number of matched students

Pre-test

average (%)

Post-test average (%)

Average normalized gain g*

Standard error of g**

PHY 2048L All Labs Reformed

87

35.04

59.9

0.385

.024

3-Year Summary of Lab Assessment Results.

During the PhysTEC project we have collected FCI data in the lecture sections of all 2048 classes, and we have collected CLASS data in the reformed lab sections of all labs. The data are summarized in the table and figures below.

 

Reformed Lab

(N=380 for FCI,

N=791 for CLASS)

Traditional Lab

(N=493 for FCI,

N=28 for CLASS)

No Lab

(N= 387 for FCI,

N=32 for CLASS)

 

Mean

SE

Mean

SE

Mean

SE

FCI Pre

31.4%

0.841%

29.8%

0.686%

35.9%

0.843%

FCI Post

55.6%

1.060%

45.7%

0.878%

46.6%

1.020%

Raw Gain

24.2%

0.768%

15.9%

0.616%

10.7%

0.766%

Norm’d Gain

37.0%

1.290%

23.5%

1.030%

16.5%

1.640%

CLASS Shift

-2.55

1.13

-12.07

4.73

5.55

4.39

Lab Grade

3.77

0.025

3.44

0.037

-

-

Course Grade

2.59

0.052

2.57

0.049

1.98

0.064


Fall 2009/Spring 2010 Reformed Physics II Lab Results - Reform labs were implemented in 4 of 9 labs in Fall 2009 and 8 of 15 lab sections in Spring 2010 (24 student cap)

The reform of Physics II labs began in Fall 2009 using ISLE-based labs. We collected data on the initial reform of 2049 labs by collecting CLASS data in the labs. We attempted to collect BEMA data in the lectures, but had minimal participation by faculty.

 

Reformed Lab (N=220)

Trad Lab (N=120)

 

Mean

SE

Mean

SE

CLASS Shift

-1.49

2.61

-10.20

3.87

Lab Grade

3.75

0.02

3.68

0.055

         
         
         
         
 

Effect Size

Reform Lab to Trad Lab

 

-0.22

0.01

-0.44

 
         
         

*Average normalized gain g =

**Standard error =

Measures of Disposition

MPEX 2 – Spring 2008 & Fall 2008

In an effort to measure possible attitudinal shifts, the Maryland Physics Expectation Survey 2 (MPEX 2) was administered to reformed lab students [1]. This survey measures students’ attitudes and beliefs about physics and learning physics. Within the survey, clusters of questions characterize three main epistemological beliefs in introductory physics: coherence, concepts, independence [2].

Definitions:

Coherence - The extent to which the student sees physics knowledge as coherent and sensible as opposed to a bunch of disconnected pieces [2].

Concepts - The extent to which students see concepts as the substance of physics -- as opposed to thinking of them as mere cues for which formulas to use [2].

Independence - The extent to which the student sees learning physics as a matter of constructing her own understanding rather than absorbing knowledge from authority [2].

MPEX2 was given as pre- and post- surveys to all students in the introductory physics lab sections that were subject to the FIU PhysTEC reform efforts. In the Spring 2008 semester, there were 73 matched surveys. In Fall 2008, there were 194 matched surveys.

Figures 1 and 2 present percent favorable and unfavorable pre- and post- results for the MPEX 2 coherence, concepts, and independence clusters for each of the two semesters. Also, for each cluster, a favorability score, % favorable - % unfavorable, was calculated for each student. This score accounts for decreases in unfavorable responses as well as increases in favorable responses. Matched t-test analyses of pre and post favorability scores show no significant differences in any of the three main clusters.

Although there were no significant positive changes in favorability scores for any of the three main epistemological characteristics describing student attitudes and beliefs about physics, it is generally accepted that this is a positive result as traditional and even some reform experiences result in negative shifts. The lack of a significant negative shift means that FIU’s reform treatment did not result in students adopting more novice-like, as opposed to expert-like, beliefs.


CLASS Results for Spring 2009-07-20

In Spring 2009, FIU PhysTEC moved from the MPEX 2 to the Colorado Learning Attitudes about Science Survey (CLASS) as a means of measuring student dispositions and attitudes about science and learning science. Physics education researchers at the University of Colorado Boulder built on the work of previous attitude surveys, such as MPEX 2, to include and refine questions about student attitudes and beliefs that have been found to be important in education research [3]. FIU decided to replace the MPEX 2 with the CLASS as our Learning Assistants (LA) program is founded upon and similar to the LA program at Boulder, so this allows us to more readily compare the results of our work.

There were 225 matched surveys for Spring 2009. There were significant shifts in three of the CLASS categories. See Figure 3. A significant positive shift was evident in Conceptual Connection (shift = 4.1, Std Err = 1.9) categories. This is exceptional in the literature. As well, a negative shift in general attitudes toward problem solving (PS General, shift = 2.6, Std Err = 1.2) is interesting as students may feel that “problems” are the formula-based, number-centric items found in typical homework assignments and tests. Given that there was also a considerably larger positive shift in the Conceptual Connection category, it would make sense for students to become more adverse to solving these traditional type “problems.” However, there was also a negative shift in the Sense Making category. There was a loss in the favorable attitudes as well as a gain in the unfavorable. There were no significant shifts in any of the other categories. This, as with the MPEX 2, is considered a positive result as students are not becoming more novice-like in their attitudes.

References.

  1. Elby, A. (2001). American Journal of Physics, 69,S54.
  2. Hammer, D. (1994). Cognition and Instruction, 12(2), 151-183.
  3. University of Colorado Physics Education Research Group at Colorado website. http://www.colorado.edu/sei/class/