PLACEMENT INTO FIRST COLLEGE MATHEMATICS COURSE: A
COMPARISON OF THE RESULTS OF THE MICHIGAN STATE UNIVERSITY
PROCTORED MATHEMATICS PLACEMENT EXAMINATION AND THE
UNPROCTORED MATHEMATICS PLACEMENT EXAMINATION
By
Samuel Drake

1

A DISSERTATION
Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of
DOCTOR OF PHILOSOPHY
Measurement and Quantitative Methods
2010

ABSTRACT
PLACEMENT INTO FIRST COLLEGE MATHEMATICS COURSE: A
COMPARISON OF THE RESULTS OF THE MICHIGAN STATE UNIVERSITY
PROCTORED MATHEMATICS PLACEMENT EXAMINATION AND THE
UNPROCTORED MATHEMATICS PLACEMENT EXAMINATION
By
Samuel Drake
The primary purpose of this study was to compare the results of the Michigan
State University (MSU) unproctored examination to the results of the proctored
examination. Both examinations are used to determine whether first time freshmen at
MSU are ready for a standard mathematics course or if a remedial course is necessary. In
addition to producing higher placement examination scores, the unproctored examination

ii

placed students into higher level courses and a larger proportion of student who was
placed with the proctored examination enrolled in a course at a level lower than the
course in which they were placed. Therefore, the first conclusion was that the
unproctored examination produced more inappropriate placements than the proctored
examination.
The second conclusion was that when the mathematics placement examination
was considered alone, it was a significant predictor of the log odds of success in
Intermediate Algebra (MTH1825), College Algebra (MTH103), and Calculus 1
(MTH132). When ACT Mathematics score, their high school GPA, the type of exam
used for placement, whether a student enrolled in mathematics during his or her senior
year of high school, and the last high school mathematics course taken were considered,
the prediction of the log odds of success was improved for each of these courses. The
additional variables improved the “hit rate” of the model containing only placement
ii

examination score. In addition, the additional variables decreased the false positive rate
of the model containing placement examination only. Therefore, the placement
examination alone is not sufficient for placing students into their first college
mathematics course.
Thirdly, students placed into remedial mathematics less often with the
unproctored examination. In fact, the odds of placing into one of MSU‟s non-remedial
mathematics courses with the proctored examination was approximately 1.5 times greater
than the odds of placing into a non-remedial mathematics course with the unproctored
examination. Therefore, placement into remedial mathematics was dependent on the
type of examination used for placement.
Finally, there were students who enrolled in courses lower than the level in which

iii

they were placed. For example, approximately 30.8% of the students who enrolled in
MTH103 and were placed with the proctored examination were eligible to enroll in a
higher level course. Approximately 45.9% of the students who enrolled in MTH103 and
were placed with the unproctored examination were eligible to enroll in a higher level
course. This difference in percentages was significant.
It is important to the validity of the placement examination as well as the
comparability of the proctored and unproctored placement examinations to determine
why students enroll in courses lower than the level in which they were placed. Study
limitations are discussed and suggestions for future research are given.

iii

Acknowledgements
There were so many people who have supported me during the entire doctoral
process. First of all, I would like to thank Dr. Sharif Shakrani for agreeing to serve as my
dissertation advisor. Despite his busy schedule, he made time for me. I appreciate the
careful reading of my many drafts as well as the advice and encouragement that he
provided. In the same vein, I would like to thank my dissertation committee – Dr. Mark
Reckase, Dr. Tenko Raykov, and Dr. Robert Floden. Without their support, I could not
have finished this process.
I would like to thank Dr. Kimberly Maier for taking the time to speak with me
about my program plan after my original advisor left the university. I appreciate the
guidance that she provided up until I choose my dissertation advisor.

iv

I would like to thank my friends and colleagues Dr. Fred Barton, Dr. Bonita
Curry, Jennifer Malinowski, John Dowell, Jessica Deforest and the rest of the staff of the
Learning Resources Center and Undergraduate University Division for all of their
support.
The final thank you is for my best friend and wife, Laree, for her constant prayers
and unwavering faith in me. I am truly blessed to have her in my life.

iv

Table of Contents
List of Tables ................................................................................................................... viii
List of Figures .................................................................................................................. xiii
CHAPTER 1 ....................................................................................................................... 1
STATEMENT OF THE PROBLEM .................................................................................. 1
Introduction ..................................................................................................................... 1
Unproctored Placement Examinations ............................................................................ 4
Purpose of the Study ....................................................................................................... 6
Research Questions ......................................................................................................... 6

v

CHAPTER 2 ....................................................................................................................... 8
LITERATURE REVIEW ................................................................................................... 8
Structure of the chapter ................................................................................................... 8
Selecting a Mathematics Placement Test ........................................................................ 9
Pre-college Variables Affecting Mathematics Placement ............................................. 11
Predictors of Success in First College Mathematics Course ......................................... 14
Validating Placement Examinations ............................................................................. 17
Unproctored and Proctored Examinations .................................................................... 22
Mathematics Placement Procedures .............................................................................. 25
MSU Mathematics Placement Examination ................................................................. 27
Description of the Michigan State University Mathematics Placement Examination .. 28
Placement ...................................................................................................................... 30
Who Must Take the Mathematics Placement Examination? ........................................ 32
Summary ....................................................................................................................... 32
CHAPTER 3 ..................................................................................................................... 34
METHOD ......................................................................................................................... 34
Design............................................................................................................................ 34
Sample ........................................................................................................................... 35
Data Collection .............................................................................................................. 37
Letter Grade Conversion ............................................................................................... 37
Definition of Pre-college Characteristic Variables ....................................................... 39
Definition of College Variables .................................................................................... 42
Demographic Variables ................................................................................................. 43
Analysis ......................................................................................................................... 43
Multiple Linear Regression (MLR)............................................................................... 43
Assumptions .................................................................................................................. 44
Model Fit ....................................................................................................................... 45
Binary Logistic Regression (BLR) ................................................................................ 46
Model Fit ....................................................................................................................... 47
Two Independent Sample T-Test .................................................................................. 49
Assumptions .................................................................................................................. 49
Pearson Chi-Square Test of Independence.................................................................... 49
v

Assumptions .................................................................................................................. 50
Effect Size Measures ..................................................................................................... 50
Question 1 ..................................................................................................................... 53
Question 2 ..................................................................................................................... 55
Multiple Hypothesis Testing ......................................................................................... 57
Question 3 ..................................................................................................................... 60
Question 4 ..................................................................................................................... 62

vi

CHAPTER 4 ..................................................................................................................... 64
RESULTS ......................................................................................................................... 64
Introduction ................................................................................................................... 64
Question 1 ..................................................................................................................... 64
Homoscedastiscity Assumption................................................................................. 74
Multiple Linear Regression (MLR)............................................................................... 79
Normality Assumption. ............................................................................................. 79
Multicollinearity Assumption .................................................................................... 83
Multiple Linear Regression Model ............................................................................... 83
Residual Analysis of the Multiple Linear Regression Model ....................................... 84
Question 2 ..................................................................................................................... 87
Reliability ...................................................................................................................... 87
Correlation for Combined Data (Proctored data and Unproctored Data) ..................... 88
Correlations for Proctored Data .................................................................................... 91
Correlations for Unproctored Data ................................................................................ 93
Placement into Intermediate Algebra (MTH1825) ....................................................... 96
Multiple Linear Regression Model for the Combined Data .......................................... 97
Residual Analysis of Overall Regression Model .......................................................... 98
Multiple Linear Regression Model for the Proctored Data ......................................... 102
Multiple Linear Regression Model for the Unproctored Data .................................... 107
Success Rate for Combined Data ................................................................................ 113
Success in Intermediate Algebra (MTH1825) ............................................................ 114
Success in College Algebra (MTH103) ...................................................................... 124
Success in College Algebra and Trigonometry (MTH116) ........................................ 125
Success in Survey of Calculus 1 (MTH124) ............................................................... 126
Success in Calculus I (MTH132) ................................................................................ 126
Question 3 ................................................................................................................... 127
Predicting Success in Intermediate Algebra (MTH1825) ........................................... 128
Predicting Success in College Algebra (MTH103) ..................................................... 133
Predicting Success in College Algebra and Trigonometry (MTH116) ....................... 137
Predicting Success in Survey of Calculus I (MTH124) .............................................. 141
Predicting Success in Calculus 1( MTH132) .............................................................. 144
Question 4 ................................................................................................................... 148
Intermediate Algebra (MTH1825) .............................................................................. 148
College Algebra (MTH103) ........................................................................................ 152
Finite Mathematics (MTH110) ................................................................................... 157
College Algebra and Trigonometry (MTH116) .......................................................... 158
Survey of Calculus 1 (MTH124) ................................................................................. 161
vi

Calculus 1 (MTH132) ................................................................................................. 163
CHAPTER 5 ................................................................................................................... 167
DISCUSSION ................................................................................................................. 167
Conclusion 1................................................................................................................ 167
Conclusion 2................................................................................................................ 168
Conclusion 3................................................................................................................ 169
Limitations .................................................................................................................. 170
Future Research ........................................................................................................... 172
Summary ..................................................................................................................... 173
Appendix ......................................................................................................................... 175
Description of Mathematics Courses .......................................................................... 176
Bibliography ................................................................................................................... 177

vii

vii

List of Tables

TABLE 1:

TABLE 2:

LETTER GRADE CONVERSION CHART USED TO COMPUTE
STUDENTS’ HIGH SCHOOL MATHEMATICS GPA ............................ 39

TABLE 3:

EXAMPLE OF A 2 X 2 CONTINGENCY TABLE ................................... 52

TABLE 4:

DESCRIPTIVE STATISTICS FOR ACT MATH SCORES, HIGH SCHOOL
GPA, HIGH SCHOOL MATHEMATICS GPA, AND MATHEMATICS
PLACEMENT EXAMINATION FOR PROCTORED AND
UNPROCTORED GROUP ...................................................................... 65

TABLE 5:

viii

DISTRIBUTION OF RACE AND GENDER FOR PROCTORED AND
UNPROCTORED DATA.......................................................................... 36

TEST OF THE EQUALITY OF VARIANCES AND EQUALITY OF
MEANS..................................................................................................... 75

TABLE 6:

2 X 2 CONTINGENCY TABLE FOR SENIOR YEAR MATH BY TYPE OF
EXAMINATION ....................................................................................... 76

TABLE 7:

4 X 2 CONTINGENCY TABLE OF LAST HIGH SCHOOL
MATHEMATICS COURSE BY TYPE OF EXAMINATION ................... 77

TABLE 8:

CORRELATION MATRIX FOR ACTM, HSGPA, AND TYPE ................ 83

TABLE 9:

MULTIPLE LINEAR REGRESSION RESULTS ...................................... 84

TABLE 10:

RELIABILITY ESTIMATES OF THE OVERALL EXAMINATION,
THE PROCTORED EXAMINATION, AND THE UNPROCTORED
EXAMINATION ..................................................................................... 87

TABLE 11:

CORRELATION MATRIX FOR THE COMBINED DATA (PROCTORED
AND UNPROCTORED) .......................................................................... 90

TABLE 12:

CORRELATION MATRIX FOR THE PROCTORED DATA ................... 92

TABLE 13:

CORRELATION MATRIX FOR UNPROCTORED DATA ...................... 94

TABLE 14:

P-VALUES FOR THE TEST OF EQUALITY OF CORRELATIONS ...... 95

TABLE 15:

CROSSTABULATION OF TYPE OF COURSE BY TYPE OF EXAM .... 96

viii

TABLE 16:

TABLE 17:

SUMMARY OF MULTIPLE LINEAR REGRESSION COEFFICIENTS
USED TO PREDICT PLACEMENT USING THE PROCTORED
EXAMINATION (N=450) .................................................................... 102

TABLE 18:

SUMMARY OF MULTIPLE LINEAR REGRESSION COEFFICIENTS
USED TO PREDICT PLACEMENT USING THE UNPROCTORED
EXAMINATION (N=909) ...................................................................... 108

TABLE 19:

2 X 2 CONTINGENCY TABLE OF MATH COURSE OUTCOME BY
TYPE OF EXAMINATION..................................................................... 113

TABLE 20:

2 X 2 CONTINGENCY TABLE OF MTH1825 COURSE OUTCOME BY
TYPE OF EXAMINATION..................................................................... 114

TABLE 21:

ix

RESULTS OF MULTIPLE LINEAR REGRESSION COEFFICIENTS USE
TO PREDICT MATHEMATICS PLACEMENT EXAMINATION SCORES
(N=1304) ................................................................................................. 98

2 X 2 CONTINGENCY TABLE OF COURSE OUTCOME BY TYPE OF
EXAMINATION FOR THE WHITE STUDENTS ENROLLED IN
MTH1825 ............................................................................................... 115

TABLE 22:

2 X 2 CONTINGENCY TABLE OF COURSE OUTCOME BY TYPE OF
EXAMINATION FOR THE BLACK STUDENTS ENROLLED IN
MTH1825 ............................................................................................... 117

TABLE 23:

2 X 2 CONTINGENCY TABLE OF COURSE OUTCOME BY TYPE OF
EXAMINATION FOR THE MALE STUDENTS ENROLLED IN
MTH1825 ............................................................................................... 118

TABLE 24:

2 X 2 CONTINGENCY TABLE OF COURSE OUTCOME BY TYPE OF
EXAMINATION FOR THE FEMALE STUDENTS ENROLLED IN
MTH1825 ............................................................................................... 119

TABLE 25:

2 X 2 CONTINGENCY TABLE OF COURSE OUTCOME BY TYPE OF
EXAMINATION FOR STUDENTS WHO WERE NOT ENROLLED IN
MATH DURING SENIOR YEAR OF HIGH SCHOOL AND ENROLLED
IN MTH1825 .......................................................................................... 120

TABLE 26:

2 X 2 CONTINGENCY TABLE OF MTH1825 COURSE OUTCOME BY
TYPE OF EXAMINATION FOR STUDENTS WHO WERE ENROLLED IN
MATH DURING SENIOR YEAR OF HIGH SCHOOL ......................... 121

ix

TABLE 27:

TABLE 28:

2 X 2 CONTINGENCY TABLE OF MTH1825 COURSE OUTCOME BY
TYPE OF EXAMINATION FOR THE STUDENTS WHOSE LAST HIGH
SCHOOL MATH COURSE WAS ALGEBRA I, ALGEBRA II, OR
GEOMETRY........................................................................................... 123

TABLE 29:

2 X 2 CONTINGENCY TABLE OF MTH1825 COURSE OUTCOME BY
TYPE OF EXAMINATION FOR THE STUDENTS WHOSE LAST HIGH
SCHOOL MATH COURSE WAS STATISTICS OR PROBABILITY ...... 124

TABLE 30:

2 X 2 CONTINGENCY TABLE OF MTH103 COURSE OUTCOME BY
TYPE OF EXAMINATION..................................................................... 125

TABLE 31:

2 X 2 CONTINGENCY TABLE OF MATH 116 COURSE OUTCOME BY
TYPE OF EXAMINATION..................................................................... 125

TABLE 32:

x

2 X 2 CONTINGENCY TABLE OF THE MTH1825 COURSE
OUTCOME BY TYPE OF EXAMINATION FOR STUDENTS WHOSE
LAST HIGH SCHOOL MATH COURSE WAS CALCULUS, PRECALCULUS, OR TRIGONOMETRY IN HIGH SCHOOL ................ 122

2 X 2 CONTINGENCY TABLE OF MTH124 COURSE OUTCOME BY
TYPE OF EXAMINATION..................................................................... 126

TABLE 33:

2 X 2 CONTINGENCY TABLE OF MTH132 COURSE OUTCOME BY
TYPE OF EXAMINATION .................................................................. 127

TABLE 34:

SUMMARY OF LOGISTIC REGRESSION RESULTS OF MODEL 1 .. 128

TABLE 35:

CLASSIFICATION TABLE CORRESPONDING TO MODEL 1 .......... 129

TABLE 36:

SUMMARY OF LOGISTIC REGRESSION COEFFICIENTS FOR
MODEL 2 ............................................................................................... 130

TABLE 37:

CLASSIFICATION TABLE FOR MODEL 2 ......................................... 133

TABLE 38:

SUMMARY OF LOGISTIC REGRESSION RESULTS FOR
MODEL 3 ............................................................................................... 134

TABLE 39:

CLASSIFICATION TABLE CORRESPONDING TO MODEL 3 .......... 134

TABLE 40:

SUMMARY OF LOGISTIC REGRESSION RESULTS FOR
MODEL 4 ............................................................................................... 136

TABLE 41:

CLASSIFICATION TABLE CORRESPONDING TO MODEL 4 .......... 137

x

TABLE 42:

TABLE 43:

CLASSIFICATION TABLE CORRESPONDING TO THE MODEL 5 .. 138

TABLE 44:

LOGISTIC REGRESSION RESULTS FOR MODEL 6.......................... 140

TABLE 45:

CLASSIFICATION TABLE CORRESPONDING TO MODEL 6 ...... 141

TABLE 46:

LOGISTIC REGRESSION RESULTS FOR MODEL 7 ............ ERROR!
BOOKMARK NOT DEFINED.

TABLE 47:

CLASSIFICATION TABLE CORRESPONDING TO MODEL 7 .......... 142

TABLE 48:

LOGISTIC REGRESSION RESULTS FOR MODEL 8.......................... 143

TABLE 49:

CLASSIFICATION TABLE CORRESPONDING TO MODEL 8 .......... 144

TABLE 50:

xi

SUMMARY OF LOGISTIC REGRESSION RESULTS FOR
MODEL 5 ............................................................................................... 138

SUMMARY OF LOGISTIC REGRESSION RESULTS FOR
MODEL 9 ............................................................................................... 145

TABLE 51:

CLASSIFICATION TABLE FOR MODEL 9 ......................................... 145

TABLE 52:

LOGISTIC REGRESSION RESULTS FOR MODEL 10........................ 147

TABLE 53:

CLASSIFICATION TABLE FOR MODEL 10 ....................................... 148

TABLE 54:

CROSSTABULATION OF GRADES IN INTERMEDIATE ALGEBRA BY
PLACEMENT EXAMINATION SCORE ................................................ 150

TABLE 55:

STUDENTS WHO PLACED INTO COLLEGE ALGEBRA BUT
ENROLLED IN INTERMEDIATE ALGEBRA ....................................... 151

TABLE 56:

CROSSTABULATION OF GRADES IN INTERMEDIATE ALGEBRA BY
PLACEMENT EXAMINATION SCORE BY TYPE OF
EXAMINATION ..................................................................................... 152

TABLE 57:

CROSSTABULATION OF GRADES IN COLLEGE ALGEBRA BY
PLACEMENT EXAMINATION SCORE ................................................ 153

TABLE 58:

CROSSTABULATION OF GRADES IN COLLEGE ALGEBRA BY
PLACEMENT EXAMINATION SCORE BY TYPE OF
EXAMINATION ..................................................................................... 155

TABLE 59:

STUDENTS WHO WERE ELIGIBLE TO ENROLL IN CALCULUS I, BUT
ENROLLED IN COLLEGE ALGEBRA ................................................. 156
xi

TABLE 60:

TABLE 61:

CROSSTABULATION OF GRADES IN FINITE MATHEMATICS BY
PLACEMENT EXAMINATION SCORES BY TYPE OF PLACEMENT
EXAMINATION ..................................................................................... 158

TABLE 62:

CROSSTABULATION OF GRADES IN COLLEGE ALGEBRA AND
TRIGONOMETRY BY PLACEMENT SCORES .................................... 159

TABLE 63:

CROSSTABULATION OF GRADES IN COLLEGE ALGEBRA AND
TRIGONOMETRY BY PLACEMENT EXAMINATION SCORES BY TYPE
OF EXAMINATION ............................................................................... 160

TABLE 64:

CHARACTERISTICS OF THE STUDENTS WHO WERE ELIGIBLE
FOR MTH132, BUT ENROLLED IN MTH116 AND WERE
UNSUCCESSFUL ................................................................................ 161

TABLE 65:

xii

CROSSTABULATION OF GRADES IN FINITE MATHEMATICS BY
PLACEMENT......................................................................................... 157

CROSSTABULATION OF GRADES IN SURVEY OF CALCULUS BY
PLACEMENT SCORES ......................................................................... 161

TABLE 66:

CROSSTABULATION OF GRADES IN MTH124 BY PLACEMENT
SCORES BY TYPE OF EXAM ............................................................... 163

TABLE 67:

CROSSTABULATION OF GRADES IN CALCULUS 1 BY PLACEMENT
EXAMINATION SCORES ...................................................................... 164

TABLE 68:

CROSSTABULATION OF GRADES IN CALCULUS 1 BY PLACEMENT
EXAM SCORE BY TYPE OF EXAMINATION ...................................... 165

xii

List of Figures
FIGURE 1:

FIGURE 2:

HISTOGRAM OF ACT MATHEMATICS SCORES ............................. 66

FIGURE 3:

NORMAL PROBABILITY PLOT OF ACT MATHEMATICS
SCORES ................................................................................................... 67

FIGURE 4:

HISTOGRAM OF HIGH SCHOOL GPA ................................................ 68

FIGURE 5:

NORMAL PROBABILITY PLOT OF HIGH SCHOOL GPA............... 69

FIGURE 6:

HISTOGRAM OF HIGH SCHOOL MATH GPA ................................... 70

FIGURE 7:

NORMAL PROBABILITY PLOT OF HIGH SCHOOL
MATHEMATICS GPA ............................................................................ 71

FIGURE 8:

xiii

STRUCTURE OF THE MSU MATHEMATICS PLACEMENT
EXAMINATION ...................................................................................... 29

HISTOGRAM OF MATHEMATICS PLACEMENT EXAMINATION
SCORES ................................................................................................... 72

FIGURE 9:

NORMAL PROBABILITY PLOT OF MATH PLACEMENT EXAM
SCORE...................................................................................................... 73

FIGURE 10:

SCATTERPLOT OF MATH PLACEMENT SCORES AND ACT MATH
SCORES ................................................................................................... 80

FIGURE 11:

SCATTERPLOT OF ACT MATH SCORES AND HIGH SCHOOL
GPA .......................................................................................................... 81

FIGURE 12:

SCATTERPLOT OF HIGH SCHOOL GPA AND MATH PLACEMENT
SCORES ................................................................................................... 82

FIGURE 13:

HISTOGRAM OF THE RESIDUALS ASSOCIATED WITH THE
LINEAR REGRESSION MODEL DESCRIBED IN TABLE 9.............. 85

FIGURE 14:

NORMAL PROBABILITY PLOT OF THE RESIDUALS ASSOCIATED
WITH THE REGRESSION MODEL IN TABLE 9 ................................ 86

FIGURE 15:

HISTOGRAM OF THE RESIDUALS OF THE REGRESSION MODEL
FOR THE COMBINED DATA ............................................................... 99

FIGURE 16:

NORMAL PROBABILITY PLOT OF RESIDUALS FOR THE
OVERALL DATA .................................................................................. 100
xiii

FIGURE 17:

SCATTERPLOT OF STANDARDIZED RESIDUALS VS.
STANDARDIZED PREDICTED VALUE ............................................ 101

FIGURE 18:

HISTOGRAM OF THE RESIDUALS OF THE REGRESSION MODEL
FOR THE PROCTORED REGRESSION MODEL .............................. 104

FIGURE 19:

NORMAL PROBABILITY PLOT OF THE RESIDUALS FOR THE
PROCTORED REGRESSION MODEL ................................................ 105

FIGURE 20:

SCATTERPLOT OF STANDARDIZED RESIDUALS OF THE
PROCTORED REGRESSION MODEL ................................................ 106

FIGURE 21:

HISTOGRAM OF RESIDUALS OF THE REGRESSION MODEL OF
THE UNPROCTORED REGRESSION MODEL ................................. 109

FIGURE 22:

NORMAL PROBABILITY PLOT OF RESIDUALS OF THE
UNPROCTORED REGRESSION MODEL .......................................... 110

FIGURE 23:

SCATTERPLOT OF THE STANDARDIZED RESIDUALS VS.
STANDARDIZED PREDICTED VALUES FOR THE UNPROCTORED
REGRESSION MODEL......................................................................... 111

xiv

xiv

CHAPTER 1
STATEMENT OF THE PROBLEM
Introduction
According to Jacobson (2006), a common requirement for obtaining a bachelor‟s
degree is College Algebra, Statistics, or some other type of standard mathematics course.
However, not all students have the mathematics background needed to be placed into a
standard mathematics course during their first semester of college. Colleges and
universities have taken on the responsibility of deciding on the type of mathematics
course that is most appropriate for the first time college student.
To determine the type of course in which students should begin, some colleges
and universities use information from students‟ academic background, others make
decisions by using nationally standardized tests (ACT, SAT, COMPASS, etc,), yet others
have developed their own mathematics placement examination. All of this is done in an
effort to give students the best chance of succeeding in their first college mathematics
course. In fact, Abraham (1992) conducted a study in which he found that 15 states have
used a combination of 75 different examinations to determine placement in reading,
writing, and mathematics. Also in a literature review, Sawyer and Schiel (2000) found
that approximately 90% of postsecondary institutions offer some form of placement
examination. Based on the scores on these examinations, students are labeled as either
prepared or underprepared (Flores, 2007).
Additionally, according to Morante (1989), thousands of underprepared students
enter American colleges not yet ready to handle college level coursework, and that
1

between one-third and one-half of all entering college students do not possess the basic
reading, writing and mathematics skills. In a study conducted by the American Council
on Education (1996) it was found that approximately 17% of community college students
and approximately 11% of students enrolled in four year public institutions were enrolled
in remedial courses.
Lacking these fundamental skills may cause students to reconsider their choice of
major. In fact, the Center for American Institutional Data Exchange and Analysis (as
cited in Chang, 2002) found that almost 50% of the students entering college with the
intention of majoring in science, mathematics, or engineering change from one of these 3
fields to another within the first 2 years. One of the reasons given for the change in
major was the lack of science and mathematics skills needed to persist (Chang, 2002).
If students began in the appropriate college mathematics course, the attrition rate
for majors in science, mathematics and engineering may be reduced. Requiring a
mathematics placement criterion prior to students enrolling into their first college
mathematics course could make it more likely that those students will begin in the most
appropriate mathematics course.
Morante (1987) describes placement testing as merely a means for placing
students into an appropriate first course. Therefore, institutions that use placement testing
use it precisely for the reason of placing students into the most appropriate first college
course. The most appropriate course is one that is neither too challenging nor too simple.
A course that is too challenging may have several repercussions, such as
“dropping back” to a lower level course. If the student is fortunate enough to realize
early that the course in which he or she is enrolled in not appropriate, then the student

2

may be able to “drop back” to a lower, more appropriate course during the same semester
and not have to spend a semester – possibly even two- away from mathematics.
On the other hand, the student may realize that the course is too challenging but
persists in hope that he or she will be able to just pass the course. Perhaps the student
passes but does not gain the skills necessary to do well in subsequent courses. The
student may persist and be unsuccessful and may then be required to repeat the course.
Either of the above scenarios could lead to the student developing a negative attitude or a
more negative attitude toward mathematics. Nolting (2004) noted that students‟ attitude
toward mathematics can play a major role in their success. Furthermore, students with
poor attitudes stop attending class, procrastinate in doing homework, or avoid taking
math altogether.
Failing or withdrawing from a course could increase the amount of time needed to
graduate. In a study conducted by ACT (2008), it was found that less than one-half of
students admitted to college graduate within five years. Furthermore, ACT (2002)
reported that despite the many efforts to increase retention, graduation rates have been
declining over the last 20 years. The National Center for Education Statistics (2010)
determined that about 57% of first time students seeking a bachelor‟s degree and
attending a 4-year institution full-time in 2001-2002, graduated within six years.
Research has also indicated how important success in mathematics can be to the
length of time it takes to graduate from college. Seidman (2005) reported that students
who failed College Algebra during their first term, and who were required to take the
course at some time during their academic career, were less likely to graduate from
college than those students who took and passed the course in their first semester. In

3

contrast, Parker (2005) found that students who were more successful in mathematics
were more likely to be retained and graduate in four years.
If a student does not graduate from college, his or her earning potential could be
significantly affected. In 2002 the United States Census Bureau estimated that an
average annual income of fulltime workers between the ages of 25 and 64, with a
bachelor‟s degree, was approximately $22,000 more than the average annual income for
fulltime workers in the same age group with a high school diploma but no bachelor‟s
degree, and the gap was expected to widen (Cotter, 2007).
Incorrect placement could negatively affect what material is taught in the
classroom or how that material is taught. If a classroom have many underprepared
students, instructors may spend valuable class time working with the underprepared
students. This may result in a neglect of the quality and rigor needed for subsequent
mathematics classes.
Placing students into a course that is too easy can also have a negative effect. If
students are placed into a course in which they feel the material has been presented in
previous coursework, then the students may not put forth maximum effort. Although the
student may pass the course, he or she may not have acquired the skills necessary for
success in subsequent mathematics courses. Because of these consequences, placement
into first college courses should be evaluated regularly.

Unproctored Placement Examinations
When examinations are proctored, the examinees are monitored, identities are
confirmed, and the rules are followed. When examinations are unproctored, examinees
4

are not monitored, and there is no way to be certain that rules are followed. When
examinations are proctored, there is a reasonable assumption that distractions are
minimized (or at least the testing environment is similar for each student). This may not
be true of unproctored examinations. So besides administration differences, the
proctored and unproctored examination could differ in environmental factors.
Institutions relying on mathematics placement examinations as tools for placing
students into their first college mathematics course have the added responsibility of
evaluating the effectiveness of these examinations. This becomes especially true when
there are two examinations that purport to be a measure of a student‟s readiness for
college mathematics, but are administered under different conditions. Schmitz and
DelMas (1991) suggest that, once a placement system is employed, institutions have the
responsibility of periodically evaluating the placement system.
If scores on an unproctored examination are going to be used in the same way as
scores on a proctored examination, every effort should be made to verify that similar
scores on either administration would result in similar placement as well as similar
success in the course in which students are placed. In addition, students with similar
skills and similar pre-college characteristics should be placed similarly regardless of the
type of examination used for placement.
One concern of an examination should be its validity. Messick (1989) defines
validity as “an integrated evaluative judgment of the degree to which empirical evidence
and theoretical rationales support the adequacy and the appropriateness of inferences and
actions based on test scores” (p. 13). When an examination is given in both a proctored
and unproctored setting, the inference is that both perform equally well and it is the

5

responsibility of the college or university to make sure that the two examinations are
comparable.

Purpose of the Study
The primary purpose of this study was to compare the results of the Michigan
State University Proctored Mathematics Placement Examination (hereafter known as the
proctored examination) to the results of the Michigan State University Unproctored
Examination (hereafter known as the unproctored examination). Comparing results
mean not only comparing the scores across the two type of examinations but also the
results of the decisions made as a result of the scores received on the two examinations.
If significant differences exist across the examinations, this study will attempt to explain
those differences by examining the pre-college characteristics of the students who are
placed with the proctored examination and examining the pre-college characteristics of
the students who are placed with the unproctored examination. Since placement
decisions are made as a result of the scores on the placement examinations then
examining the results of those decisions is an important aspect of evaluating the
effectiveness of the placement examination.

Research Questions
This study was guided by the following research questions:


Q1: Are there significant differences in the pre-college characteristics of the
students who were placed with the proctored examination and the pre-college
6

characteristics of the students who were placed with the unproctored
examination. If so, what are the differences? Furthermore, after controlling for
these differences, are there significant differences in placement examination
scores across the two groups?


Q2: Are the proctored and unproctored examination functioning similarly?



Q3: How well does the mathematics placement examination predict success in
students’ first college mathematics course? Furthermore, can the prediction of
success in students’ first college mathematics course be improved using the ACT
Mathematics score, high school GPA, the type of mathematics courses taken in
high school, and whether or not students took a mathematics course during their
senior year of high school?



Q4: How do the grades in each course compare across different levels of
placement examination scores?

Placing students into the correct mathematics course is essential for the success of
student. Furthermore “it affects the morale of the students and the workload of the
instructors; and it ultimately affects the reputation of the university” (Rodgers & Wilding,
1998, p.203).

7

CHAPTER 2
LITERATURE REVIEW

Structure of the chapter
This chapter is organized into several parts. First, there is a discussion of the
factors that should be considered when designing or choosing a placement examination.
This includes choice of the assessment, the content of the assessment, test validity, test
reliability, and the examination‟s ability to discriminate between examinees. Second,
there is a review of students‟ pre-college variables that are related to mathematics
placement. These variables include ACT Mathematics score, SAT Quantitative scores
high school GPA, high school mathematics GPA, race, gender, type of classes taken
during high school, and whether the student took mathematics during his or her senior
year of high school. Third, there is a review of the factors that are related to students‟
success in their first college mathematics course. These factors include ACT
Mathematics score, SAT Quantitative score, high school GPA, high school mathematics
GPA, mathematics placement examination scores, race, and gender. Fourth, there is a
discussion of the methods used to assess the validity of placement examinations. These
methods range from placement validity to validating cut scores. Fifth, there is a review
of some specific placement practices used by various colleges and universities to place
students into their first college mathematics course. Finally, there is a discussion of
proctored and unproctored examinations. This discussion includes situations in which an
unproctored examination is appropriate and methods for comparing results of

8

unproctored examinations to the results of proctored examinations to determine if the
results are comparable.

Selecting a Mathematics Placement Test
When deciding on a mathematics placement examination, institutions have the
option of using a nationally standardized assessment (i.e., COMPASS, ACCUPLACER,
ACT scores, SAT scores, etc.,) or a locally designed assessment (i.e., a placement
examination designed by the university). There are advantages and disadvantages of
each. For example, one of the more obvious advantages to using a locally designed
assessment is that its content may more closely resemble the course and the curriculum of
the department (McDonald, 1989). Burton and Ramist (2001) believed that locally
developed exams are customized to the institution‟s curriculum, student demographics,
and the institution‟s standards.
One disadvantage to using a locally designed assessment is that mathematics
faculty may not be trained in the design and validation of tests and, therefore, there is a
greater temptation to ignore periodic reviews of the assessment (McDonald, 1989). This
is important because alternate forms are an important part of testing and retesting and
most faculty either do not have or do not use the level of psychometrics required to
review these assessments in the appropriate manner (Morante, 1987). Ebel and Fisbie
(1986) found that locally developed assessment tend to have low reliability.
One of the advantages of using standardized tests is that they tend to have high
reliability; often 0.9 or higher (McDonald 1989).

9

.

When deciding on whether to use a standardized examination or a locally
designed examination for placement, Morante (1987) recommends that the following
nine factors be considered:


Test content. With respect to a mathematics placement test, arithmetic
and elementary algebra are essential components. Arithmetic exercises
should be both problem solving and word problems and make use of
fractions, decimals, and percents. While algebraic exercises should
also include problem solving and word problems, they should also
include linear equations involving numeral, fractional, and literal
components.



Criterion reference. Levels of difficulty and proficiency should be
established by faculty judgments of what students should know.



Discriminatory power. Discriminate accurately among students along a
continuum of proficiencies.



Speededness. Time limit should be such that 100 percent of the students
can complete at least 75% of the items and 90% of the students can
attempt all the items



Reliability. Test-retest and split half are methods most often used.
Reliability should be at least 0.90.



Validity. Content, concurrent, and predictive.

10



Guessing. Guessing can inflate scores. Provisions should be made to
reduce the effects of guessing.



Alternate forms. Every placement examination should have an alternate
form for retesting or post-testing



Cost. Includes cost of materials, cost to administer, and cost to score.

When deciding on a placement examination it is clear that 1) faculty need training
in test design, test validation, and psychometrics if they are going to design a local
assessment and 2) whether local or standardized, there are important guidelines to
consider.

Pre-college Variables Affecting Mathematics Placement
Pre-college variables are factors such as high school rank or high school grade
point average, type of mathematics courses taken in high school, grades earned in high
school mathematics courses, taking mathematics during senior year of high school, and
standardized achievement tests such as the ACT or SAT. When placing students into
their first college mathematics course, these pre-college variables can each have an
isolated effect on placement or can be put together to have a combined effect on
placement. Either way determining how these pre-college variables effect placement
may not be a simple task.
Hudson (1989), Matthew-Lopez (1989), Pugh and Lowther (2004), and Hill
(2006) found strong correlations between ACT Mathematics scores and scores on locally
developed mathematics placement examinations. Mathew-Lopez (1989) conducted a
study using 200 students who took the Ohio University mathematics placement
11

examination and found the correlation between ACT Mathematics score and scores on
the mathematics placement examination is r(198) = 0.767 (p <.01). In their study of 920
students at Auburn University, Pugh and Lowther (2004) found a correlation of 0.682
and, in his study of 2386 students at Michigan State University, Hill (2006) found a
correlation of 0.647. Even for “high risk” students, Pugh and Lowther (2004) found a
significant correlation between ACT Mathematics scores and mathematics placement
examination scores of 0.506. Pugh defined „high risk” students as students who did not
take mathematics during their senior year or go beyond algebra II.
Wattenbarger and McLeod (2008) conducted a study of 605 first time college
students in which they correlated scores on the mathematics portion of the ACT and SAT
with grades earned in first college mathematics courses and found that over half of the
correlations were negative and only two of the positive correlations were above 0.5.
When Hill (2006) examined his data further, he found that students who earned a
grade of B or better in AP Calculus improved their placement examination score (beyond
what was predicted by their ACT Mathematics score); some of the improvements were
statistically significant while others were not. Hill also found that students who received
an A- or better in FST (functions, statistics, and trigonometry) received significantly
higher placement examination scores than what was predicted by their ACT Mathematics
scores. Overall, students who took challenging high school mathematics course were
more likely to increase their placement score beyond what was predicted by ACT
Mathematics score alone.
Pugh and Lowther (2004) also found significant correlations between high school
GPA and scores on a mathematics placement examination (r(2384) =0.462, p < .001) and

12

students who had higher high school GPAs tend to receive higher placement examination
scores. They also found that students who had high GPAs and no mathematics course
during the senior year of high school resulted in lower placement scores than students
who had high GPAs and were enrolled in mathematics during their senior year of high
school.
Hill (2006) looked at students who placed into remedial mathematics and found
that taking a mathematics course during senior year of high school was not enough to
increase placement. He found that 18.4% of the students in the remedial mathematics
course had taken a mathematics course during their senior year of high school. However
these students either took a non-algebraically demanding course (below algebra II) or
took an algebraically demanding course (above algebra II) and received a low grade (less
than a C).
Matthew-Lopez (1998) developed a model to predict math placement examination
score in an effort to circumvent the use of a placement test. She found that both high
school percentile rank and ACT Mathematics score were correlated with math placement
scores and together, they significantly predicted scores on the placement examination.
Prior research suggests that although universities are using placement
examination scores, variables such as ACT Mathematics scores, high school GPA, type
of mathematics course taken during high school, and taking mathematics during senior
year of high school should also be considered when placing students.

13

Predictors of Success in First College Mathematics Course
Determining factors that predict success in mathematics can also be a very
difficult process. First there are the factors that precede students‟ enrollment into their
first college mathematics course. These factors include high school background, ACT
Mathematics scores, and SAT Quantitative scores. These factors have been found to
produce an isolated effect as well as a combined effect.
Pugh and Lowther (2004) examined the individual correlation of high school
GPA, high school math GPA, ACT Mathematics scores, and SAT Quantitative scores on
grades in first college mathematics courses. They found all of these pre-college factors to
be significantly correlated with grades in first college mathematics courses with the
highest correlation occurring between high school math GPA and grade in first college
mathematics course. They then proceeded to construct a regression model and found that
both high school mathematics GPA and math placement test accounted for 25% of the
variation in first college mathematics course grade. Furthermore, the mathematics
placement test significantly added to the prediction of course grade beyond the prediction
using only high school mathematics GPA. However, when the “at risk” students were
examined not only did the strength of the relationships with the dependent variables and
first college math course grade decrease, but placement examination scores no longer
added to the prediction of first college mathematics grade beyond high school
mathematics GPA.
Nelson and Leganza (2006) looked at the relationship between several pre-college
factors and grade in first college mathematics course for three different courses: liberal

14

arts mathematics, applied calculus, and theoretical calculus. They found that high school
GPA, gender, SAT Quantitative score, high school class rank, and SAT verbal score were
significant predictors of grades in the liberal arts mathematics course. However, for the
applied calculus course, only SAT Quantitative score, high school GPA, and gender were
significant predictors of course grade. For theoretical calculus, the logistic regression
model produced significant effect from high school GPA, SAT Quantitative score, and
type of institution (public or private), but there was no gender effect. They also found
that gender became less significant for higher level math courses. High school GPA was
more significant for applied calculus and less significant for liberal arts mathematics.
Using information from 266 first time freshmen enrolled in a College Algebra
course, Rodgers and Wilding (2006) developed a regression equation to predict final
course grade. They found that the best model to predict grades in College Algebra used a
combination of pre-college variables: algebra placement score, percentile rank, and SAT
Quantitative score.
Hudson (1989) conducted a study involving 1854 students and found neither the
ACT subscores nor the ACT composite score to be significantly correlated with final
grades in students‟ first college mathematics course. However, when Hudson performed a
multiple regression analysis, she found that ACT-M to be a significant predictor of final
grade in only one of the three math courses included in the study.
Bridgeman and Wendler (1989) studied a total of nine different algebra and precalculus courses at several universities and found that SAT Quantitative scores
significantly improved prediction of course grade. They also found that the gender of the
student improved the prediction of course grade above SAT Quantitative scores alone,

15

but gender had no increased effect in three of the courses where both SAT Quantitative
and high school GPA were used as predictors of course grade. At one college, adding the
experience score (the highest level mathematics course taken in high school) to SAT
Quantitative and high school GPA significantly improved the prediction of course grade.
At another college, when high school GPA and placement test were considered together,
there was a greater effect on course grade than each of high school GPA and SAT
mathematics considered alone.
Armstrong (2000) developed a model to predict course grade using mathematics
placement score, student‟s situational variables (employment hours, support for attending
school, financial aid, part-time ore full-time attendance, and family responsibilities),
dispositional variables (cognitive, behavioral and affective traits), and instructor
characteristics (instructor grading practices, full-time instructor, part-time instructor) and
found that instructor characteristics accounted for the greatest variation in course grade.
Armstrong also found that placement exam score was not a significant predictor of course
grade for courses with full-time faculty members and believed that the interaction of
instructor characteristics and student characteristics may explain the lack of the effect of
the placement exam on course grade.
Gupta, Harris, Carrier, and Caron (2006) did not use placement scores to predict
course grade. Instead, they used students‟ demographic information (sex, race, and
college major), factors that could impact study time (number of hours employed per
week, other coursework, number of children at home), academic background (number of
high school math courses, number of year since last math course was taken),student
learning behaviors (class absences and number of hours of tutoring), students‟ attitude

16

toward mathematics, and students‟ course experience (type of technology used, instructor
rank, number of times per week that course met). The final model indicated that older,
male students, who are taught by lower ranked instructors, have positive attitudes, have
taken a high number of high school mathematics courses, and have missed fewer classes
are more likely to receive better course grades. All of this supports that no single
measure should be used when placing students into their first college mathematics course.

Validating Placement Examinations
McDonald (1989) and Schmitz and Demas (1991) offered a set of guidelines for
validating placement examinations. McDonald suggested that department faculty
examine the items to make sure the items relate to the curriculum. Additionally, a sample
of students should be tested, descriptive statistics computed, and the distribution of the
scores from the sample should agree with what is expected. McDonald also suggested
that the scores from the placement examinations should be correlated with one or more
suitable criteria. McDonald says caution should be used when using end of course grades
as a criterion because “grades are a composite measure of achievement that tend to vary
greatly across instructors (pg 22).” The guidelines presented by Schmitz and DelMas
(1991) focused on improving the criterion – related validity of the placement
examination.
Schmitz and DelMas (1991) developed a set a guidelines for validating placement
examinations. First evidence that the placement examination contributes to the
prediction of course grades needs to be gathered before placement recommendations are
put into effect. Otherwise, decisions made after students have been placed are
17

confounded. Second, multiple regression, rather than correlation, is the preferred
approach to judging the predictive validity of placement examination scores.
Furthermore, it is suggested that the incremental validity of the examination be assessed.
Incremental validity refers to the unique contribution an examination makes to a
prediction equation and thus an examination is said to be valid if it produces a significant
increment in the predictive accuracy over other entry level data (Schmitz and DelMas,
1991). Third, cutscores should be considered. Cutscores should be set so that the
maximum number of correct decisions is made.
In 1969, Shevel and Whitney wanted to determine if the mathematics placement
examination offered sufficient improvement in the prediction of college mathematics
grades to warrant its addition to a college testing program. Additionally, they wanted to
compare the predictive validity of the mathematics placement examination to that of ACT
scores and high school mathematics grades for classes that differed in average math
ability and classes that covered different types of material. They concluded that the
addition of the mathematics placement examination improved college mathematics
grades predictions over ACT scores and high school mathematic grades. Also, the
improvement was greater for higher level courses.
Pugh and Lowther (2006) developed a regression equation and found that the
placement test significantly added to the predictive power of the regression equation;
beyond high school math GPA. However, when using students that were categorized as
“at risk” the placement examination did not significantly add to the predictive power of
the regression equation; beyond high school math GPA.

18

Other researchers have examined the cut scores as a way of supporting the
validity of a placement test. Slark et al. (1991) examined the final grades of students
scoring above and below the cut score across three different math courses: elementary
algebra, elementary algebra review, and Intermediate Algebra. They found that of the
students in elementary algebra, 48% of those scoring above the placement test cut score
were successful compared with 41% of those scoring below the placement test cut score.
Of the students in elementary algebra review, 62% of those scoring above the placement
test cut score was successful compared with 47% of those scoring below the placement
test cut score. Of the students in Intermediate Algebra, 65% of those scoring above the
placement cut score were successful in the course compared with 58% of those scoring
below the placement cut score. A successful student was defined as a student who
received a grade of C or higher in the course.
Gabe (1989) examined the academic achievement of students who scored below
or just above college level on a math placement examination and who did not enroll in a
college preparatory (remedial) mathematics course at Broward Community College. A
score of 12 or more resulted in placement into a college level mathematics course. Gabe
defined just above college level placement as receiving a score of 12 or 13 on the
placement test. Of the 1986 cohort who enrolled in college level mathematics
(MAT1033), 60% were not successful. Furthermore, approximately 70% of the student
who scored just above the cut score did not successfully complete MAT1033 within 7
terms.
Gabe also found that approximately 61% of the 1987 cohort who enrolled in
MAT1033 with scores of 13 or below was not successful. Furthermore, approximately

19

85% of the students who scored just above the cut score did not successfully complete
MAT1033 in 4 terms.
In his dissertation, Cotter (2007) examined the cut scores of a mathematics
placement examination by computing the odds of success for different cut scores in three
different courses at Georgia State University: Math Modeling for Non-Science Majors
(Math1101), College Algebra (Math1111), and Pre-Calculus (Math1113). When
defining success in first math course as receiving a course grade of D or better, Cotter
found that of the students who met or exceeded the cut score for placement Math1101,
Math1111, or Math1113 were approximately 2 times, 3.5 times, and 2 times more likely
to be successful than those students who scored below the cut scores for placement into
those courses. When defining success as receiving a grade of C or better, students who
scored above the cut score for placement into the same courses were approximately 2
times, 3 times, and 1.5 times more likely to be successful than students who scored below
the cut score. In both cases, those who scored above the cut score had a greater
probability of being successful in the course than those students who scored below the cut
score.
Sawyer (1989) developed placement validity indices based on logistic regression
methods for determining course placement. Logistic regression can be used to estimate
the conditional probability that a student would be successful in a course given the
student‟s score on a given predictor variable. Using these conditional probabilities of
success, the placement validity indices are computed. The validity indices are based on
the following four possible estimated outcomes of a give cut off score

20



True positive: A student who is predicted to pass the course actually passes the
course. (correct decision)



False positive: A student who is predicted to pass the course actually fails the
course. (incorrect decision)



True negative: A student who is predicted to fail the course actually fails the
course. (correct decision)



False negative: A student who is predicted to fail the course actually passes the
course. (incorrect decision)
Ang & Nobel (1993) used these four decisions to compute the following two

indices: accuracy rate (AR) and change in accuracy rate (AR). AR is the proportion of
students for whom the correct decision would be made using the cutoff score and the
success criterion. The optimum cut off score will be the score for which the conditional
probability of success is approximately 0.5. This cut off score is where the maximum AR
will be attained. AR is the difference between the maximum AR and the “base line”
AR – the proportion of correct decisions associated with using the lowest possible cutoff
score.
Matthews-Lopez (1998) used a cross validation technique to assess the validity of
a mathematics placement examination. First a simple random sample of 200 students
was drawn from a population of 3200 students who took the mathematics placement tests.
Second, a multiple regression equation was developed using the scores from the
placement examination as the dependent variable and ACT Mathematics exam and high
school percentile rank as the independent variables. Third, an independent sample of 200
students was drawn from the same population and the multiple regression equation was

21

used to predict scores on the placement test for this independent sample of students. The
results of the comparison between predicted and actual placement level (for either data
set) agreed in approximately 55% of the cases. Of the remaining 45%, recommended
placement was lowered in 18% of the cases and in about 8% of the cases, the placement
was raised.
The above research establishes that the validation of placement examinations
should be an ongoing process and should consist of a combination of approaches. These
approaches range from establishing the predictive validity of the examination to
validating the cut scores to establishing the consistency of examination results for
comparable groups to examining the decisions that are made as a result of the scores.

Unproctored and Proctored Examinations
There is extensive research on online unproctored examinations. Some of the
advantages are the reduction in the number of computers and manpower needed to
administer the test (McCloy, 2008; Sticha and Barber, 2003). Additionally, there is the
reduction in travel costs for the test taker as well as the expansion of the applicant base
(McCloy, 2008). One of the more serious disadvantages is the increase likelihood of
cheating.
Proctored testing is most relevant when the examination is high stakes (Rovai,
2001). However, if the examination is not high-stakes then an unproctored examination
may be appropriate because students may not be motivated to cheat. Kennedy, Nowak,
Raghuraman, Thomas, and Davis (2000) reported that cheating was more likely to occur
doing online tests than on tests that were administered face to face. However, another
22

study, Charlesworth, Charlesworth, and Vlica (2006) reported that cheating was no more
likely to occur in online examinations than on examinations administered in a face to face
setting.
Harmon, Lambrinos, and Kennedy (2008) developed a model to predict the scores
on an unproctored economics final examination. In order to detect cheating they used
the model for the proctored examination to predict the final examination score for the
unproctored examination. Harmon et al believed that if the class which took the
unproctored examination had many students whose predicted score was far from their
actual score, this would be an indication that cheating has occurred.
Davies, Norris, Turner, and Wadlington (2005) conducted an analysis of an
unproctored administration of The Hogan Personality Inventory. They believed that if
cheating was occurring then it would manifest itself as near perfect assessment scores.
However, they did not find enough evidence to support cheating across administrations.
If examinations are given online as an unproctored examination it should be assumed that
the test items and scores may be compromised (Sticha and Barber, 2003). To minimize
the negative consequences of the potential compromised information, procedures must be
put in place.
Examples of such procedure would be to use items that have been retired from
previous forms of an examination, administer the items in an adaptive format, have an
extensive item pool, have ongoing item development and calibration, or verify the results
with additional assessments under supervised conditions (McCloy, 2008).
Sticha and Barber (2003) describe a verification process associated with the
administration of an unproctored online version of the Armed Services Vocational

23

Aptitude Battery (ASVAB). When an examinee takes the unproctored internet version
of the ASVAB, they must have their scores verified by going to a Military Enlistment
Processing Station (MEPS) and taking a short verification test; maybe less than 15
minutes. If the examinee does not pass this verification test, he or she would be forced to
retake the complete test. If a verification process is used, this fact must be made clear to
the examinees. Otherwise too many people will fail the verification process (Sticha and
Barber 2003).
Rueda and Sokolowski (2004) compared the results of an unproctored placement
examination to the results of previous monitored examination. They found no difference
in the percentages of students who placed into the various mathematics courses.
Schumacher and Smith (2008) conducted a study in which they compared the
scores of a proctored paper and pencil mathematics placement test (given in 2004) to the
scores of an unproctored online test (given in 2005). They found that scores on the
unproctored placement examination was significantly higher than scores on the proctored
placement examination. However, they noted that students who took the unproctored
placement examination had – on average – higher SAT Quantitative scores than students
who took the proctored placement examination (576.81 unproctored vs. 568.58
proctored). Thus there is an implication that students who took the unproctored
examination had higher mathematics ability than those who took the proctored
examination the previous year and therefore the higher placement scores should be
expected.
Finally, Schumacher and Smith (2008) develop a logistic regression model using
placement into remedial math as the dependent variable and SAT verbal, SAT

24

Quantitative, placement test scores, and high school GPA as independent variables. The
2005 values were substituted into the model that used the 2004 data. This resulted in a
prediction accuracy of approximately 94%.
Davies and Wadlington (2006) compared scores on The Hogan Personality
Inventory (HPI) across proctored and unproctored settings. They compared the means,
standard deviations, and coefficient alpha reliabilities of each personality scale across the
two groups (proctored and unproctored). Although they found that the unproctored
administration of the assessment resulted in significantly higher means on four of the
scales, the effect sizes were small. Davies and Wadlington also conducted a DIF analysis
and found that at least one item with significant DIF was found on each scale and four of
the seven scales had more items with significant DIF across the two groups than would
be expected by chance.
The above studies indicate that determining if an unproctored examination and a
proctored examination are comparable, there mean scores can be compared and a DIF
analysis can be conducted to see if items and functioning differently across examination
types. To protect against cheating, a verification process can be implemented. This
verification process can increase the validity of the unproctored examination. One vital
piece of information that is missing is the examination of the course outcomes once
students are placed by each type of examination.
Mathematics Placement Procedures
Colleges and Universities vary in the procedures used to place students into their
first college mathematics course. These colleges and universities vary in the type of
placement test used and how placement decisions are made. For example, Odell and
25

Schumacher (1995) developed a formula for placing students into their first college
mathematics course: 2*placement test score + Math SAT score. At the time this equation
was developed, this particular university was using a test designed by the Mathematics
Association of America.
At St. Olaf College, students choose one of three different placement exams. This
choice depends on the student‟s academic plans. Using placement test scores and a large
number of regression equations, students were placed into courses in which they would
have a high probability of being successful (Flores, 2007).
A placement process described by Krawczyk and Toubassi placed students into
their first mathematics course according to their score on one of two placement
examinations and other factors such as high school GPA (as cited in Rueda and
Sokolowski 2004).
To understand the similarities and differences of the current placement systems at
various United State Colleges and Universities, a non scientific study was conducted in
which information was gathered about the placement procedures at a sample of 50
colleges and universities. These institutions responded as follows:


17 schools indicated that they currently administered a placement examination
developed by the Mathematics Department.



5 schools indicated that they use ACT Mathematics scores or SAT Quantitative
scores to determine initial placement. If the students would like to improve their
placement, they must take a placement examination designed by the Mathematics
Department.

26



12 schools indicated that they offer the placement exam only in an unproctored
setting.



7 of the schools indicated that the results of the placement test were used as
recommended placement and students were allowed to enroll in a course at a
level higher or lower than what was indicated by their placement exam score.



10 of the schools indicated that they use a combination of placement exam scores
and high school information to place students into their initial mathematics
course

Procedures for placement vary across universities. Some universities place students
using nationally standardized examinations, others use their own examination, and others
use a combination of variables. Some universities use proctored examinations, others
use unproctored examinations, and others use a combination of the two. Some allow
multiple attempts and others allow only a single attempt. Whatever the case may be,
these procedures need to be evaluated regularly and modified if necessary.

MSU Mathematics Placement Examination
Approximately 7000 first time college students enter Michigan State University
(MSU) each fall with hopes of obtaining an undergraduate degree some years later. To
graduate from MSU, each student must fulfill certain university requirements. These
requirements consist of a series of courses which students must complete regardless of
the major they have chosen. Among these courses are choices of several mathematics
courses. In order to determine the most appropriate beginning math course, most
students must take the MSU mathematics placement examination. The results of the

27

examination are used to determine the most appropriate course mathematics course in
which students must begin.

Description of the Michigan State University Mathematics Placement Examination
Figure 1 shows the structure of the MSU Mathematics Placement Examination.
The placement examination was designed by the MSU Mathematics Department. The
examination consists of a total of 28 multiple-choice items. There are about 14 different
versions of each item thus allowing for multiple forms. An examinee begins with a group
of 14 items (Group A). These initial 14 items are pre-calculus items. The examinee
received 1 point for a correct answer and 0 points for an incorrect answer. If the
examinee responds to 8 or fewer of these items correctly, the examinee is given another
group of 14 algebra items (Group B). A correct response to a group B item is worth ½
point while an incorrect response is worth 0 points. The score from group B is added to
the score from group A and this sum is rounded up to the next integer. This score is the
examinee‟s total score on the placement examination and is used to determine the highest
level course in which the student will be allowed to enroll. Examinees who go from
group A to group B, can receive a maximum score of 15.
If an examinee responds to 9 or more of the initial 14 items correctly, the
examinee is given another group of 14 items (Group C) that contains algebra,
trigonometry, and precalculus items. A correct response is worth 1 point and an
incorrect response is worth 0 points. The score from group A is added to the score from
group C and this sum is the examinee‟s total placement examination score. Thus an

28

examinee that goes from group A to group C will have a total placement examination
score between 9 and 28.

Group A
14 pre-calculus items
1pt each

8 or less

9 or more

Group B
14 algebra items
½ pt each

Group C
14 algebra, trigonometry, and precalculus items
1pts each

Figure 1. Structure of the MSU Mathematics Placement Examination

29

Placement
Based on the total score, students are placed into one of five levels of courses.
These levels are described below. A description of the courses is given in appendix A.
Each course is a one semester course. The levels are a modified version of the tiers used
by Hill (2006).

Level 1: Intermediate Algebra (MTH1825). This is the university’s remedial
mathematics course. The credits received in this course do not count toward a
student’s degree but it is included in the calculation of a student’s GPA.

Level 2: College Algebra (MTH103) or Finite Mathematics and Elements of
College Algebra. Students who enroll in MTH103 must take at least one
additional math course above MTH103 to satisfy the university mathematics
graduation requirement. MTH103 is worth 3 credits and MTH110 is worth 5
credits. The credits received in any of these courses count toward the minimum
number of credits needed to obtain a degree. Placement at this level requires a
minimum placement examination score of 10.

Level 3: College Algebra and Trigonometry (MTH116). This 5 credit course is
intended to prepare students for technical calculus. The credits received in this
course counts toward the minimum number of credits needed to obtain a degree.
Placement at this level requires a minimum placement examination score of 12.

30

Level 4: Finite Mathematics: Applications of College Algebra (MTH112), Survey
of Calculus 1(MTH124), Elementary Mathematics for Teachers (MTH201) or
Statistical Methods (STT200 or STT201). MTH112, MTH124, MTH201, and
STT200 are each 3 credits and STT201 is 4 credits. The credits earned in any of
these courses do count toward the minimum number of credits needed to obtain a
degree. Placement at this level requires a minimum placement examination score
of 15.

Level 5: Calculus 1 (MTH132). This 3 credit course requires a minimum
placement examination score of 19.

The MSU mathematics placement examination is offered as a proctored
examination or as an unproctored examination. Students are required to complete the
examination before enrolling in classes at MSU or before their MSU Academic
Orientation Program (AOP) session; whichever occurs first. Students who decide to take
the proctored examination can do so at AOP or by making arrangements at one of several
MSU extension offices. Students who decide to take the unproctored examination can do
so from any location that has full internet access. Students can attempt the mathematics
placement examination up to three times, but can only take the proctored examination
once. Students must wait at least four weeks between each attempt. If students attempt
the placement examination more than once, the highest score obtained will be used to
determine placement. Any student receiving a minimum score of 19 – on the proctored
examination - will be given credit for the university‟s mathematics requirement.

31

Who Must Take the Mathematics Placement Examination?
While most first time college freshmen are required to take the mathematics
placement examination, there are a few exceptions. According to the MSU Mathematics
Placement Service any students falling into any of the following categories are not
required to take the mathematics placement examination:


Students with credit in at least one of the following: MTH103, MTH116,
1

2

MTH124, MTH132, MTH133 , MTH234 or any math course above MTH234.


Students who have already been granted Advanced Placement (AP) Mathematics
credits for calculus before their Academic Orientation Program (AOP).



Students who have ACT Mathematics scores of at least 28 or SAT Quantitative
scores of at least 640 and who will be taking math at MSU.

Summary
This chapter points out several ideas. First, predicting mathematics placement as well
as success in first college mathematics course is a very complex task. Using high school
background information can be difficult because high school transcripts are difficult to
interpret. In addition, predicting student‟s grades in their first college mathematics
course using pre-course information can be difficult because of student‟s situational
factors, dispositional factors, and even variability in instructors‟ grading pattern that take
effect once students enroll in the course.
Second, because of the important decisions that are made as a result of the placement
examination, it is important that the validity of the examination be assessed regularly.
1
2
2

Calculus II: Integral Calculus
Calculus III: Multivariate Calculus
32

This includes assessing the predictive validity of the examination, examining cut scores,
and investigating false positives.
Lastly, “how a test is administered can also significantly affect student‟s scores and
therefore test validity” (Akst and Hirsh, 1991, p4). Therefore, it is important that the
scores across the different type of examinations be compared. If the scores are found to
be different, then finding reasons for the difference is the only next logical step.
What is missing from the research on proctored and unproctored examination is the
investigation of the course outcomes for each course in which students are placed into as
a result of the each type of examination. This study will add to the body of research by
investigating these course outcomes.

33

CHAPTER 3
METHOD
This chapter describes the study‟s design and research methodology. First there is
a brief description of the design of the study, followed by a description of how the sample
was determined. Second, there is an overview of the data collection process. This
includes where the data was obtained, how the data was coded and a description of the
variables created for this study. Finally, the methods of data analysis, for each research
question, are introduced.
This study takes placed at Michigan State University (MSU); a public research
university located in the northeastern part of the United States with a total student
population of approximately 47,000. Each year, the university accepts approximately
7000 new freshmen. Many of these students are required to take a mathematics
placement examination prior to enrolling in any mathematics course. Students have the
option of taking a proctored examination or an unproctored examination. Hence the
opportunity to evaluate the effectiveness of each type of examination presented itself.

Design
The study‟s general design is an ex post facto, quasi-experimental design with
two non- randomized groups. The participants were divided into two groups according to
the type of examination that was used as placement. The first group was the group of
students who were placed with the proctored examination. This group consisted of 598
first time college freshmen who enrolled in mathematics during fall semester 2008. The

34

second group was the group of examinees who were placed with the unproctored
examination. This group consisted of 4382 first time college freshmen who enrolled in
mathematics during fall semester 2008.

Sample
Because of the small number of students who were placed with the proctored
examination, the entire group of 598 students was used in this study. To determine the
minimum sample size needed for the group of students who were placed with the
unproctored examination, the following formula was used

( 1)

Where
n is the minimum sample size
z-score is the standard normal score that cuts off the top (1 – desired confidence level)
area of the normal curve
p is the estimated proportion of the attribute that exists in the population (for this study,
the attribute was the proportion of students who were successful in their first college
mathematics course)
e is the level of precision

35

For this study, a confidence level of 95% was used with a precision level of 3%.
Since the variability was unknown, p =.5 allowed for maximum variability. Thus, the
minimum sample size needed was computed as

The students in the unproctored group were placed in alphabetical order (by last
name) and a systematic random sample was taken using every 4th student. This
procedure produced a sample of 1098 students from the unproctored group. Table 1
gives the demographics for the sample of students used in this study.

Table 1
Distribution of Race and Gender for Proctored and Unproctored Data
Group 1
Group 2
(Proctored) (Unproctored)
Male
Female
White
Black
Asian
Hispanic
American
Indian
Not
indicated

47.32%
52.68%
66.56%
17.39%
2.85%
1.84%
0.50%

44.08%
55.92%
75.77%
8.11%
5.10%
3.01%
0.73%

10.87%

7.29%

36

Data Collection
Data was collected from students‟ college and high school transcripts obtained
from the Michigan State University Registrar‟s Office. The registrar‟s office was asked
to alphabetize the list of students in group 2 and take a systematic sample of every 4th
student. The registrar‟s office then provided a file that contained information for all
students in group 1 and the selected students from group 2. The file contained each
student‟s mathematics placement examination score, ACT Mathematics score, type of
examination used to place the student (proctored or unproctored), first college
mathematics course, grade in first college mathematics course, gender and race.
The registrar‟s office then provided the high school transcripts of the students
who were selected for this study. Taken from these transcripts were students‟ high
school mathematics courses from ninth grade through twelfth grade and the grades
received in each course. These grades were used to compute each student‟s mathematics
GPA. This information was added to the file received from the registrar‟s office.

Letter Grade Conversion
The grading system varied across high schools. For example, some high schools
graded students using only the traditional letter grades: A, B, C, D, and F. In addition to
these traditional letter grades, other high schools used grades of A+, B+, C+, and D+.
Additionally, some high schools gave grades of A-, B-, C-, and D-. Instead of letter
grades, some schools awarded number grades. Some school awarded grades on a four
point scale (4.0, 3.0, 2.0, 1.0, 0.0) while others included half grades (3.5, 2.5, 1.5) as well.
Some school awarded grades that were not included in the calculation of students‟ GPA
37

(P = Pass, CR = Credit, NC = No Credit). These grades were not included in the
calculation of students‟ mathematics GPA.
High schools also varied in the quality points awarded to each letter grade. For
example, at one high school a B+ grade was awarded 3.5 points while at another high
school a grade of B+ was awarded 3.33 points. Yet another high school awarded 3.30
points for a grade of B+.
Quality points also varied within high schools. Within schools, a grade of B
received for an honors course was awarded more quality points than a B grade in a nonhonors course. However, for this study, grades earned in an honors course were given the
same quality points as similar grades earned in a non-honors course.
To develop a letter grade to number grade conversion table, each letter grade was
converted to a range of quality points and the midpoint of the range was used as the point
value for that letter grade. The range of quality points was determined from an analysis of
the high school transcripts. The analysis of the transcripts revealed that the lowest
quality points awarded to a grade of C- was 1.67 while the highest quality points awarded
was 1.75. For the next highest grade, C, every high school awarded 2 points. So the
quality point range for a C- was 1.67 to 2.00 (including 1.67 but not including 2.00). The
midpoint of this range is 1.84 and this value was use as the point value for a grade of C-.
The point value for the other letter grades was computed similarly. Table 2 lists the
quality point range and the value used as the point value for each letter grade

38

Table 2
Letter Grade Conversion Chart Used to Compute Students’ High School Mathematics
GPA
QUALITY
POINT
GRADE RANGE
A+
4.30 +
A
4.00 - 4.30
A3.67 - 4.00
B+
3.30 - 3.67
B
3.00 - 3.30
B2.67 - 3.00
C+
2.30 - 2.67

QUALITY
POINT
VALUE GRADE RANGE
4.30
C
2.00 - 2.30
4.15
C1.67 - 2.00
3.84
D+
1.30 - 1.67
3.49
D
1.00- 1.30
3.15
D0.67 - 1.00
2.84
F
0 - 0.67
2.49

VALUE
2.15
1.84
1.49
1.15
0.84
0.34

Definition of Pre-college Characteristic Variables
For this study, there were several pre-college characteristic variables. One
variable was students‟ high school grade point average (HSGPA). This continuous
variable was included with the data file that was received from the MSU registrar‟s
office. Because some schools award grades above 4.0, several students had high school
GPAs greater than 4.0
Another pre-college characteristic variable was students‟ high school mathematics
grade point average (HSMGPA). This continuous variable was constructed by
converting the letter grade received in each mathematics course to its corresponding point
value using table 2. These values were then averaged to produce a HSMGPA for each
student. Some students were enrolled in one mathematics class for an entire school year
while others were enrolled in one mathematics class during the first semester and a

39

different mathematics class during the second semester. Students who were enrolled in
one class for an entire school year usually had two grades recorded on their transcripts:
one grade for semester 1 and another grade for semester 2. When this happened, both
grades were used in the calculation of HSMGPA.
The course in which a student was enrolled at each grade level was used as a
th

variable. The variable MATH9S1 and MATH9S2 represented students‟ 9 grade
mathematics course for semesters 1 and 2 respectively. If a student was enrolled in the
same course for the entire ninth grade, then that course was recorded for both MATH9S1
and MATH9S2. Similarly MATH10S1, MATH 10S2, MATH11S1, MATH11S2,
MATH12S1, and MATH12S2 represented the courses in which students were enrolled
during each semester of their tenth, eleventh, and twelfth grade years respectively.
GR9LS1 and GR9LS2 are the letter grades that students received in the ninth
grade of semester 1 and semester 2 respectively. Similarly GR10LS, GR10LS2,
GR11LS1, GR11LS2, GR12LS1, and GR12LS2 are the letter grades for each semester of
students‟ tenth, eleventh, and twelfth grades respectively.
The last mathematics course taken in high school was also used as a pre-college
characteristic variable. For this study, calculus, pre-calculus, and trigonometry were
considered algebraically demanding courses and therefore, the most challenging courses
in which a high school student could enroll. A course in algebra or Geometry was not
considered algebraically demanding and not challenging enough for students who
intended to pursue some type of postsecondary education. Therefore these courses were
considered to be the lowest level of courses in which a high school student could enroll.
Then, there were courses that were considered to be more challenging than algebra or
40

geometry, but not as challenging as calculus, pre-calculus, or trigonometry. Courses in
probability or statistics were considered to be one level below calculus, pre-calculus, or
trigonometry. Courses in finite math, discrete math, math analysis, and FST (functions
statistics, and trigonometry) were considered to be a level above algebra and geometry.
The variable LAST was created and was given the value 1 if a student‟s last high school
mathematics course was calculus (including AP calculus), pre-calculus, or trigonometry,
the value 2 if a student‟s last high school math course was probability or statistics
(including AP statistics), the value 3 if a student‟s last high school mathematics course
was FST, discrete math, finite math, or math analysis, and the value 4 if a student‟s last
math course was algebra or geometry.
For this study a passing grade in a high school course was considered to a grade
of C or better. From this, the variable CABOVE was created. This variable had a value
of 1 if a student received a grade of C or better in his or her last high school mathematics
course and the value 0 if the student received a grade lower than C in his or her last
mathematics course.
Whether or not a student took a mathematics course during his or her senior year
was also included as a pre-college variable. This categorical variable was named SYM
and was given the value 0 if the student was not enrolled in a mathematics course during
his or her senior year of high school or the value 1 if the student was enrolled in a
mathematics course during his or her senior year of high school.
ACT Mathematics score was used as a pre-college characteristic variable. This
continuous variable was named ACTM. This variable was included in the data file that
was received from the MSU registrar‟s office.

41

Definition of College Variables
Several college level variables were included in this study. One variable was the
mathematics placement examination score. This continuous variable was named MPE
and was included in the data file. This variable had a range of 0 to 28.
There were two type of mathematics placement examinations administered to
students: proctored or unproctored. This dichotomous variable was named TYPE and
was given the value 0 if the student was placed into their first mathematics course with
the unproctored examination and the value 1 if the student was placed into their first
mathematics course with the proctored examination.
First college mathematics course was another college variable. This categorical
variable was called FMC. For this study, students‟ first mathematics course was one of
the following: Intermediate Algebra (MTH1825), College Algebra (MTH103), Finite
Mathematics (MTH110), College Algebra and Trigonometry (MTH116), Finite
Mathematics: Application of College Algebra (MTH112), Survey of Calculus 1
(MTH124), Calculus 1 (MTH132), Math for Elementary Teachers 1 (MTH201), or
Statistical Methods (STT200/STT201).
The variable GRADE was used to represent the grades given in the MSU
mathematics course. MSU uses the following grading scale 4.0, 3.5, 3.0, 2.5, 2.0, 1.5,
1.0, 0.0, W (withdrawal), or I (incomplete). No student in this study received a grade of
W or I.

42

Demographic Variables
There were two demographic variables used in this study. The first was Gender.
This dichotomous variable was coded 0 if the student was a male and 1 if the student was
female. The other demographic variable was RACE. This variable represented the
race/ethnicity of the student and was coded 0 if the student was white, 1 if the student
was black, 2 if the student was Hispanic, 3 if the student was American Indian, 4 if the
student was Asian or a Pacific Islander, and 9 if a student‟s race was not indicated.

Analysis
The statistical analysis software package PASW Statistics 18 (formerly SPSS)
was used to conduct the data analysis for this study. The research questions required the
use of several statistical techniques. Among them were multiple linear regression (MLR),
binary logistic regression (BLR), 2 independent sample t – tests, and Pearson chi-square
test.

Multiple Linear Regression (MLR)
MLR expresses the linear relationship between a dependent variable (or outcome
variable) and two or more independent variables (or predictor variables). The dependent
variable must be continuous and the independent variables can be continuous or
categorical. The prediction equation is given by:

(2)

43

Where
is the estimated value of the dependent variable

b0 is the estimated value of the regression intercept
th

bi is the estimated value of the regression slope for the i predictor
th

xi is the value of the i predictor

Assumptions
MLR has several assumptions. When these assumptions are satisfied, the estimates of
the regression coefficients are unbiased, efficient, and consistent (Ostrom, 1990). An
estimator is unbiased if its expected value is equal to the true value of the parameter. An
estimator is efficient if its variance is smaller than the variance of any other estimator. An
estimator is consistent if it converges in probability to the true value of the parameter.
MLR requires that the following assumptions be met:


Linearity: MLR requires that a linear relationship exists between the outcome
variable and each of the predictors. Visual inspection of a scatterplot can verify
this assumption (Triola, 2010).



Multicollinearity: Multicollinearity occurs when two or more predictor variables
are highly correlated. This assumption can be checked by inspecting the
correlation matrix of the predictors.

44



Normality: The variables used in the model should be normally distributed. This
assumption can be checked by visually inspecting the histogram associated with
each variable used in the model.



Normally distributed errors with mean = 0: The histograms of the error terms can
be visually inspected for normality. Additionally, a normal probability plot can
be used to assess the normality of the errors. If the points form a straight line, the
normality assumption of errors is supported.



Homoscedastiscity: Homoscedastiscity means that the variance of the errors is the
same across all levels of each predictor and can be checked by visually inspecting
the scatterplot of the standardized residuals and the standardized predicted values
(Osborn and Waters, 2002).

Model Fit
In addition to checking the model assumptions, the model fit should also be checked.
The fit of the MLR model can be checked in three ways.


Test of overall model fit: PASW Statistics 18 produces an ANOVA table that is
associated with the MLR model. This table has an F statistics that can be used to
test the overall fit of the model. A significant finding indicates that at least one of
the model coefficients is significantly different from zero.



Test of individual model coefficients: The significance of each model coefficient
can be tested using a t-test. A significant finding indicates the regression
coefficient is significantly different from zero. The test of the individual model
coefficients is given in PASW Statistics 18.
45



2

Coefficient of Determination (R ): This is a measure of the proportion of
variation in the dependent variable accounted for by its linear relationship with
2

the predictor variables. R can also be used as a test of global model fit. R

2

ranges in value from 0 (no variation in the dependent variable is accounted for
and therefore poor model fit) to 1 (all the variation is accounted for and therefore
2

perfect model fit). R can be requested in the multiple regression procedure in
PASW statistics 18.

Binary Logistic Regression (BLR)
BLR is similar to MLR but is used when the dependent variable is dichotomous. The
independent variables could be continuous or categorical. BLR models the log odds of
belonging to a particular group. Usually the dependent variable takes on the value 1 when
an observation belongs to the group and 0 when the observation does not belong to the
group. The BLR model is given by:

(3)

Where
P is the probability that the dependent variable takes on the value 1

b0 is the regression intercept
th

bi is the estimated value of the regression slope for the i predictor

46

xi is the ith predictor

BLR can be used to calculate the conditional probability of achieving a score of 1.
Manipulating equation (3), the conditional probability of achieving a score of 1 is given
by

(4)

The following components are usually included when presenting BLR results; 1)
overall goodness – of –fit statistics; 2) estimates of the model coefficients with test
statistics, odds ratios, p –values, confidence intervals; and 3) classification tables (Cotter,
2007; Garson, 2010).
Model Fit


The Hosmer Lemeshow Test: The Hosmer and Lemeshow test is usually preferred
as the test of overall fit of the BLR model (Hosmer and Lemeshow, 2000). A nonsignificant finding indicates that the BLR model fits the data well. The Hosmer
and Lemeshow test can be requested as part of the logistic regression procedure in
PASW Statistics 18.



Omnibus Test of Model Coefficient: This test is similar to the F test of overall
model fit in MLR and tests the null hypothesis that that the model with additional
predictors does not fit the data better than the intercept only model.

47



2

Wald  statistic: This statistic is similar to the t-test used to test the significance
of the individual coefficients in MLR. A significant finding indicates that the
2

coefficient is different from zero. The Wald  statistic is a standard output of the
logistic regression procedure in PASW Statistics 18.


Classification Table: The classification table can be used to determine the
percentage of correct decisions made by the model. When the model predicts that
a student will be successful and the student is actually successful, then the model
has produced a correct decision. Also, when the model predicts that a student will
be unsuccessful and the student is actually unsuccessful, the model has also
produced a correct decision. The percentage of correct decisions made will be
called the “hit rate”. The classification table can also be used to compute the
percentage of false positives and false negatives produced by the BLR model

When variables are added to a BLR model, one is concerned about whether the
additional variables improve the prediction of the model. The deviance statistic can be
used to determine if additional variables improve the prediction of the model. In PASW
Statistics 18, the deviance statistic is given by the -2log likelihood statistic. To determine
if the additional variables improve the fit of the overall model, the deviance of the model
that was constructed with the additional variables is subtracted from the deviance of the
model without the additional variables. This difference has a chi-square distribution with
the degrees of freedom equal to the number of variables added to the model. A
significant finding indicates that the additional variables improve the prediction.

48

Two Independent Sample T-Test
A 2 independent sample t-test is used to determine if there is a statistically
significant difference in the means of two independently sampled groups. A significant
finding indicates that the two group means are different.

Assumptions


Groups are sampled from two different populations: This assumption is part of the
research design.



Normality of independent variables: This can be checked by visually inspecting
the histogram of the independent variables



Homogeneity of variances: Levene‟s test can be used to test this assumption.
Levene‟s test is part of the PASW Statistics 18 output. A non-significant finding
indicates that the group variances are equal.

Pearson Chi-Square Test of Independence
When two variables are not associated they are considered to be statistically
independent (Frankfort-Nachmias & Leon-Guerrero, 2009). This study looks for
dependence in course success across various groups by using Pearson‟s chi-square test of
independence. This is done by computing the cell frequencies that are expected under
the null hypothesis that there is no association between the two variables. If these
expected cell frequencies are statistically different from the observed cell frequencies,

49

then the variables are said to be statistically dependent (Frankfort-Nachmias & LeonGuerrero, 2009).

Assumptions
The chi-square test of independence has two important assumptions:


The sample size is large



Each expected cell count is at least 5

If either of these assumptions is not satisfied, then Fisher‟s exact test can be used as
an alternative to Pearson‟s chi-square test. Fisher‟s exact test computes the conditional
probability of getting the observed contingency table. Pearson chi-square test and
Fisher‟s exact test can be requested as output in PASW statistics 18.

Effect Size Measures
Neill (2008) noted that statistical significance is not a direct indicator of effect
size and that statistical significance is a function of sample size. Also, in the case of large
samples, significant testing can give misleading results because even small or trivial
effects can be statistically significant. Thus a measure of effect size can be used in these
2

situations. The effect size measures used in this study is R , odds ratio, and Cohen‟s
standardized difference (Cohen‟s d).
An effect size measure that is often used with MLR is the proportion of variance
in the dependent variable that is due to its relationship with the independent variables.
50

2

2

This is known as R . The value of R ranges from 0 to 1 with proportions close to one
2

indicating perfect model fit. The value of the adjusted R is also reported when there are
a large number of independent variables. If there are a small number of independent
variables, R2 and adjusted R2 will be close but if there are a large number of independent
variables, the adjusted R2 may be lower than R2. This effect size is reported with all
MLR models used in this study. Frankfort-Nachmias & Leon-Guerrero, 2009, Gravetter
2

& Wallnau, 1985 and Triola, 2010 should be consulted for more information on R .
For binary logistic regression, the odds ratio is used as a measure of effect size. If
the independent is dichotomous, the odds ratio can be interpreted as the odds of success
of one group relative to the odds of success for the other group. If the independent
variable is continuous, then the odds ratio can be interpreted as the factor by which the
independent variable changes (increases or decreases) the odds of the dependent
variables. The odds ratio was used as an effect size measure for all BLR models used in
this study.
The odds ratio can also be used as an effect size when performing a 2 x 2
contingency table analysis. In this situation, the odds ratio can be used to determine
whether two dichotomous variables are associated. Below is an example of a 2 x 2
contingency table. To calculate the odds ratio, one can simply divide the product of A
and D by the product of B and C (e.g. (A*D)/(B*C) ) . Of course, the computation of the
odds ratio will depend on the odds of interest. For this study, the odds ratio was
computed for each analysis involving a 2 x 2 contingency table.

51

Table 3
Example of a 2 x 2 Contingency Table
Variable
1
Variable
2

A
C
A+C

Total
B
D
B+D

A+B
C+D

Cohen‟s standardized difference (Cohen‟s d) is the effect size measure most used
with 2 independent sample t-tests. Cohen‟s d is given by:

(5)

Where
= mean score of the independent variable on the proctored examination
= mean score of the independent variable on the unproctored examination

= variance of the independent variable on the proctored examination
= variance of the independent variable on the unproctored examination
= sample size of the independent variable on the proctored examination
= sample size of the independent variable on the unproctored examination

52

This value gives the mean difference of two variables expressed in standard deviation
units (Neill, 2008). For Cohen‟s d, an effect size below 0.2 is a small; an effect size
between 0.2 and 0.5 is a moderate, and an effect size greater than 0.5 is a large (Cohen,
1989). For this study, Cohen‟s d was reported for each test involving the comparison of
two means.

Question 1
Are there significant differences in the pre-college characteristics of students
who were placed with the proctored examination and the pre-college characteristics of
those students who were placed with the unproctored examination? If so, what are the
differences? Furthermore, after controlling for these differences, do significant
differences exist in the placement examination scores across the two groups?
One important aspect of comparing group performance is to determine whether
the groups are equivalent. This could be of particular importance when the researcher
was not involved in the group assignment process. If the two groups are not similar then
difference in performance on the placement examination or in the grade received in their
first college mathematics course, may not be attributed to differences the type of test used
for placement. Therefore the following null hypotheses were tested:
H01: There is no significant difference in the mean ACT Mathematics scores of
the students who were placed with the proctored examination and the mean ACT
Mathematics scores of the students who were placed with the unproctored
examination.

53

H02: There is no significant difference in the mean high school GPA of the
students who were placed with the proctored examination and the mean high
school GPA scores of the students who were placed with the unproctored
examination.

H03: There is no significant difference in the mean high school mathematics GPA
of the students who were placed with the proctored examination and the mean
high school mathematics GPA of the students who were placed with the
unproctored examination.

If there are significant differences in the mathematics ability of each group, then it
should come as no surprise to see differences in placement examination scores across the
groups. If similar students are considered, then there should be no difference in
placement examination scores. To address this question a multiple linear regression
model was constructed. This model used mathematics placement examination score
(MPE) as the criterion and ACT Mathematics score, high school GPA, high school
mathematics GPA, and type of exam used for placement as predictors (TYPE). If the
variable TYPE was significant then students with similar combinations of ACT
Mathematics scores, high school GPA, and high school mathematics GPA, received
higher placement examination scores on one type of examination. The MLR model was
run using PASW Statistics 18. The model assumptions were checked and the value of R2
was reported.
54

Question 2
Are the proctored and unproctored examinations functioning similarly?
This question is an important one because it can determine if a particular exam
type is more (or less) advantageous for a particular group of students. If the exams are
used in the same way, then similar students should see similar results.
First of all, the reliability of the placement examination was computed using the
Kuder-Richardson 21 (KR21) reliability index as an estimate. KR21 was computed using
the following formula:

(6)

where

rxx is the estimate of the test reliability
n is the number of items on the examination
is the mean examination score
is the variance of the examination scores
The data for this study consisted of total examination scores; not the item responses.
Thus, KR21 reliability was the most appropriate reliability index to compute because the
formula uses the total number of test items as well as the mean and variance of the
examination scores. These values could be easily computed from the data. KR21
assumes that all items are of equal difficulty. When this is not the case KR21 will
underestimate the exam‟s true reliability (Allen and Yen, 1979). Thus, KR21 is a

55

conservative estimate of the reliability of the exam and the exam‟s true reliability would
be higher.
Second, a separate correlation matrix was constructed for the entire data set, for the
proctored data, and for the unproctored data. The following variables were included in
each of the correlation matrices:


ACT Mathematics score



High school GPA



High school mathematics GPA



Math placement examination score



Grade in college mathematics course



Gender (1 if female, 0 if male)



Senior year math (1 if the student took math during their senior year of high
school, 0 other wise)



CPT (1 if student took calculus, pre-calculus, or trigonometry; 0 otherwise)



RACE/ETHNICITY (1 if the student was white, 0 otherwise)

If the proctored and unproctored examinations are functioning similarly, the
correlation matrix for the unproctored data should be similar to the correlation matrix for
the proctored data. For example, variables that are highly correlated in the unproctored
matrix should be highly correlated in the proctored matrix. So the correlations of the
variables in the unproctored correlation matrix were compared to the correlations of the
variables in the proctored correlation matrix.
To test the equality of the correlation matrix each correlation was normalized
using Fisher‟s Z transformation (Thorndike and Dinnel, 2001). This formula is given by
56

(7)

where r is the value of the correlation coefficient.
After each correlation is transformed, a statistical test is conducted to determine if
the corresponding entries in the proctored and unproctored correlation matrices are
statistically equivalent. The test statistic can be found in Thorndike and Dinnel (2001)
and is given by:

(8)

where

ZrP is the Fisher‟s Z transformation of the proctored correlation
ZrU is the Fisher‟s Z transformation of the unproctored correlation
NP is the sample size used to compute the proctored correlation
NU is the sample size used to computed the unproctored correlation

Multiple Hypothesis Testing
When a hypothesis test is conducted, the decision is either to reject the null
hypothesis or retain the null hypothesis. If the null hypothesis is actually true and the
decision is to reject it, then a Type I error has been committed. This Type I error (usually
referred to as alpha () is usually assigned at the beginning of the study. One of the
57

problems with conducting multiple hypothesis tests is that the Type I error becomes
inflated and it becomes more likely that the null hypothesis will be rejected when it is
true (Abdi, 2010).
There are several methods of correcting for the inflation in the Type 1 error rate
and the reader is encouraged to consult Shaffer (1995). One way of controlling this
inflation in Type 1 error is by testing each of the n significance tests at a significant level
of

. The procedure is known as the Bonferroni Correction Method. Any hypothesis

test with a p –value ≤

is rejected. The Bonferroni Correction Method was chosen

because it is easy to calculate.
Another method used to determine if the two examinations are functioning
similarly was to create a model to predict placement examination scores using the
proctored data and fitting that model to the unproctored data. This would give an
estimate of the placement exam scores in which a student who was placed with the
unproctored examination would have received had he or she been placed with the
proctored examination. Predicted placement should agree with actual placement.
Similarly, a model to predict placement examination scores using the unproctored data
was created. This model was fitted to the proctored data to determine the unproctored
placement of the students who were placed with the proctored examination.
Without concern for their significance, the MLR model used to predict placement
examination scores used MPE as the dependent variable and ACTM, HSGPA, HSMGPA,
SYM, GENDER, RACE, LAST, C_ABOVE and TYPE as the independent variable and
PASS as the dependent variable. Dummy variables were created for the variables LAST

58

and RACE. LAST was a categorical variable with 4 levels and RACE was a categorical
variable with 5 levels.
For LAST, students whose last mathematics course was calculus (including AP),
pre calculus, or trigonometry were used as the reference group. The variables PSTT,
ALG, and OTHER were created as dummy variables for LAST. PSTT took on the value
1 if a student‟s last class was probability or statistics (including AP statistics) and 0
otherwise. ALG took on the value 1 if a student‟s last high school math class was algebra
1, algebra 2, algebra 3, algebra 4, or geometry and the value 0 otherwise. OTHER took
on the value 1 if a student‟s last high school mathematic course FST (functions, statistics,
and trigonometry), discrete math, math analysis, or finite math. The reference group was
represented in each of PSTT, ALG, and OTHER have a value of 0.
White was used as the reference group for RACE. Thus, the dummy variables for
RACE were BLACK (1 if the student was black and 0 otherwise), HISPANIC (1 if the
student was Hispanic and 0 otherwise), INDIAN (1 if the student was American Indian
and 0 otherwise), and ASIAN (1 if the students was ASIAN and 0 otherwise).
Three groups were created. The first group, AGREE, consisted of the subjects
whose predicted placement agreed with their actual placement. For example, if a
student‟s predicted placement with the proctored regression model was MTH103 and the
student‟s actual placement using the unproctored examination was MTH103, then this
student would belong to the AGREE group.
The second group, LOWER, consisted of students whose predicted placement
with the proctored regression model was lower than their actual placement with the
unproctored examination. For example, if a student‟s predicted placement using the

59

proctored model was MTH103 but the student was actually placed into MTH132 with the
unproctored examination, then this student would belong to the LOWER group.
The HIGHER group consisted of students whose predicted proctored placement
was higher than their actual placement. For example, if a student‟s predicted placement
using the proctored regression model was MTH132 but the student was actually placed in
MTH132 with the unproctored examination, then this student would belong to the
HIGHER group. If the unproctored examination and proctored examination are
functioning similarly, there should be a high level of agreement between the placement
predicted by the proctored model and the actual placement by the unproctored
examination.
Also, to help support the idea that both examinations are functioning similarly, the
success rates in each course across both exams were considered.

To determine if the

success rates differ across examinations, a series of contingency table analyses were
performed. The Pearson chi–square test was used to determine if the success rates were
different. If there were significant differences in success rates across the two types of
examinations, then the success rates among different groups of examinees across
examination types were examined.

Question 3
How well does the mathematics placement examination predict success in students’
first college mathematics course? Furthermore, can the prediction of success in
students’ first college mathematics course be improved using ACT Mathematics score,

60

high school GPA, the type of mathematics courses taken in high school, and whether
or not students took a mathematics course during their senior year of high school?
At the time of this study, MSU used only the placement examination to place
students into their first college mathematics course. Thus the obvious concern is whether
the placement examination significantly predicts success in first college mathematics
course. A binary logistic regression model was developed to predict the log odds of
success in Intermediate Algebra (MTH1825), College Algebra (MTH103), College
Algebra and Trigonometry (MTH116), Survey of Calculus 1 (MTH124) and Calculus 1
(MTH132) was constructed using placement examination scores as the predictor and the
variable PASS (1 if the student received a 2.0 or better and 0 otherwise) as the outcome.
The next task was to determine if the prediction of success in first college
mathematics course can be improved by using student background information in
addition to placement examination scores. To do this, ACT Mathematics scores
(ACTM), high school GPA (HSGPA), whether a student took a math course during
senior year of high school (SYM), type of examination used for placement (TYPE) and
the dummy variables PSTT, ALG, and Other were added to the model containing MPE of
each of the courses.
To determine if the new model added to the prediction of success in first college
mathematics course, the change in the deviance was tested. The deviance is also known
as the -2 log likelihood in PASW Statistics 18. The change in the deviance has a chisquared distribution with degree of freedom equal to the number of predictors added to
the model. A significant finding indicates that the model with the predictors added
improved the prediction of success beyond MPE. If the model containing the additional

61

variables provides a better prediction of success, then the additional variables should
improve the “hit rate” and decrease the false positives.
Question 4
How do the grades in each course compare across different levels of placement
examination scores?
When a placement examination places a student into a mathematics course, the
exam is predicting that the student will be successful in the course in which he or she is
placed. Furthermore, it is assumed that the student will be successful in any course lower
than the one in which he or she was placed. When students are predicted to be successful
and are actually unsuccessful, we say that the exam has given a false positive.
A crosstabulation of course grades by placement examination score was done to
examine the distribution of grades at each placement exam score for MTH1825,
MTH103, MTH110, MTH116, MTH124, and MTH132. As the placement examination
scores become higher, more students should receive high course grades than low course
grades. This crosstabulation also shows the number of false positives. Crosstabulations
of course grade by placement examination score by type of examination was also
examined to determine which examination, if any, produces the greater number of false
positives.
Some of the students in this study enrolled in a course lower than that which was
recommended by their placement examination score. Some of these students received
placement scores just above the minimum score while others received placement scores
well above the minimum score. For this study, a student is said to have received a score
just above the minimum score needed for placement into a course if the student was
62

eligible for a course one level higher than the course in which he or she enrolled.
Additionally, a student is said to have received a score well above the minimum score
needed for placement if the student was eligible to enroll in a course two or more levels
above the course in which he or she enrolled.
It is important to the validity of the placement examination that 1) explanations be
sought as to why a student would enroll in a course lower than the level at which he or
she was placed and 2) students who scored well above the minimum cut score needed for
placement into a course would be unsuccessful in the course. If these reasons are
associated with the examination, then it is the researchers recommendation that the
examination be revised (e.g. revision or deletion of items, change in administration
procedures).
This section demonstrates that validating placement examination requires a
variety of evidence from different sources. The process becomes even more tedious when
there are two types of examinations that are administered differently. Not only is there a
concern about the validity of the placement examination as a whole, but the
comparability of the results from the two types of examinations.

63

CHAPTER 4
RESULTS

Introduction
The primary purpose of this study was to determine if the proctored and
unproctored examinations are functioning similarly. This means that no group should be
advantaged (or disadvantages) by the type of examination used for placement. This
means that placement is independent of examination used for placement. This means that
the type of examination used for placement should not have an effect on course
outcomes. This section is divided according to the research questions.

Question 1
Are there significant differences in the pre-college characteristics of students who
were placed with the proctored examination and the pre-college characteristics of those
students who were placed with the unproctored examination? If so, what are the
differences? Furthermore, after controlling for these differences, are there significant
differences in placement examination scores across the two groups?

Table 4 shows that the mean ACT Mathematics score, mean high school GPA,
mean high school mathematics GPA, and mean placement examination score were all
higher for the students in the unproctored group than for students in the proctored group.
It came as no surprise that the mean placement examination score for the unproctored
examination was higher than the mean placement examination score for the proctored
64

examination. A larger proportion of the students who were placed with the proctored
examination may have been given the secondary group of items that are scores ½ point
each (Group B). In contrast, a larger proportion of the students who were placed with the
unproctored examination may have been given the secondary group of items that are
scored 1 point each (group C). This difference alone could account for the large
difference in the mean placement examination scores. Nonetheless, if math achievement
and skill is determined by ACT Mathematics score and high school math GPA, it appears
that the students who were placed with the proctored examination were less talented,
mathematically, than the students who were placed with the proctored examination. The
differences in the mean of the proctored examination and the mean of the unproctored
examination (P – U) were each tested using the independent sample t-test in PASW
Statistics 18.

Table 4
Descriptive Statistics for ACT Math Scores, High School GPA, High school Mathematics
GPA, and Mathematics Placement Examination for Proctored and Unproctored Group
Proctored (P)
Category
N
M SD
ACTM
521 22.87 4.16
HSGPA
544 3.52 0.35
HSMGPA 576 3.32 0.58
MPE
598 12.06 5.05

Unproctored (U)
N
M SD
1006 23.58 3.83
1032 3.57 0.31
1090 3.39 0.49
1098 14.18 5.65

65

P-U
M
-0.72
-0.05
-0.07
-2.12

Figures 2 thru 9 show the histograms and the normal probability plots for ACTM,
HSGPA, HSMGPA, and MPE. The assumption of normality does not appear to hold for
HSMGPA (Figures 6 and 7). The histogram is negatively skewed and the points on the
normal probability plot stray from the straight line at the low and high ends. The normal
probability plot (Figure 8) for MPE show the existence of few outliers, how not enough
to effect the normality of the distribution. Otherwise the assumption of normality does
not appear to be violated for HSGPA and ACTM.

Figure 2 Histogram of ACT Mathematics Scores

66

Figure 3. Normal Probability Plot of ACT Mathematics Scores
67

Figure 4. Histogram of High School GPA

68

Figure 5. Normal Probability Plot of High School GPA
69

Figure 6. Histogram of High School Math GPA

70

Figure 7. Normal Probability Plot of High School Mathematics GPA
71

Figure 8. Histogram of Mathematics Placement Examination Scores

72

Figure 9. Normal Probability Plot of Math Placement Exam Score

73

Homoscedastiscity Assumption: Levene‟s test was used to check the homoscedastiscity
(equal variance) assumption. PASW Statistics 18 conducts Levene‟s test as part of its 2
independent sample t-test. Table 5 gives the results of Levene‟s test for each dependent
variable, the results of the t-test, and the value of Cohen‟s mean difference effect size.
For Levene‟s test, the F statistic is significant for ACTM, HSGPA, and MPE. Thus equal
variances cannot be assumed and the corresponding test statistics are significant.
The students who were placed with the proctored examination appear to have
lower mean ACT Mathematics scores, lower mean HSGPA, and lower mean placement
examination scores. Cohen‟s d indicates that the difference in the mean ACT
Mathematics scores was small (d = 0.117) and the difference in the mean high school
GPA was small (d = 0.151), however the difference in mean placement examination
scores is somewhat more substantial (d = 0.584).

74

Table 5
Test of the Equality of Variances and Equality of Means
Levene's Test
for Equality of T-test for Equality of
Variance Means
F
p
t
df
Equal variances
assumed
Equal variances
not assumed
Equal variances
HSGPA
assumed
Equal variances
not assumed
Equal variances
HSMGPA
assumed
Equal variances
not assumed
Equal variances
MPE
assumed
Equal variances
not assumed
ACTM

10.41

p

d
.177

957.477

.001

.002 -2.681

1504

.007

947.412

.010

.271 -1.014

1504

.311

-0.994
11.261

.000

-2.595
1.214

1504

-3.399
9.454

.001 -3.498

980.123

.320

.001 -6.879

1504

.000

-7.109

1133.37

.000

.151

.055

.584

To determine if there were group differences in the proportion of students who
took math during their senior year or group differences in the proportion of students who
were enrolled in algebraically demanding courses (calculus, pre-calculus, or
trigonometry), a Pearson‟s chi-square test was used. Table 6 shows the contingency table
.of senior year math by examination type. Approximately 78.5% of the students who
were placed with the proctored examination were enrolled in a math course during their
senior year of high school compared to approximately 82% of the students who were
placed with the unproctored examination. The chi-square test statistics was not significant

75

2

 (1, N =1147) = 29.75, p =.085. Therefore, the proportion of students who took a math
class during senior year of high school and were placed with the proctored examination
was not different from the proportion of students who took mathematics during their
senior year and was placed with the proctored examination.

Table 6
Table 6
2 x 2 Contingency Table for Senior Year Math by Type of Examination
Type of Exam
Senior Year
Math
No
Yes
Total

Proctored
21.55%
78.45%
100.00%

Unproctored
17.93%
82.07%
100.00%

It is also possible that the two groups could have differed on the type of
mathematics classes in which they were enrolled in high school. Therefore, a chi-square
test was conducted to determine if there was a difference in the type of high school math
courses in which students were enrolled. Table 7 show a 4 x 2 contingency table with the
type of high school math course by type of examination. From the table, approximately
57% of the students who were placed with the proctored examination had calculus, precalculus, or trigonometry as their last high school math course compared to
2

approximately 61%. The chi-square statistic was not significant  (3, 1669) = 3.838, p
=.281. Therefore there was no difference in the distribution of last high school math
courses taken by the students in each group.

76

Table 7
Table 7
4 x 2 Contingency Table of Last High School Mathematics Course by Type of
Examination
Type of Exam
Last High School
Math Course
Proctored
CPT

a

Unproctored

57.34%

c

ALG

7.98%

15.20%

PSTT

61.01%

9.84%

b

12.94%

d

OTHER
17.62%
18.07%
Total
100.00% 100.00%
a
b
c
CPT = calculus, pre-calculus, or trigonometry, PSTT = probability or statistics, ALG
d

= algebra I, algebra II, or geometry, OTHER = FST (function, statistics, and
trigonometry), business math, consumer math, or math analysis
The students who were placed with the proctored examination were different from
the students who were placed with the unproctored examination due to their lower
average ACT Mathematics score; lower highs school GPA, and lower placement
examination scores. However, both groups appear to have taken similar type of courses
and enrolled in mathematics during their senior year of high school.
.

It was already determined that the students who were placed with the unproctored

examination received higher placement examination scores (M = 14.18, SD =5.65) than
the students who were placed with the unproctored examination (M =12.06, SD =5.05).
This should come as no surprise because the mean ACT Mathematics score of the
students who were placed with the unproctored examination (M =23.58, SD = 3.83) was
higher than the mean ACT Mathematics score of the student who were placed with the
77

proctored examination (M = 22.87, SD = 4.16) and the mean high school GPA of the
students who were placed with the unproctored examination (M = 3.57, S = 0.31) was
higher than the mean high school GPA of the students who were placed with the
proctored examination (M = 3.52, S = 0.35). However, if ACT Mathematics scores and
high school GPA were controlled, is there still a significant difference between the mean
placement examination score of the students who were placed with the proctored
examination and the mean placement examination score of the students who were placed
with the unproctored examination?
To determine if controlling for ACT Mathematics score and high school GPA
would still produce an examination effect, a multiple linear regression (MLR) equation
was developed using PASW Statistics 18. MPE was used as the dependent variable and
ACTM, HSGPA, and TYPE were used as predictors.

78

Multiple Linear Regression (MLR)
Normality Assumption. The normality of ACTM, HSGPA, and MPE was
illustrated in Figures 2 thru 9 above.
Linearity Assumption: Figures 10, 11, and 12 shows a scatterplot of ACTM,
HSGPA, and MPE. The scatterplot shows that both ACTM and HSGPA are linearly
related to MPE. Also, ACT Mathematics scores appear to have a stronger linear
relationship with placement examination score.

79

Figure 10. Scatterplot of Math Placement Scores and ACT Math Scores
80

Figure 11. Scatterplot of ACT Math Scores and High School GPA
81

Figure 12. Scatterplot of High School GPA and Math Placement Scores

82

Multicollinearity Assumption: Table 8 gives the correlation matrix for the independent
variables used in the MLR model. The predictors are not highly correlated with one
another and, therefore, multicollinearity is not a problem.

Table 8
Table 8
Correlation Matrix for ACTM, HSGPA, and TYPE
HSGPA TYPE
ACTM 0.366* -0.086*
HSGPA 1.000
-0.079*
TYPE
1.000
* p < 0.01

Multiple Linear Regression Model
A multiple linear regression model was constructed to determine if, after
controlling for ACTM and HSGPA, the students who were placed with the unproctored
examination received higher placement examination scores than the students who were
placed with the proctored examination. The regression equation was given by:
MPE = -13.858 + -0.807*ACTM + 2.415*HSGPA – 1.230*TYPE

The test of overall fit was significant F(3,1509) = 459.619, p = .000, indicating that at
least one of the regression coefficients is significantly different from zero.
Table 9 lists the regression coefficients, standard error, t value, and p-value. The
table shows that each of the regression coefficients are a significant predictor of
placement examination score.

83

The multiple linear regression equation also indicates that keeping ACTM and
HSGPA constant, students who were placed with the proctored examination scored, on
average, just over 1 point less on the placement examination score than the students who
are placed with the unproctored examination.

Table 9
Multiple Linear Regression Results

Predictor
Intercept
ACTM
HSGPA
TYPE

b
-13.858
0.807
2.415
-1.230

Std.
Error
t
1.126 -12.304
0.027 30.157
0.330
7.307
0.207 -5.927

p
0.000
0.000
0.000
0.000

Residual Analysis of the Multiple Linear Regression Model
Figure 13 below shows the histogram of the residuals generated by PASW Statistics
18. The figure clearly shows that the distribution of the error terms is normal with a mean of
0. Additionally, the normal probability plot in figure 14 shows that the points form a straight
line and the figure supports the assumption of the normality of the error terms.

84

Figure 13. Histogram of the Residuals Associated with the Linear Regression Model
Described in Table 9

85

Figure 14. Normal Probability Plot of the Residuals Associated with the Regression Model in Table 9

86

Question 2
Are the proctored and unproctored examinations functioning similarly?
Reliability
Determining whether the two types of examinations are functioning similarly
required several analyses. Table 10 shows the estimated reliabilities of the combined
placement examination, the proctored examination, and the unproctored examination
using equation (6). As indicated, the reliability estimate of the unproctored examination
is higher than the reliability estimate of the proctored examination.

All the reliability

estimates are greater than 0.70 so the reliabilities are high enough for the examinations to
be used. The formula for KR21 uses mean score and variance. Since the proctored
examination produced higher mean examination scores and higher variance, it was no
surprise that the reliability estimate for the unproctored examination was higher than the
reliability estimate for the proctored examination. Additionally, students were able to
take the proctored examination more than once. This could have accounted for the higher
reliability estimate of the unproctored examination.

Table 10
Table 10
Reliability Estimates of the Overall Examination, the Proctored Examination, and the
Unproctored Examination
Overall
0.80

Proctored
0.76

Unproctored
0.81

87

The next step was to compare the correlation matrix of the variable using the
proctored data to the correlation matrix of the variables using the unproctored
examination. Second was to determine if placement was dependent on the type of
examination used for placement. Third, a multiple linear regression equation was created
using the proctored data. This equation was then fitted to the unproctored data to
determine if the students who were placed with the unproctored examination would have
received the same placement if they had been placed with the proctored examination.
Also, a multiple linear regression equation was created using the unproctored data. This
equation was fitted to the proctored data to determine if the students who were placed
with the proctored examination would have received the same placement if they would
have been placed with the unproctored examination.
Finally, the success rates of the different courses were examined to determine if
success rates were dependent on type of examination used for placement.

Correlation for Combined Data (Proctored data and Unproctored Data)
Table 11 is the correlation matrix for all the variables that will be used in the
overall (proctored data and unproctored data combined) MLR model. As indicated, all
the variables are significantly correlated with the scores on the mathematics placement
examination. The table shows that the variable that is most highly correlated with
placement examination scores is ACTM. Thus, student with high ACT Mathematics
scores tend to receive higher placement examination scores. ACTM is also the variable
that is correlated most highly with grades in first college mathematics course. HSGPA,

88

HSMGPA and MPE are moderately correlated with the scores on the mathematics
placement examination as well as grades in first college mathematics course.
The correlations between the dichotomous variables (C_ABOVE, GENDER,
SYM, WHITE, and CALC) and MPE or GRADE are point biserial correlations. A point
biserial correlation is used as a measure of association between a dichotomous variable
and a continuous variable (Allen & Yen, 1979). Positive values are an indication that
moving from a value of 0 to a value of 1, for example, tends to result in higher scores on
the continuous variables. Similarly negative values of the point biserial correlation
indicates that moving from a value of 0 to 1 tend to result in a decrease on the continuous
variable. Thus, the correlation between CPT and MPE indicates that students, who took
calculus, pre-calculus, or trigonometry as their final high school math course; tend to
have higher placement examination scores than the students who took a course other than
calculus, pre-calculus, or trigonometry as their final high school math course. Also,
females tend to receive lower placement examination scores than males, non-whites tend
to receive lower placement examination scores than whites, students who received a
grade of C or better in their last high school mathematics course tend to received higher
placement scores than students who received a grade lower than a C in their final high
school math course. The reader should ignore the perfect correlation between the variable
G and W. Both variables are dichotomous and therefore the Pearson‟s correlation
coefficient is not an appropriate measure of association for the two variables. Also, the
correlations between these two variables have no substantive interpretation.

89

Table 11
Table 11
Correlation Matrix for the Combined Data (Proctored and Unproctored)

A

GR

H

HM

C

S

W

G

-.275

.035

.064*

-.012

-.005

.019

1.000** -.099**

-.154**

A

1.000 .383** .366**

.376**

.156**

.059*

-.275**

.294**

.666**

.353**

.392**

.137**

.064*

.035

.150**

.364**

1.000

.670**

.197**

.017

.064*

.186**

.379**

1.000

.411**

.028

-.012

.116**

.379**

1.000

-.038

-.005

-.048

.122**

1.000

.019

.279**

.121**

1.000

-.099**

-.154**

1.000

.341**

GR

1.000

H
HM
C
S
W
CPT

CPT

MPE

MPE

1.000

Note: G = Gender, A = ACTM, GR = GRADE, H = HSGPA, HM = HSMGPA, C =
CABOVE, S = SYM, W = WHITE, CPT = Calculus, pre-calculus, or trigonometry
*p < .05. **p < .01

90

Correlations for Proctored Data
Table 12 gives the correlation of variables using the proctored data. Of particular
interest are the variables that are correlated with placement examination score and grade
in first college mathematics course. As indicated in the matrix, all variables are
significantly correlated with scores on the mathematics placement examination with ACT
mathematic score being the variable that is most highly correlated with mathematics
placement examination score.
The matrix also indicates that all the variables are significantly correlated with
grades in first college mathematics course except taking a math course during senior year
of high school (S) and race/ethnicity (W). Therefore, those students who took a
mathematics course during their senior year of high school was did not tend to received
better grades in their first college mathematics course. Additionally, white students who
were placed with the proctored examination did not tend to do better in their mathematics
course than non-whites who were placed with the proctored examination.

91

Table 12
Correlation Matrix for the Proctored Data
A
G
A
GR

GR

-.289**

.006

H

HM
.016

C

-.042

S
-.060

W

CPT

.034 1.000**

MPE

-.047 -.214**

1.000 .493** .429** .450** .188**

.027

-.289** .222**

.722**

1.000 .421** .413** .139**

.071

.006 .174**

.448**

1.000 .697** .221**

.000

.016 .197**

.394**

1.000 .476**

-.009

-.042

.104*

.393**

1.000

-.042

-.060

-.055

.152**

.034 .298**

.084*

H
HM
C
S

1.000

W

1.000

CPT

-.047 -.214**
1.000

MPE

.334**
1.000

Note: G = Gender, A = ACTM, GR = GRADE, H = HSGPA, HM = HSMGPA,
C = CABOVE, S = SYM, W = WHITE, CPT = Calculus, Pre-calculus, or Trigonometry
*p < .05; **p < .01

92

Correlations for Unproctored Data
Table 13 shows the correlation matrix for the unproctored data. Just as with the
proctored data, the interest in these correlations is in the variables that are correlated with
placement examination scores and grades in first college mathematics course. As shown
in the table, all the variables are significantly correlated with the unproctored placement
examination with ACT Mathematics score being the variable that is most highly
correlated with scores in placement examination score.
The correlation matrix also shows that all the variables are significantly correlated
with grades in first college mathematics course except S and W (similar to the correlation
matrix for the proctored data). However, unlike the proctored correlation matrix, high
school mathematics GPA is the variable that is most highly correlated with grades in first
college mathematics course.

93

Table 12
Table 13
Correlation Matrix for Unproctored Data
A
G
A
GR

GR

-.289**

.006

H

HM
.016

C

-.042

S
-.060

W

CPT

.034 1.000**

MPE

-.047 -.214**

1.000 .493** .429** .450** .188**

.027

-.289** .222**

.722**

1.000 .421** .413** .139**

.071

.006 .174**

.448**

1.000 .697** .221**

.000

.016 .197**

.394**

1.000 .476**

-.009

-.042

.104*

.393**

1.000

-.042

-.060

-.055

.152**

.034 .298**

.084*

H
HM
C
S

1.000

W

1.000

CPT

-.047 -.214**
1.000

MPE

.334**
1.000

Note: G = Gender, A = ACTM, GR = GRADE, H = HSGPA, HM = HSMGPA,
C = CABOVE, S = SYM, W = WHITE, CPT = Calculus, Pre-calculus, or Trigonometry
*p < .05; **p < .01

94

Table 14 shows the matrix of p-values associated with the test of equality of correlations.
There were a total of 44 tests. Thus, each test was conducted using the Bonferroni
Correction Method with a significance level of

. Table 13 shows that

correlation between ACT Mathematics score and grades in first college mathematics
course could not be assumed equal across examination types. Also, the correlation
between grades in first college mathematics course and placement examination scores
was marginally significant.

Table 14
P-values for the Test of Equality of Correlations

G
A
GR
H
HM
C
S
W
CPT

A
.718

GR
.410
.000

H
.156
.019
.010

HM
.373
.010
.372
.137

C
.088
.278
.800
.354
.006

S
.639
.396
.735
.653
.282
.942

W
.718
.410
.156
.373
.088
.639

CPT
.131
.029
.478
.751
.707
.840
.541
.131

MPE
.119
.003
.001
.509
.571
.214
.373
.119
.671

Note: G = Gender, A = ACTM, GR = GRADE, H = HSGPA, HM = HSMGPA,
C = CABOVE, S = SYM, W = WHITE, CPT = Calculus, Pre-calculus, or Trigonometry

95

Placement into Intermediate Algebra (MTH1825)
Table 15 shows the crosstabulation for placement into remedial mathematics by
type of examination. The table shows that approximately 25% (147/598) of the students
who were placed with the proctored examination were placed into MTH1825 and
approximately 17% (188/1098) of the students who were placed with the unproctored
examination were placed into remedial mathematics. The Pearson chi-square statistic
2

was significant,  (1, n = 1698) = 14.772, p =.000, indicating that the percentage of
students placing into remedial mathematics was higher for those students placed with the
proctored examination. The odds ratio associated with table 13 was computed to be
approximately 1.6 ((24.92*82.87)/(75.08*17.13)) indicating that the odds of being placed
into a remedial mathematics course with the proctored examination was about 60% more
than the odds of being placed into a remedial mathematics course with the unproctored
examination.

Table 13
Table 15
Crosstabulation of Type of Course by Type of Exam

Course Type
Non-Remedial
Remedial
Total

Type of Exam
Proctored
Unproctored
Total
75.08%
82.87% 78.95%
24.92%
17.13% 21.05%
100%
100%

96

Multiple Linear Regression Model for the Combined Data
Table 16 gives the results of the regression coefficients of the overall MLR model
used to predict placement examination scores. The type of examination used to place
students was a significant predictor of placement examination scores. The coefficient of
TYPE indicates that when holding all other predictors constant, students who were placed
with the proctored examination scored, on average, about 1 point lower on the placement
examination than the students who were placed with the unproctored examination. The
2

multiple linear regression equation has R = 0.523 so that the predictors explain 52.3%
of the variation in mathematics placement examination scores. The overall model is
significant, F(14, 1289) =100.947 p = .000.

97

Table 16
Results of Multiple Linear Regression Coefficients Use to Predict Mathematics
Placement Examination Scores (N=1304)
Predictor
b
SE(b) t
Intercept
-11.819
1.422 -8.312
GENDER
-0.239
0.216 -1.108
ACTM
0.739
0.032 23.080
HSGPA
1.245
0.459
2.710
HSMGPA
1.109
0.304
3.645
SYM
1.393
0.295
4.714
CABOVE
-0.520
0.451 -1.153
BLACK
-0.129
0.363 -0.357
HISPANIC -0.356
0.593 -0.601
INDIAN
-0.564
1.110 -0.508
ASIAN
0.257
0.495
0.520
TYPE
-1.186
0.216 -5.496
PSTT
-2.448
0.352 -6.957
ALG
-1.151
0.360 -3.196
OTHER
-1.474
0.300 -4.922
2
2
Note. R = 0.523 and adjusted R = 0.517

p
0.000
0.268
0.000
0.007
0.000
0.000
0.249
0.721
0.548
0.612
0.603
0.000
0.000
0.001
0.000

Residual Analysis of Overall Regression Model
Figure 15 below shows the histogram of the residuals generated by PASW
Statistics 18. The figure clearly shows that the distribution of the error terms is normal
with a mean of 0.
Also, the normal probability plot in figure 16 also supports the assumption of the
normality of the error terms. Although there are a few point that stray from the straight
line,

98

Figure 15. Histogram of the Residuals of the Regression Model for the Combined Data

99

Figure 16. Normal Probability Plot of Residuals for the Overall Data
Figure 17 checks the homoscedastiscity assumption. Clearly the errors are
randomly distributed about zero and therefore the assumption of homoscedastiscity is
verified.

100

Figure 17. Scatterplot of Standardized Residuals vs. Standardized Predicted Value

The assumptions of the multiple linear regression model are satisfied and
therefore, the model fits the data well.

101

Multiple Linear Regression Model for the Proctored Data
Table 17 gives the results of the regression coefficients used to predict
mathematics placement examination scores using the proctored data. The predictors in
this model explained 58.6% of the variation in the proctored mathematics placement
examination. The overall model fit was significant (F(13,436) = 47.524, p = .000). Unlike
the overall model, HSGPA and HSMGPA were not significant predictors of scores on the
proctored examination.

Table 14
Table 17
Summary of Multiple Linear Regression Coefficients used to Predict Placement using the
Proctored Examination (N=450)
Predictor
b
SE(b) t
Intercept
-11.080
2.152 -5.149
GENDER
-0.471
0.319 -1.474
ACTM
0.752
0.049 15.371
HSGPA
0.866
0.697
1.243
HSMGPA
0.789
0.469
1.680
SYM
1.380
0.428
3.223
CABOVE
-0.396
0.627 -0.632
BLACK
0.496
0.497
0.999
HISPANIC -1.068
0.988 -1.081
INDIAN
1.705
1.847
0.923
ASIAN
0.507
0.899
0.564
PSTT
-2.375
0.498 -4.772
ALG
-1.107
0.531 -2.084
OTHER
-1.026
0.439 -2.339
2
2
Note. R = 0.586 and adjusted R = 0.574

p
0.000
0.141
0.000
0.215
0.094
0.001
0.528
0.319
0.280
0.357
0.573
0.000
0.038
0.020

102

Residual Analysis of Proctored Multiple Linear Regression Model
Just as with the overall multiple regression model, the errors were checked for
normality and the equal variance assumption was checked. Figure 18 shows a histogram
of the error terms. The histogram shows that the error terms are normally distributed
with a mean of zero. The Q-Q plot (figure 19) also supports the normality of the errors
assumption.
Figure 20 is a plot of the standardized residuals against the standardized predicted
values. The errors appear to be randomly distributed about zero, so the homoscedastiscity
assumption is verified.

103

Figure 18. Histogram of the Residuals of the Regression Model for the Proctored
Regression Model

104

Figure 19. Normal Probability Plot of the Residuals for the Proctored Regression Model

105

Figure 20. Scatterplot of Standardized Residuals of the Proctored Regression Model

106

When the unproctored data was substituted into the above proctored model the
resulting scores were an estimate of the scores in which the students who were placed
with the proctored examination would have received if they had been placed with the
proctored examination. The correlation between these predicted scores and the actual
unproctored placement scores was significant, r(909) = 0.682, p = .000, indicating the
presence of a linear correlation between the predicted unproctored examination scores
and the actual unproctored examination scores.
Despite the high correlation coefficient, only about 35% of the students who were
placed with the unproctored examination would have received the same placement if they
would have been placed with the proctored examination. Additionally, approximately
21.5% and 43.5% of the students would have received higher and lower placements
respectively. Therefore, the unproctored examination does not place students in the same
way as the proctored examination. In fact, the unproctored examination appears to be
more likely to place students into higher level mathematics courses than the proctored
examination.

Multiple Linear Regression Model for the Unproctored Data
Table 18 gives the results of the regression coefficients used to predict
mathematics placement examination scores using the unproctored data. The predictors in
this model explained 48.2% of the variation in the unproctored mathematics placement
examination. The overall model fit was significant F(13,840) = 60.149, p = .000,
indicating that at least one of the predictors significantly predict the scores on the
unproctored placement examination. In this model, both HSGPA and HSMGPA are
107

significant predictors of the unproctored mathematics placement examination scores
whereas HSGPA and HSMGPA were not significant predictors of proctored placement
examination scores.

Table 15
Table 18
Summary of Multiple Linear Regression Coefficients used to Predict Placement using the
Unproctored Examination (N=909)
Predictor
b
SE(b) t
Intercept
-13.103
1.854 -7.068
GENDER
-0.100
0.284 -0.354
ACTM
0.737
0.042 17.724
HSGPA
1.454
0.598 2.431
HSMGPA
1.323
0.393 3.369
SYM
1.344
0.394 3.410
CABOVE
-0.600
0.620 -0.968
BLACK
-0.940
0.521 -1.804
HISPANIC
-0.102
0.739 -0.139
INDIAN
-1.263
1.391 -0.908
ASIAN
0.188
0.600 0.313
PSTT
-2.458
0.475 -5.170
ALG
-1.117
0.477 -2.343
OTHER
-1.700
0.396 -4.295
2
Note. R = 0.482 and adjusted R2 = 0.474.

p
0.000
0.723
0.000
0.015
0.001
0.001
0.333
0.072
0.890
0.364
0.754
0.000
0.019
0.000

Residual Analysis of the Unproctored Regression Model
Just as with the overall multiple regression model, the errors were checked for
normality and the equal variance assumption was checked. Figure 21 shows a histogram
of the error terms. The histogram shows that the error terms are normally distributed

108

with a mean of zero. The Q-Q plot (figure 122) also supports the normality of the errors
assumption.
Figure 23 is a plot of the standardized residuals against the standardized predicted
values. The errors appear to be randomly distributed about zero, so the homoscedastiscity
assumption is verified.

Figure 21. Histogram of Residuals of the Regression Model of the Unproctored
Regression Model

109

Figure 22. Normal Probability Plot of Residuals of the Unproctored Regression Model

110

Figure 23. Scatterplot of the Standardized Residuals vs. Standardized Predicted Values
for the Unproctored Regression Model

111

When the proctored data was substituted into the unproctored model the resulting
scores were an estimate of the scores in which the students who were placed with the
proctored examination would have received if they had been placed with the unproctored
examination. The correlation between these predicted proctored scores and the actual
proctored placement scores was significant, r(450) = .749, p = .000, indicating the
presences of a linear relationship between predicted proctored examination scores and
actual proctored examination scores.
Despite the high correlation coefficient, approximately 41% of the students who
were placed with the proctored examination would have received the same placement if
they would have been placed with the unproctored examination. Additionally,
approximately 39% and 20% of the students would have received higher and lower
placements respectively. Although there is indication that more of the students who were
placed with proctored examination would have received the same placement if they
would have been placed with the proctored examination than vice versa, this agreement
rate was less than ½.
Another technique that was used to determine if the unproctored examination is
functioning in the same way as the proctored examination was to determine if a particular
group of students received an advantage as a result of being placed with a particular
examination. So the success rates of several groups of students, under each type of exam
were determined using a contingency table analysis.

112

Success Rate for Combined Data
Table 19 shows the 2 x 2 contingency table of course success by type of
examination. As indicated almost 79% of the students who were placed with the
proctored examination were successful in their first college mathematics course
compared to approximately 86% of the students who were placed with the unproctored
examination. This difference in percentage of students who were successful was
2

significant  (1, n=1698) = 5.883, p = .015, indicating that course success was dependent
on the type of examination used for placement with the students who were placed with
the unproctored examination being more successful in their first college mathematics
course. Also, the ratio associated with table 17 is approximately 1.36 ( (83.2*21.6)/
(78.4*16.8) ) indicating that odds of a student being successful in their first college
mathematics course when placed with the unproctored examination is about 36% more
than the odds of a student being successful when placed with the proctored examination.
This was a surprising outcome.

Table 19
Table 16
2 x 2 Contingency Table of Math Course Outcome by Type of Examination

Course
Outcome
Unsuccessful
Successful
Total

Type of Examination
Proctored Unproctored
(n = 598) (n = 1098)
21.6%
16.8%
78.4%
83.2%
100.00% 100.00%

113

Success in Intermediate Algebra (MTH1825)
Table 20 shows a 2 x 2 contingency table of MTH1825 course outcome by the
type of examination. According to the table, approximately 73.3% of the students who
were placed with the unproctored examination were successful in MTH1825. Also,
approximately 53.1% of the students who were placed with the proctored examination
were successful in MTH1825. This difference in percentage of students who were
2

successful was significant  (1, n = 335) = 14.328, p = .000, indicating that the students
who were placed into MTH1825 with the unproctored examination were more likely to
be successful in MTH1825 than the students who were placed with the proctored
examination. The odds ratio associated with table 18 is 2.42
((73.30*46.90)/(53.10*26.70). So the odds of being successful in MTH1825 under the
unproctored examination was over 140% more than the odds of being successful in
MTH1825 under the proctored examination.

Table 17
Table 20
2 x 2 Contingency Table of MTH1825 Course Outcome by Type of Examination

Course
Outcome
Unsuccessful
Successful
Total

Type of Examination
Proctored Unproctored
(n = 147) (n = 188)
46.90%
26.70%
53.10%
73.30%
100.00% 100.00%

114

White students: Table 21 shows a 2 x 2 contingency table of MTH1825 course
outcome by the type of examination for the white students who were placed into
MTH1825. According to the table, approximately 78.2% of the white students who were
placed with the unproctored examination were successful in MTH1825 compared to
approximately 77.4% of the white students who were placed with the proctored
examination. The value of the test statistic to test the null hypothesis that the proportion
of white students in MTH1825 students who were successful when placed with the
unproctored examination is the same as the proportion of white students in MTH1825
students who were successful when placed with the unproctored examination was not
significant 2(1, n = 262) = 0.013, p = .908. Therefore there were no significant
difference in the percentage of white students who were successful in MTH1825 with the
proctored examination and the percentage of white students who were successful in
MTH1824 who were placed with the unproctored examination.

Table 21
Table 18
2 x 2 Contingency Table of Course Outcome by Type of Examination for the White
Students Enrolled in MTH1825

Course
Outcome
Unsuccessful
Successful
Total

Type of Examination
Proctored Unproctored
(n = 53)
(n = 119)
22.60%
21.80%
77.40%
78.20%
100.00% 100.00%

115

Black Students: Table 22 shows a 2 x 2 contingency table of MTH1825 success
rate by type of examination for the black students who were placed in MTH1825. As
indicated, approximately 38% of the black students who were placed into MTH1825 with
the proctored examination were successful compared to approximately 60.4% who were
placed with the unproctored examination. The value of the test statistic to test the null
hypothesis that the proportion of black students in MTH1825 students who were
successful when placed with the unproctored examination is the same as the proportion of
black students in MTH1825 students who were successful when placed with the
unproctored examination was significant 2(1, n = 119) = 5.762, p = .016. Thus, there
was a significant difference in the success rate for the black students who were placed
into MTH1825 with the proctored examination and the success rate of the black students
who were placed into MTH1825 with the unproctored examination.
The odds ratio of 2.49 ( (60.40*62)/(38*39.6) ) indicates that the odds of being
successful for black students who took the unproctored examination and enrolled in
MTH1825 were 149% more than the odds of being successful for black students who
took the proctored placement examination and enrolled in MTH1825.

116

Table 19
22
2 x 2 Contingency Table of Course Outcome by Type of Examination for the Black
Students Enrolled in MTH1825

Course
Outcome
Unsuccessful
Successful
Total

Type of Examination
Proctored Unproctored
(n = 71)
(n = 48)
62.00%
39.60%
38.00%
60.40%
100.00% 100.00%

Gender: Table 23 shows a 2 x 2 contingency table of MTH1825 course outcome
by type of examination for the male students who were placed in MTH1825. As
indicated, approximately 55% of the male students who were placed into MTH1825 with
the proctored examination were successful compared to approximately 73.3% who were
placed with the unproctored examination. The value of the test statistic to test the null
hypothesis that the proportion of male students in MTH1825 students who were
successful when placed with the unproctored examination is the same as the proportion of
male students in MTH1825 students who were successful when placed with the
2

unproctored examination was significant  (1, n =111) = 4.109, p =.043. Thus, for the
male students who placed into MTH1825, success was dependent on the type of
examination used for placement. Furthermore, the odds ratio of 2.26
(73.30*45.10)/(54.9*26.70) indicates that the odds of being successful for the male
students who enrolled into MTH1825 with the unproctored examination was 126% more
than the odds for the males who enrolled with the proctored examination.

117

Table 23
2 x 2 Contingency Table of Course Outcome by Type of Examination for the Male
Students Enrolled in MTH1825

Course
Outcome
Unsuccessful
Successful
Total

Type Of Examination
Proctored Unproctored
(n = 51)
(n = 60)
45.10%
26.70%
54.90%
73.30%
100.00%
100.00%

Table 24 shows a 2 x 2 contingency table of MTH1825 course outcome by type of
examination for the female students. As indicated, approximately 54.2% of the female
students who were placed into MTH1825 with the proctored examination were successful
compared to approximately 73.4% who were placed with the unproctored examination.
The value of the test statistic to test the null hypothesis that the proportion of female
students in MTH1825 students who were successful when placed with the unproctored
examination is the same as the proportion of female students in MTH1825 students who
2

were successful when placed with the unproctored examination was significant  (1, n =
124) = 8.976, p = .003. Furthermore, the odds ratio of 2.33 ((73.40*45.80)/(54.20*26.60)
indicates that the odds of being successful for the female students who were placed into
MTH1825 with the unproctored examination was 133% more than the odds for females
who were placed with the proctored examination.

118

Table 20
Table 24
2 x 2 Contingency Table of Course Outcome by Type of Examination for the Female
Students Enrolled in MTH1825

Course
Outcome
Unsuccessful
Successful
Total

Type of Examination
Proctored Unproctored
(n = 96)
(n = 128)
45.80%
26.60%
54.20%
73.40%
100.00% 100.00%

Senior Year Math: Table 25 shows a 2 x 2 contingency table of MTH1825
course outcome by type of examination for the students who were not enrolled in
mathematics during their senior year of high school. As indicated, approximately 51.6%
of the students who were placed into MTH1825 with the proctored examination were
successful compared to approximately 71.7% who were placed with the unproctored
examination. The value of the test statistic to test the null hypothesis the proportion of
students in the proctored group who were successful is the same as the proportion of
2

students in the unproctored group who were successful was not significant  (1, n = 84)
= 3.437, p = .064.

119

Table 25
Table 21
2 x 2 Contingency Table of Course Outcome by Type of Examination for Students Who
Were Not Enrolled in Math during Senior Year of High School and Enrolled in MTH1825

Course
Outcome
Unsuccessful
Successful
Total

Type Of Examination
Proctored Unproctored
(n = 31)
(n = 53)
48.40%
28.30%
51.60%
71.70%
100.00% 100.00%

Table 26 shows a 2 x 2 contingency table of MTH1825 course outcome by type of
examination for the students who were enrolled in a mathematics course during their
senior year of high school. As indicated, approximately 56.0% of the white students
who were placed into MTH1825 with the proctored examination were successful
compared to approximately 73.8% who were placed with the unproctored examination.
2

The difference in the percentages was significant  (1, n =321) = 80.63, p = .005. The
odds ratio 2.21 ((73.8*44.00)/(56.00*26.20)). So the students who were enrolled in a
mathematics course during their senior year of high school were more likely to be
successful in MTH1825 if placed with the unproctored examination.

120

Table 26
2 x 2 Contingency Table of MTH1825 Course Outcome by Type of Examination for
Students Who Were Enrolled in Math during Senior Year of High School

Course
Outcome
Unsuccessful
Successful
Total

Type Of Examination
Proctored Unproctored
(n = 109) (n = 122)
44.00%
26.20%
56.00%
73.80%
100.00% 100.00%

Last high school mathematics course: Table 27 is a 2 x 2 contingency table of
the MTH1825 course outcome by type of examination for students whose last high
school math course was either calculus, pre-calculus, or trigonometry. As indicated,
approximately 60% of the students who were placed into MTH1825 with the proctored
examination were unsuccessful compared to approximately 76.5% who were placed with
the unproctored examination. The value of the test statistics used to test the null
hypothesis that there is no difference between the proportion of these students who were
successful in the proctored group is the same as the proportion of these students who
2

were successful in the unproctored group was significant  (1,151) = 4.794, p = .029.
The odds ratio was 2.17 ((76.50*40.00)/(60.00*76.50)) indicating that the students who
were placed with the unproctored examination and took calculus, pre-calculus, or
trigonometry during their senior year of high school were more likely to be successful in
MTH1825 than the students who were placed with the proctored examination.

121

Table 22
Table 27
2 x 2 Contingency Table of the MTH1825 Course Outcome by Type of Examination for
Students Whose Last High School Math Course was Calculus, Pre-Calculus, or
Trigonometry in High School

Course
Outcome
Unsuccessful
Successful
Total

Type Of Examination
Proctored Unproctored
(n = 70)
(n = 81)
40.00%
23.50%
60.00%
76.50%
100.00% 100.00%

Table 28 shows the 2 x 2 contingency table of the MTH1825 course outcome by
type of examination for students whose last high school mathematics class was either
algebra I, algebra II, or geometry. As indicated, approximately 37.5% of these students
who were placed with the proctored examination were successful compared to
approximately 71.4% who were placed with the unproctored examination. The value of
the test statistics used to test the null hypothesis that there is no difference between the
proportion of male students who were successful in the proctored group is the same as the
proportion of students who were successful in the unproctored group was significant
2

 (1, n = 81) = 9.149, p = .002. The odds ratio was 2.17 ((76.5*40)/(60*23.5)). So the of
being successful for students who enrolled in MTH1825, who were placed with the
unproctored examination and took either algebra I, algebra II, or geometry as their last
high school math course was over 2 times the odds of being successful for students who

122

enrolled in MTH1825, who were placed with the proctored examination, and took either
algebra I, algebra II, or geometry as their last high school math course.

Table 28
2 x 2 Contingency Table of MTH1825 Course Outcome by Type of Examination For The
Students Whose Last High School Math Course Was Algebra I, Algebra II, or Geometry

Course
Outcome
Unsuccessful
Successful
Total

Type Of Examination
Proctored Unproctored
(n = 32)
(n = 49)
62.50%
28.60%
37.50%
71.40%
100.00% 100.00%

Table 29 shows the 2 x 2 contingency table of the MTH1825 course outcome by
type of examination for students whose last high school mathematics course was statistics
or probability. As indicated, approximately 57.1% of these students that were placed
with the proctored examination were successful in MTH1825 compared to 66.7% of
those who were placed with the unproctored examination. The value of the test statistics
used to test the null hypothesis that there is no difference between the proportion of male
students who were successful in the proctored group is the same as the proportion of
2

students who were successful in the unproctored group was significant  (1, n = 67) =
0.632, p = .427.

123

Table 29
2 x 2 Contingency Table of MTH1825 Course Outcome by Type of Examination for the
Students Whose Last High School Math Course Was Statistics or Probability

Course
Outcome
Unsuccessful
Successful
Total

Type Of Examination
Proctored Unproctored
(n = 28)
(n = 39)
42.90%
33.30%
57.10%
66.70%
100.00% 100.00%

Success in College Algebra (MTH103)
Table 30 shows a 2 x 2 contingency table for MTH103 course outcome by type of
examination. As indicated, approximately 85.1% of the students who were placed in
MTH103 with the proctored examination were successful compared to approximately
77.1% who were placed with the unproctored examination. The value of the test statistics
used to test the null hypothesis that there is no difference between the proportion of
MTH103 students who were successful in the proctored group is the same as the
proportion of MTH103 students who were successful in the unproctored group was not
2

significant  (1, n = 270) = 2.016, p = .156.

124

Table 30
2 x 2 Contingency Table of MTH103 Course Outcome by Type of Examination
Type Of Examination
Course
Proctored
Unproctored
Outcome
(n = 101)
(n = 169)
Unsuccessful 14.90%
21.90%
Successful
85.10%
77.10%
Total
100.00%
100.00%

Success in College Algebra and Trigonometry (MTH116)
Table 31 shows the 2 x 2 contingency table of the math 116 course outcome by
type of examination. As indicated, approximately 81.1% of the students who were placed
in math 116 with the proctored examination were successful compared to approximately
81.1% who were placed with the unproctored examination. The value of the test statistics
used to test the null hypothesis that there is the proportion of students who were
successful in the proctored group is the same as the proportion of students who were
2

successful in the unproctored group was not significant  (1, n = 127) = 0.000, p = .994.

Table 31
2 x 2 Contingency Table of Math 116 Course Outcome by Type of Examination

Course
Outcome
Unsuccessful
Successful
Total

Type of Examination
Proctored Unproctored
(n = 53)
(n = 74)
18.9%
18.9%
81.1%
81.1%
100.0%
100.0%

125

Success in Survey of Calculus 1 (MTH124)
Table 32 shows the percentage of students who were unsuccessful in MTH124 by
the type of examination used to place the students. As indicated, approximately 94.1% of
the students who were placed in MTH124 with the proctored examination were
successful compared to approximately 90.8% who were placed with the unproctored
examination. The value of the test statistics used to test the null hypothesis that there is
no difference between the proportion of students who were successful in the proctored
group is the same as the proportion of students who were unsuccessful in the unproctored
2

group was not significant  (1, n = 121) = 0.354, p = .552.

Table 32
2 x 2 Contingency Table of MTH124 Course Outcome by Type of Examination
Course
Outcome
Unsuccessful
Successful
Total

Proctored
(n = 34)
5.9%
94.1%
100.0%

9.2%
90.8%
100.0%

Success in Calculus I (MTH132)
Table 33 shows the percentage of students who were unsuccessful in MTH132 by
the type of examination used to place the students. As indicated, approximately 93.7% of
the students who were placed in MTH132 with the proctored examination were
successful compared to approximately 86.1% who were placed with the unproctored
examination. The value of the test statistics used to test the null hypothesis that there is
126

no difference between the proportion of students who were successful in the proctored
group is the same as the proportion of students who were unsuccessful in the unproctored
2

group was not significant  (1, n = 147) = 1.368, p = .242.

Table 33
2 x 2 Contingency Table of MTH132 Course Outcome by Type of Examination
Course
Outcome
Unsuccessful
Successful
Total

Proctored
(n = 32)
6.3%
93.8%
100.0%

Unproctored
(n = 115)
13.9%
86.1%
100.0%

The contingency table analysis of the overall data set revealed that the students
who were placed with the unproctored examination were more likely to be successful in
the course in which they were placed. Upon further analysis, it was revealed that this
phenomenon was only true of the university‟s remedial mathematics course (MTH1825)
and not the non-remedial mathematics courses.

Question 3
How well does the mathematics placement examination predict success in students’
first college mathematics course? Furthermore, can the prediction of success in
students’ first college mathematics course be improved using ACT Mathematics score,
high school GPA, the type of mathematics courses taken in high school, and whether
or not students took a mathematics course during their senior year of high school?
127

Michigan State University currently uses the results of the mathematics placement
examination to place students into their first college mathematics course.

Predicting Success in Intermediate Algebra (MTH1825)
Since Michigan State University uses the scores on the placement examination to
place students into their first college math course, the first logistic regression model
(Model 1) uses mathematics placement examination score (MPE) as the predictor and
PASS as the criterion. Table 34 shows the logistic regression results. As indicated in the
table, the odds ratio for MPE is 1.386. Thus, for a 1 point increase in placement
examination score, the odds of passing MTH1825 increase by about 39%. The Hosmer
2

and Lemeshow Test is not significant ( (5 , N = 335 ) = 9.877, p = .079) indicating that
there is a linear relationship between MPE and the log odds of being successful in
Intermediate Algebra and therefore Model 1 fits the data well. The binary logistic
regression equation is given by
Ln(odds(success)) = -1.470 + 0.326*MPE

Table 34
Summary of Logistic Regression Results of Model 1

Predictor b
MPE
0.326
Constant -1.470

S.E. Wald
0.065 25.617
0.423 12.097

df
1
1

Sig.
0.000
0.001

128

Odds
Ratio
1.386
0.230

95% C.I. for the
Odds Ratio
(1.221, 1.573)

Student #218 was unsuccessful in MTH1825. This student had a placement
examination score of 5. Using the logistic regression equation above and equation 4, the
predicted probability of success in MTH1825 for student #216 was 0.539. Thus the above
model predicted this student to be successful although the student was not successful.
Table 35 is the classification table that corresponds to the logistic regression
model above. The model correctly predicted 87.6% (191/218) of the successes but only
21.4% (25/117) of the non-success. Overall, the correct decision was made 64.4% of the
time. Also, a total of 283 students were predicted to be successful in MTH1825 based on
their scores on the placement examination but 92 of them were not successful. Hence this
model has a false positive rate of 32.5% (92/283). Similarly, the model has a false
negative rate of 52% (27/52).
Table 35
Classification Table Corresponding to Model 1
Predicted

observed

Unsuccessful
Successful

Unsuccessful Successful
25
92
27
191
Hit Rate =64.4%

In an attempt to improve the prediction of success in Intermediate Algebra,
ACTM, HSGPA, SYM, last math course taken in high school, and the type of
mathematics used for placement was added to Model 1. The resulting model will be
referred to as Model 2. The logistic regression results are given in table 36.
The table shows that SYM, TYPE, and the last math course taken in high school.
SYM and the last high school math course taken in high school were kept in the model
129

because researchers (Hill, 2006; Adelman 1999, Berry 2003) concluded that taking a
challenging math course in high school (beyond algebra 2) and taking mathematics
during senior year is important to success in college level mathematics as well as college
success overall. Additionally, TYPE remained in the model because of the a priori
interest in the effect of type of examination on success in first college math course.

Table 36
Summary of Logistic Regression Coefficients for Model 2

Predictor
MPE
ACTM
HSGPA
TYPE
SYM
PSTT
ALG
a
Other
Constant
a

b
0.231
0.235
1.337
-0.406
0.227
0.380
0.072

SE(b)
0.085
0.060
0.436
0.292
0.356
0.528
0.395

-0.261 0.388
-9.584 1.878

Wald
7.487
15.476
9.400
1.932
0.407
0.519
0.033

df
1
1
1
1
1
1
1

0.453 1
26.036 1

p
0.006
0.000
0.002
0.164
0.523
0.471
0.856

Odds
Ratio
1.260
1.264
3.806
0.667
1.255
1.462
1.074

95% C.I. for
the Odds Ratio
(1.068 , 1.487)
(1.125 , 1.421)
(1.620 , 8.943)
(0.376 , 1.181)
(0.625 , 2.520)
(0.520 , 4.112)
(0.495 , 2.332)

0.501
0.000

0.770
0.000

(0.360 , 1.649)

Includes FST (functions, statistics, and trigonometry), Business math, Consumer math,

Math Analysis, Discrete Math

MPE, ACTM, and HSGPA were all significant predictors of success in
Intermediate Algebra. According to the table, a 1 point increase in the placement
examination score increases the odds of being successful in MTH1825 by about 26%. A
1 point increase in ACT Mathematics score also increases the odds of success in
Intermediate Algebra of about 26%. The odds ratio for HSGPA is 3.806 and therefore, a
1 point increase in HSGPA increases the odds of being successful in MTH1825 by about
130

280%. However, if an increase of 0.1 points in HSGPA is considered, then the odds ratio
would be 3.806

0.1

= 1.14. Thus an increase of 0.1 points in the HSGPA results in an

increase in the odds of being successful in Intermediate Algebra of about 14%. The odds
ratio for TYPE indicated that the students who were placed into MTH1825 with the
proctored examination had odds of success that were 0.667 times the odds of success of
the students who were placed into MTH1825 with the unproctored examination. This
odds ratio was not statistically significant.
The reader should be reminded that the variables PSTT, ALG, and Other are
dummy variables created from variable LAST (last high school mathematics course
taken) where a calculus, pre-calculus, or trigonometry was used as the reference group.
Thus the odds ratio for ALG indicates that an Intermediate Algebra student, whose last
high school math course was algebra I, algebra II, or geometry had odds of passing
Intermediate Algebra that were 1.074 times greater than the odds of an Intermediate
Algebra student who took calculus, pre-calculus, or trigonometry as their last high school
math course. However, this odds ratio was not significant.
The values of the deviance for Model 1 and Model 2 were 403.989 and 298.639
2

respectively. This difference was significant  (7, n = 332) = 105.35, p = .000. Thus,
Model 2 improves the prediction of success in Intermediate Algebra beyond Model 1.
The logistic regression equation is given by
Ln(odds(success)) = -9.584 + 0.231*MPE + 0.235*ACTM + 1.337*HSGPA –
0.406*TYPE – 0.227*SYM + 0.380*PSTT +0.072*ALG – 0.261*Other
Students #218 had MPE = 5, ACTM = 17, HSGPA = 2.78 , was placed with the
proctored examination, and was enrolled in probability and statistics during his senior year of

131

high school Using the logistic regression model and equation 4, this student‟s predicted
probability of passing MTH1825 of 0.374. If this student was placed with the unproctored
examination, the probability of being successful would increase to 0.472. Using only the
placement examination score, this student was predicted to pass. Using information in the
student‟s background, in addition to placement examination score, this student was predicted
to be unsuccessful; and he was unsuccessful. With the low ACT Mathematics score, low
mathematics placement examination scores, and high school GPA below 3.0, this student
could be singled out and referred to academic resources within the university (e.g. tutoring).
Table 37 gives the classification table associated with Model 2. The model correctly
predicted 89.5% (170/190) of the successes and 54.2% (52/96) of the non-successes. Overall,
the correct decision was made 77.6% of the time. Also, using this model, a total of 214
students were predicted to be successful in MTH1825, but 44 of them were not successful.
Hence this model has a false positive rate of 20.6% (44/214). Similarly, the model has a false
negative rate of 27.7% (20/72).

132

Table 37
Classification Table for Model 2
Predicted

observed

Unsuccessful Successful
52
44
20
170
Hit Rate =77.6%

Unsuccessful
Successful

As indicated, Model 2 fits the MTH1825 data better than Model 1. Furthermore,
Model 2 has a smaller false positive rate than Model 1.

Predicting Success in College Algebra (MTH103)
Table 38 gives the logistic regression results for the logistic regression model
used to predict success in MTH103 with placement examination scores (Model 3). As
indicated by the model, placement examination score is a significant predictor of the log
odds of being successful in College Algebra. The odds ratio for MPE indicates that a 1
point increase in the placement examination score increases the odds of being successful
in college algebra by about 25%. The Hosmer and Lemeshow Test produced a
2

significant chi-square value,  (5, N = 477) = 4.014, p = .547. Therefore, Model 3 fits
the data well. Furthermore the Omnibus Test of Model Coefficients produces a
2

significant chi-square value,  (1, N = 477) = 15.502, p = .000 so that Model 3 fits the
data better than the intercept only model. The binary logistic regression equation is given
by
Ln(odds(success)) = -0.854 + 0.224*MPE
133

Table 38
Summary of Logistic Regression Results for Model 3

Predictor
MPE
Constant

b SE(b)
0.224 0.060
-0.854 0.647

Wald df
14.096 1
1.742 1

p
0.000
0.187

Odds
Ratio
1.251
0.426

95% C.I. for the
Odds Ratio
(1.113, 1.407)

Student #1681 received a placement examination score of 10. Using Model 3 and
equation 4 the predicted probability of success in college algebra for a student who scores
10 on the placement examination is 0.80. Student #1681 was not successful.
The classification table corresponding to Model 3 is given in table 39. The model
correctly predicted 100% of the successes and 1.3% of the failures for an overall “hit
rate” of 83.4% (388/477). Model 3 predicted that 476 of the students would be
successful in College Algebra. However, 79 of these students were unsuccessful. Thus,
Model 3 has a false positive rate of 16.6% (79/476).

Table 39
Classification Table Corresponding to Model 3
Predicted

observed

Unsuccessful
Successful

Unsuccessful Successful
1
79
0
397
Hit Rate =83.4%

Attempting to improve on the prediction of success in MTH103, Model 4 was
created by adding the variables ACTM, HSGPA, SYM, TYPE, PSTT, ALG, and Other to
Model 3. The logistic regression results of Model 4 are given in Table 40. MPE, ACTM,
134

and HSGPA are all significant predictors of the log odds of being successful in college
algebra. The odds ratio for ACTM indicates that when all the other predictors are held
constant, the odds of being successful in college algebra increases by about 11% for a 1
point increase in ACT Mathematics score. The odds ratio for HSGPA indicates that,
when all the other predictors are held constant, the odds of being successful in college
algebra increases by over 270% when the for a 1 point increase in the high school GPA.
However, if an increase of 0.1 points in HSGPA is considered, then the odds ratio would
0.1

be 3.794

= 1.14 and an increase of 0.1 points in high school GPA increases the odds of

being successful in college algebra by about 11%.
The odds ratio for TYPE indicated that, when all the other predictors are held
constant, the odds of being successful in college algebra when placed with the proctored
examination is about 87% more than the odds of being successful in college algebra when
placed with the unproctored examination. However, the variable TYPE was not
significant.
The Hosmer Lemeshow chi-square produced a non significant chi-square value,
2

 (8, n = 405) = 37.15, p = .882. Thus, Model 4 fits the data well. Also, the omnibus test
2

of model coefficients produced a significant chi–square,  (7, n = 405) = 38.455, p =
.000 indicating that Model 4 fits better than the intercept only model. The value of the
deviance for Model 3 and Model 4 were 215.125 and 165.239 respectively. The
2

difference was significant  (7, n = 405) = 46.886, p = .000. Therefore the Model 4
significantly adds to the prediction of success in college algebra beyond placement
examination score. The logistic regression model is given by
135

Ln(odds(PASS)) = -8.117 + 0.293*MPE+ 0.105*ACTM + 1.333*HSGPA +
0.626*TYPE + 0.243*SYM + 0.015*PSTT – 0.628*ALG +0.240*Other

Table 40
Summary of Logistic Regression Results for Model 4

Predictor
MPE
ACTM
HSGPA
TYPE
SYM
PSTT
ALG
a
Other
Constant
a

b
0.243
0.105
1.333
0.626
0.243
0.015
-0.628

SE(b).
0.078
0.050
0.441
0.330
0.078
0.493
0.402

0.240 0.426
-8.117 2.027

Wald
9.780
4.449
9.141
3.600
9.780
0.001
2.434

df
1
1
1
1
1
1
1

0.318 1
16.039 1

p
0.002
0.035
0.002
0.058
0.002
0.976
0.119

Odds
Ratio
1.274
1.111
3.794
1.870
1.274
1.015
0.534

0.573
0.000

1.271
0.000

95% C.I. for
the Odds
Ratio
(1.095, 1.481)
(1.007, 1.224)
(1.598, 9.006)
(0.980, 3.569)
(0.480, 2.141)
(0.386, 2.667)
(0.243, 1.174)
(0.552, 2.930)

Includes FST (functions, statistics, and trigonometry), Business math, Consumer math,

Math Analysis, Discrete Math

Student #1016 was placed into college algebra with the unproctored examination
with a score of 11. This student had ACTM = 21, HSGPA = 3.722, and his last
mathematics course was Pre-calculus during his junior year of high school. Using Model
4 and equation 4, his predicted probability of success was given as 0.848. A similar
student placed with the proctored examination would have a predicted probability of
success of 0.913. This student was not successful in college algebra.

136

The classification table is given in table 41. The table shows Model 4 correctly
predicted almost 84.2% of correct course outcomes. Model 4 predicted 398 successes but
62 of the students were actually unsuccessful corresponding to a false positive of 15.6%
(62/398). The model also gives a false negative rate of 28.6% (2/7). Model 4 improved
the “hit rate” of Model 3 and have lower false positive and false negative rates.

Table 41
Classification Table Corresponding to Model 4
Predicted

observed

Unsuccessful
Successful

Unsuccessful Successful
5
62
2
336
Hit Rate = 84.2%

Predicting Success in College Algebra and Trigonometry (MTH116)
Table 42 shows the binary logistic regression results for the model used to predict
the log odds of success in MTH116 using MPE as the only predictor (Model 5). The
odds ratio for MPE is 1.020 indicating that the odds of being successful in MTH116
increases by about 2% for a 1 point increase in placement examination score. This ratio is
2

not significant. Also the Hosmer and Lemeshow Test is significant ( (5 , N = 249) =
4.774, p =.573) indicating that Model 5 fits the data well. However, the omnibus test of
2

model coefficients produced a non-significant chi-square ( (1, N = 249) = 0.095, p

137

=.758) indicating that Model 5 does not fit the data better than the intercept only model.
The binary logistic regression equation is given by
Ln(odds(success) )= 1.324 + 0.019*MPE

Table 42
Summary of Logistic Regression Results for Model 5

Predictor b
MPE
0.019
Constant 1.314

SE(b).
0.063
0.927

Wald df
0.094
1
2.009
1

p
0.759
0.156

Odds
Ratio
1.020
3.722

95% C.I. for the
Odds Ratio
(0.901, 1.154)

Student #1510 scored 12 on the placement examination. Using Model 5 and
equation 4, the predicted probability of being successful in MTH116 was 0.824.
Table 43 is the classification table for Model 5. Model 5 gives a hit rate of
83.1%. Also, 249 of the students were predicted to be successful but 42 of them were
not successful corresponding to a false positive rate of 16.9%.

Table 43
Classification Table Corresponding to the Model 5
Predicted

observed

Unsuccessful
Successful

Unsuccessful Successful
0
42
0
207
Hit Rate = 83.1%
138

Attempting to improve on the prediction of the log odds of success in MTH116,
the variables ACTM, HSGPA, SYM, TYPE, PSTT, ALG, and Other were added to
Model 5. The logistic regression results of the new model (Model 6) are given in table 44.
ACTM and HSGPA are both significant predictors of the log odds of being successful in
MTH116. The odds ratio for ACTM indicates that the odds of being successful in
MTH116 increases by almost 17% for each 1 point increase in ACT Mathematics score.
The odds ratio for HSGPA indicates that the odds of being successful in MTH116
increases by over 17% for a 1 point increase in high school GPA. If we consider an
increase of 0.1 points, then the odds ratio would be 18.461

0.1

= 1.34. Thus, for every 0.1

point increase in high school GPA, the odds of being successful in MTH116 increased by
about 34%. The odds ratio for TYPE is 0.614 indicating that the odds of being
successful in MTH116 when placed with the proctored examination is about 0.614 times
the odds of being successful in MTH116 when placed with the unproctored examination.
This odds ratio is not significant. Although the only significant predictors were ACTM
and HSMGPA, the other variables remained in the model because either prior research
indicated their effect on success in mathematics or the a priori interest in their effect on
course success. The value of the deviance for Model 5 and Model 6 were 225.887 and
2

147.282 respectively. The difference was significant  (7, n = 209) = 78.605, p =.000.
Therefore the predictors that were added to Model 5 significantly adds to the prediction
of success in MTH116 beyond placement examination score. Model 5 is given by:
Ln(odd(success)) = - 12.652 + 0.052*MPE + 0.155*ACTM +2.916*HSGPA –
0.495*SYM + 0.243*TYPE- 0.530*PSTT – 0.142*ALG + 0.071*Other

139

Table 44
Logistic Regression Results for Model 6
Odds
95% C.I. for the
Ratio
Odds Ratio
1.053
(0.875 , 1.262)
1.168
(1.008 , 1.362)
18.461 (3.318 , 102.725)
0.610
(0.130 , 2.854)
1.275
(0.496 , 3.274)
0.589
(0.134 , 2.585)
1.152
(0.161 , 8.240)

Predictor
b
SE(b).
Wald df
P
MPE
0.052
0.092
0.317 1
0.573
ACTM
0.155
0.075
4.275 1
0.039
HSGPA
2.916
0.876 11.085 1
0.001
SYM
-0.495
0.788
0.395 1
0.530
TYPE
0.243
0.481
0.255 1
0.614
PSTT
-0.530
0.755
0.493 1
0.483
ALG
0.142
1.004
0.020 1
0.888
a
Other
0.071
0.637
0.013 1
0.911 1.074
(0.308 , 3.740)
Constant
-12.652
3.780 11.203 1
0.001 0.000
a
Includes FST (functions, statistics, and trigonometry), Business math, Consumer math,
Math Analysis, Discrete Math

Table 45 shows the classification table associated with Model 6. Model 6
improved the “hit rate” of Model 5 to 86.9%. Also, Model 6 predicted that 202 students
would be successful in MTH116, but 26 were unsuccessful. This resulted in a false
positive rate of 12.9% (26/202). Similarly, Model 6 resulted in a false negative rate of
25%.

140

Table 45
Classification Table Corresponding to Model 6
Predicted

observed

Unsuccessful Successful
3
26
1
176
Hit Rate = 86.9%

Unsuccessful
Successful

Predicting Success in Survey of Calculus I (MTH124)
Table 46 shows the logistic regression results for predicting the log odds success
in MTH124 using MPE as the only predictor (Model 7). The odds ratio for MPE indicate
that a 1 point increase in placement examination score results in an increase in the odds
of being successful in MTH124 of about 11%. The Hosmer and Lemeshow chi-square
2

statistic was not significant  (8, n = 256) = 8.873, p =.353 indicating that Model 7 fits
2

the data well. However the omnibus test of model coefficients was not significant  (1, n
= 256) = 2.355, p = .125 indicating that Model 7 does not fit the data better than the
intercept only model. Model 7 is given by
Ln(odds(success)) = 0.758 + 0.104* MPE
Table 46
Summary of Logistic Regression Results for Model 7

Predictor
MPE
Constant

b
0.104
0.758

SE(b).
0.067
1.206

Odds
Wald df p
Ratio
2.396 1 0.122
1.110
0.395 1 0.530
2.133

141

95% C.I. for the
Odds Ratio
(0.973,1.267)

Using Model 7 and equation 4, the probability that a person with a score of 15 on
the placement examination will be successful in MTH124 is 0.913
The classification table for Model 7 is given in table 47. Model 7 has predicted
that 256 of the students enrolled in MTH124 would be successful. However, 17 of the
students were not successful resulting in a false positive rate of 6.6%. The model has a
“hit rate” of 93.4%.

Table 47
Classification Table Corresponding to Model 7
Predicted

observed

Unsuccessful
Successful

Unsuccessful Successful
0
17
0
239
Hit Rate = 93.4%

Just as with previous models, Model 8 was constructed by adding ACTM,
HSGPA, SYM, TYPE, PSTT, ALG, and Other to Model 7. The logistic regression results
are given in Table 48. HSGPA was the only significant predictor of success in MTH124.
The odds ratio for HSGPA indicates that the odds of being successful in MTH124 is
almost 26 times the odds of being unsuccessful when high school GPA increases by 1
point. However, if an increase in HSGPA of 0.1 point is considered then the odds of
being successful in MTH124 is 25.876

0.1

= 1.38 times the odds of being unsuccessful in

MTH124. Also, when considering an increase in HSGPA of 0.1 point, the 95%
confidence interval is reduced to (1.09, 1.73). The Hosmer and Lemeshow test produced
142

2

a non significant chi-square  (8 , n =183) = 3.833, p = .872. Therefore, Model 7 fits
the data well. Although the other variables were not significant, they remained in the
model because of the interest in their effect on success. The deviance statistics for Model
2

7 and 8 were 122.697 and 71.286.respectively. This difference was significant ( (7 , n
=183) = 51.411, p =.000). Therefore, the predictors that were added to Model 7 added to
the prediction of the log odds of success in MTH124.

Table 48
Logistic Regression Results for Model 8
Odds
Ratio
0.983
25.876
0.737
0.912
1.38E+08
0.587

95% C.I. for the
Odds Ratio
(0.773,1.249)
(2.749 ,243.522)
(0.058,7.994)
(0.207,4.031)

Predictor b
SE(b).
Wald
df p
ACTM
-0.017
0.122
0.020
1 0.887
HSGPA
3.253
1.144
8.089
1 0.004
SYM
-0.305
1.216
0.063
1 0.802
TYPE
-0.092
0.758
0.015
1 0.904
PSTT
18.731 8317.058
0.000
1 0.998
ALG
-0.533
1.245
0.183
1 0.669
(0.051,6.739)
a
Other
-0.265
0.913
0.084
1 0.771
0.767
(0.128,4.583)
Constant
-8.045
4.836
2.768
1 0.096
0.000
a
Includes FST (functions, statistics, and trigonometry), Business math, Consumer math,
Math Analysis, Discrete Math

Table 49 shows the classification table for Model 8. Model 8 improved the “hit
rate” to 94%. Model 8 has a false positive of 6% (11/182).

143

Table 49
Classification Table Corresponding to Model 8
Predicted

observed

Unsuccessful
Successful

Unsuccessful Successful
0
11
1
171
Hit Rate = 94.0%

Predicting Success in Calculus 1( MTH132)
Table 50 shows the logistic regression results for predicting success in MTH132
with math placement examination scores only (Model 9). Model 9 indicates that the odds
of being successful in MTH132 is 1.298 times the odds of being unsuccessful in
MTH132 for a 1 point increase in the placement examination score. The Hosmer and
2

Lemeshow test produced a non significant chi-square value  (7, N = 177) = 6.415,
0.492. This indicates that the model containing MPE fits the data well. The logistic
regression equation is given by
Ln (odds(success)) = -1.929+ 0.181*MPE

144

Table 23
Summary of Logistic Regression Results for Model 9

Predictor b
SE(b)
MPE
0.181 0.057
Constant
-1.929 1.123

Wald
9.953
2.949

Odds
df p
Ratio
1 0.002 1.298
1 0.060 0.145

95% C.I. for the
Odds Ratio
(1.071 , 1.340)

Using Model 9 and equation 4, a person who scores a 19 on the placement
examination has a predicted probability of being successful in MTH132 of 0.819.
The classification table for Model 9 is given in table 51. Model 9 has a hit rate of
84.2%. Furthermore, the model predicted that 175 students would be successful but 27
students were not successful. Thus, the model had a false positive of 15.4 % (27/175).

Table 51
Classification Table for Model 9
Predicted

observed

Unsuccessful
Successful

Unsuccessful Successful
1
27
1
148
Hit Rate = 84.2%

Model 10 was created by adding ACTM, HSGPA, SYM, TYPE, PSTT, ALG, and
Other. MPE and HSGPA are the only significant predictors in Model 10. However, just
as in the previous models all the predictors were retained in the model. The logistic
regression results are given in table 52. The odds ratio for ACTM indicates that the odds
of being successful in MTH132 are 1.142 times the odds of being unsuccessful.
145

However, the odds ratio is not significant. The odds ratio for HSGPA indicates that the
odds of being successful in MTH132 are 18.355 times the odds of being unsuccessful. If
an increase of 0.1 in the GPA is considered then the odds of being successful in MTH132
is 1.34 (18.355

0.1

) times the odds of being unsuccessful. The confidence interval is then

reduced to (1.065, 1.679). The model also shows that the odds ratio for TYPE is 5.132
indicating that the odds of being successful in MTH132 when placed with the proctored
examination is over 5 times the odds of being successful in MTH132 when placed with
the unproctored examination. This odds ratio is not significant. The table also shows
that students whose last high school mathematics course was either calculus, precalculus, or trigonometry had higher odds of being successful in MTH132, than the
students whose last high school mathematics course was of course other than calculus,
pre-calculus, or trigonometry. The Hosmer and Lemeshow test produced a significant
2

chi-square (  (8 , n =177) = 18.567, p = .017). The omnibus test of model coefficient,
2

however, produces a significant chi-square statistic  (7 , n =177) = 25.61, p =.001
indicating the Model 10 fits better than the intercept only model. The deviance statistics
for Model 9 and Model 10 were 144.200 and 84.94 respectively. The difference was
2

significant  (6 , n =121) = 59.506, p = .000. Thus, Model 10 fits the MTH132 data
better than Model 9. Model 10 is given by
Ln(odds(success)) = -16.357 + 0.241*MPE+ 0.133*ACTM + 2.910*HSGPA +
1.636*TYPE – 0.809*SYM - 0.455*PSTT – 3.775*Other

146

Table 52
Logistic Regression Results for Model 10
Odds
Ratio
1.273
1.142
18.355
0.445
5.132
0.635

95% C.I. for the Odds
Ratio
(1.053 , 1.538)
(0.937, 1.393)
(1.889, 178.315)
(0.046, 4.343)
(0.777, 33.892)
(0.052, 7.748)

Predictor b
SE(b)
Wald df p
MPE
0.241
0.096
6.248 1 0.012
ACTM
0.133
0.101
1.731 1 0.188
HSGPA
2.910
1.160
6.292 1 0.012
SYM
-0.809
1.162
0.485 1 0.486
TYPE
1.636
0.963
2.884 1 0.089
PSTT
-0.455
1.276
0.127 1 0.722
a
Other
-3.775
1.561
5.852 1 0.016
0.023
(0.001, 0.488)
Constant
-16.357
5.509
8.817 1 0.003
0.000
a
Includes FST (functions, statistics, and trigonometry), Business math, Consumer math,
Math Analysis, Discrete Math

Student #1666 was placed into MTH132 with a score of 22 on the unproctored
examination, had an ACT Mathematics score of 27, had a high school GPA of 3.369, and
his last course was pre-calculus during his senior year of high school. Model 10 and
equation 4 predicted that the student would be successful in MTH132 with probability
0.822.
The classification table for Model 10 is given in table 53. Model 10 improves the
“hit rate” of Model 9 to 89.1%. Model 10 also decreased the false positive rate to 11.5%.

147

Table 53
Classification Table for Model 10
Predicted

observed

Unsuccessful
Successful

Unsuccessful Successful
5
14
1
117
Hit Rate = 89.1%

Question 4
How do the grades in each course compare across different levels of placement
examination scores?
When students enroll in a course in which they received a placement examination
score that was higher than the minimum score needed for placement into that course, then
that student should be more likely to be successful in that course than someone who
scored at the cut score. This phenomenon should not be dependent on the type of
examination used for placement.
Intermediate Algebra (MTH1825)
Table 54 shows the crosstabulation of grades in Intermediate Algebra by
mathematics placement examination scores. The placement examination scores for
students who enrolled in Intermediate Algebra ranged from 1 to 11. As the placement
examination scores get higher, more students received high course grades than lower
course grades. This phenomenon would be expected. However, a student who scores 1
point on the placement examination would not be expected to receive a 4.0 in
Intermediate Algebra.
148

Student #1671 was the student who received the grade of 4.0. This student was
placed with the unproctored examination, had a high school GPA of 3.21, had a high
school math GPA of 3.15, enrolled in a pre-calculus course during her senior year of high
school and received a B in the pre-calculus course. Given her high school record, it is no
surprise that the student received a 4.0 in Intermediate Algebra. However, it is surprising
that the student would score so low on the placement examination that she would be
placed into Intermediate Algebra.
Student #295 was the student who scored 1 on the placement examination and
received a grade of 0.0 in Intermediate Algebra. This student scored 14 on the
mathematics section of the ACT, had a high school GPA of 2.8, had a high school
mathematics GPA of 2.37, did not take a mathematics course during his senior year of
high school, but took trigonometry in 11th grade and received a B. Although scoring 1
point on the placement examination is surprising for any student, it is not surprising that
the student received a 0.0 in Intermediate Algebra.
The crosstabulation shows that 117 students were not successful in MTH1825
resulting in a false positive rate of 34.9% (117/335). Recall that Model 1 (the model
used to predict success in MTH1825 using MPE as the only predictor) produced a false
positive rate that was slightly lower than the actual false positive rate (32.5%) while
Model 2 had a false positive rate of 20.6%. MTH1825 is the lowest level mathematics
course at MSU so if a student is predicted to be unsuccessful in MTH1825 it can be
recommended that they take a lower level course at a community college or these
students can be given access to academic resources within the university.

149

Table 54
Crosstabulation of Grades in Intermediate Algebra by Placement Examination Score

MPE
1
2
3
4
5
6
7
8
9
10
11
Total

Grade in Intermediate Algebra (MTH1825)
0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Total
1
0
0
0
0
0
0
1
2
4
0
0
0
0
0
1
0
5
9
1
1
4
1
1
0
0
17
7
2
0
3
8
6
0
2
28
14
2
9
8 10
4
0
0
47
9
6
7
7
8
4
5
4
50
9
7
6 15
9
7 10
4
67
7
2
7
6 11 11
9
4
57
3
2
2 17
9 12
8
4
57
0
0
0
2
0
1
1
0
4
0
0
0
0
0
0
0
1
1
63 22 32 63 56 46 34 20
335

Students who received a placement examination score of 10 or 11 can enroll in
College Algebra (MTH103). Table 52 shows that 4 of the students in Intermediate
Algebra received a score of 10 on the placement examination and 1 student received a
score of 11 on the placement examination. All 5 of these students were successful in
Intermediate Algebra.
Table 55 shows the characteristics of the five students who were eligible to enroll
in MTH103 but enrolled in MTH1825. Although student #549 scored 17 on the
mathematics section of the ACT there is nothing else in his record that would explain
why the student would not enroll in MTH103. Similarly, there is nothing in the other
students‟ background that would explain why the others would not enroll in MTH103

150

Table 55
Students Who Placed into College Algebra but Enrolled in Intermediate Algebra

ID
4
46
277
549
1489

MPE
10(P)
10(P)
11(P)
10(P)
11(U)

ACTM
22
22
27
17
24

HSGPA
3.96
3.90
3.85
3.47
3.00

HSMGPA
3.96
3.84
3.96
3.60
3.51

LAST
(Grade)
FST (A)
Calc (A-)
Calc (B)
Precalc (B+)
Math Studies (A-)

MTH1825
Grade
3.5
2.0
4.0
3.0
2.0

Pa
0.93
0.92
0.95
0.79
0.80

a

Probability of passing College Algebra

Table 56 is a crosstabulation of grades in MTH1825 by placement examination
score by type of examination. The distribution of MTH1825 course grades appears to be
exhibiting the pattern that is expected. That is, students with low placement examination
scores have low grades. However, as the placement examination scores increase, the
grades began to shift to the right and until most, if not all, of the students with high
placement examination scores have grades above 2.0. This pattern does not appear to be
happening with the unproctored data. In fact, the grades do not appear to be shifting to
the right as the placement examination scores become larger.

151

Table 56
Crosstabulation of Grades in Intermediate Algebra by Placement Examination Score by
Type of Examination

Grades in Intermediate Algebra (MTH1825)
MPE 0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Total
Proctored
1
1
0
0
0
0
0
0
0
1
2
2
0
0
0
0
0
0
0
2
3
6
1
0
1
0
0
0
0
8
4
4
1
0
0
2
4
0
0
11
5
10
1
6
4
5
1
0
0
27
6
5
1
4
4
1
0
1
1
17
7
5
6
5 10
5
4
1
0
36
8
4
0
3
2
4
4
5
1
23
9
2
0
0
7
5
2
0
2
18
10
0
0
0
1
0
1
1
0
3
11
0
0
0
0
0
0
0
1
1
Total 39 10 18 29 22 16
8
5
147
Unproctored 1
0
0
0
0
0
0
0
1
1
2
2
0
0
0
0
0
1
0
3
3
3
0
1
3
1
1
0
0
9
4
3
1
0
3
6
2
0
2
17
5
4
1
3
4
5
3
0
0
20
6
4
5
3
3
7
4
4
3
33
7
4
1
1
5
4
3
9
4
31
8
3
2
4
4
7
7
4
3
34
9
1
2
2 10
4 10
8
2
39
10
0
0
0
1
0
0
0
0
1
Total 24 12 14 33 34 30 26 15
188

College Algebra (MTH103)
Table 57 shows the crosstabulation of grades in College Algebra by placement
examination scores. The table shows the pattern of the shift in grades as the placement
examination scores become higher. There were 195 students who enrolled in MTH103

152

but were eligible to enroll in a course whose level was higher than MTH103 Eighteen of
these students were not successful in MTH103. Also, there were 5 students who
enrolled in college algebra but were eligible to enroll in Calculus 1. One of these students
was not successful. Overall, the placement examination produced a total of 80 false
positives for MTH103 resulting in a false positive rate of 16.8% (80/477). Model 5
above had a slightly lower false positive rate (15.6%). Again, the distribution of grades
appears to be exhibiting the expected pattern. There appears to be a shift to the right in
grades as the MPE scores become larger.

Table 57
Crosstabulation of Grades in College Algebra by Placement Examination Score
Grade in College Algebra (MTH103)
MPE
0.0 1.0 1.5 2.0 2.5 3.0 3.5
10 - 11 25 10 27 52 48 66 40
12 - 14
5
5
7 31 24 44 27
15 - 18
0
0
0
2
2
7 11
19 +
0
0
1
0
0
0
1
Total
30 15 35 85 74 117 79

4.0 Total
14
282
23
166
2
24
3
5
42
477

Table 58 also shows that of the 17 students who were eligible to enroll in
MTH116 but enrolled in MTH103, 15 of them were placed with the unproctored
examination. Of these 15 students

153



One had an ACT Mathematics score of 17, was enrolled in algebra during his
senior year of high school and received a D+, and had a high school mathematics
GPA of 2.443.



Five enrolled in trigonometry, pre-calculus, or calculus during their senior year of
high school.



Five had high school mathematics GPA‟s below 3.0.



One took FST (functions, statistics, and trigonometry) during his senior year of
high school and received a C+

The unproctored placement examination indicated that there were 15 students who
were prepared for MTH116 but they were not even successful in MTH103. Chances are
that these 15 students would not have been successful in MTH116.

154

Table 58
Crosstabulation of Grades in College Algebra by Placement Examination Score by Type
of Examination

MPE
Proctored
10 - 11
12 - 14
15 - 18
19 +
Total
Unproctored 10 - 11
12 - 14
15 - 18
19 +
Total

Grades in College Algebra (MTH103)
0 1 1.5 2 2.5 3 3.5 4 Total
14 3
5 19 21 30 16 2
110
0 1
1 11
5 14
9 4
45
0 0
0 0
0 1
2 0
3
0 0
0 0
0 0
1 0
1
14 4
6 30 26 45 28 6
159
11 7 22 33 27 36 24 12
172
5 4
6 20 19 30 18 19
121
0 0
0 2
2 6
9 2
21
0 0
1 0
0 0
0 3
4
16 11 29 55 48 72 51 36
318

Table 58 also appears to be exhibiting the expected pattern for both the proctored
and unproctored data. As the placement examination scores increase, the grades appear
to be shifting to the right. This pattern is what should be expected.
Table 58 shows that 49 of the 159 students (30.8%) who were placed with the
proctored examination enrolled in math 103, but were eligible to enroll in a higher level
course. Also, 146 of the 318 students (45.9%) who were placed with the unproctored
examination were eligible to enroll in a higher level course. The difference in the
percentages was significant (z = -3.16, p =.001) indicating that the percentage of students
who enrolled MTH103, but were eligible to enroll in a higher level course was higher
under the unproctored examination.

155

Table 59 shows the characteristics of the students who were eligible to enroll in
MTH132, but enrolled in MTH103 instead. Four of these five students were placed with
the unproctored examination. Student #677 scored 21 on the unproctored placement
examination and received a grade of 1.5 in college algebra. This student scored 22 on the
mathematics section of the ACT, had a high school GPA of 3.42, had a high school
mathematics GPA of 3.75, took pre-calculus during her senior year of high school and
received a grade of A-. With such a background, it is surprising that this student received
a 1.5 in college algebra. If this student received such a low grade in college algebra, one
can only conclude that the student would not have been successful in calculus.
Student #340 took AP Calculus in her senior year of high school and received a
D-. This student may not have felt comfortable with taking a college calculus course
given her poor performance in AP Calculus. The remaining 3 students had pre-college
characteristics that would suggest that they would be prepared for a college calculus
course, but these students enrolled in College Algebra.

Table 59
Students Who Were Eligible to Enroll in Calculus I, but Enrolled in College Algebra

ID
340
677
1151
1189

MPE
19(P)
21(U)
21(U)
19(U)

ACTM
26
22
27
26

1472 19(U) 25

HSGPA
3.80
3.42
4.48
4.00

HSMGPA
3.16
3.76
4.15
4.15

3.83

4.04

a

Probability of passing Calculus 1
156

LAST (Grade)
AP Calc(D-)
Pre-calculus(A-)
Trigonometry(A)
Pre-calculus(A)
Discrete
math(A)

MTH103
Grade
3.5
1.5
4.0
4.0

P
0.99
0.99
0.99
0.99

4.0

0.99

a

Finite Mathematics (MTH110)
Table 60 shows the distribution of grades in finite mathematics by scores on
mathematics placement examination. A minimum score of 10 is needed for placement
into finite mathematics. As the placement examination scores get higher fewer students
receives course grades below 2.0. There were 2 students who received were eligible to
enroll into calculus but enrolled in finite mathematics. One of the students received a
grade of 2.0 and the other received a grade of 3.5. It is this researcher‟s belief that a
student will enroll in finite mathematics instead of calculus because their choice of major
does not require a course in calculus. Overall, the placement examination produced 15
false positives for MATH110 for a false positive rate of 14% (15/104).

Table 60
Crosstabulation of Grades in Finite Mathematics by Placement
Grade in Finite Math (Math110)
MPE
0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Total
10 - 11
4
0
3
9
9 11
5
3
44
12 - 14
2
4
2
8 12 11
5 11
55
15 - 18
0
0
0
0
0
2
0
1
3
19+
0
0
0
1
0
0
1
0
2
Total
6
4
5 18 21 24 11 15
104

Table 61 shows the crosstabulation of grades in finite mathematics by placement
examination scores by type of examination. The table shows that 26 of the 49 students
(53.1%) who were placed with the proctored examination enrolled in math 110, but were
eligible to enroll in a higher level course. Also, 34 of the 55 students (63%) who were
placed with the unproctored examination were eligible to enroll in a higher level course.
157

The difference in the percentages was not significant (z = -0.9, p =.367) indicating that
there was no evidence that the percentage of students who enrolled in MTH110, but were
eligible to enroll in a higher level course, was difference across examination types

Table 61
Crosstabulation of Grades in Finite Mathematics by Placement Examination Scores by
Type of Placement Examination

MPE
Proctored
10 - 11
12 - 14
15 - 18
Total
Unproctored 10 - 11
12 - 14
15 - 18
19 +
Total

Grades in Finite Mathematics (MATH110)
0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Total
2
0
1
5
5
6
3
1
23
0
4
1
1
5
3
3
7
24
0
0
0
0
0
2
0
0
2
2
4
2
6 10 11
6
8
49
2
0
2
4
4
5
2
2
21
2
0
1
7
7
8
2
4
31
0
0
0
0
0
0
0
1
1
0
0
0
1
0
0
1
0
2
4
0
3 12 11 13
5
7
55

College Algebra and Trigonometry (MTH116)
Table 62 shows the distribution of grades in college algebra and trigonometry by
placement examination scores. The minimum placement exam score needed for
placement into this course is 12. As the placement examination scores get higher, more
students earn higher course grades. Of the 249 students who enrolled in MTH116, 42 of
them were not successful. The placement examination produced a false positive rate of

158

approximately 16.9% (42/249) form MTH116. Model 6 above produced a false positive
rate of 12.9%

Table 62
Crosstabulation of Grades in College Algebra and Trigonometry by Placement Scores
Grade in College Algebra and Trigonometry (MTH116)
MPE
0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Total
12 - 14 12
4 10 26 26 29 18 17
142
15 - 18
4
2
4
8 14 19 17 21
89
19 +
3
2
1
2
0
2
4
4
18
19
8 15 36 40 50 39 42
249

Table 63 shows the crosstabulation of grades in MTH116 by placement
examination score by type of examination. Of the students who enrolled in MTH116
and were placed with the proctored examination, approximately 34.7% (34/98) of them
were eligible to enroll in a higher level math course. Of the students who were enrolled
in MTH116 and were placed with the unproctored examination, approximately 48.3%
(73/151) of them were eligible to enroll in a higher level math course. The difference in
percentages was significant (z = -2.13, p =0.03).

159

Table 63
Crosstabulation of Grades in College Algebra and Trigonometry by Placement
Examination Scores by Type of Examination

MPE
Proctored
12 - 14
15 - 18
19 +
Total
Unproctored 12 - 14
15 - 18
19 +
Total

Grades in College Algebra and Trigonometry (MTH116)
0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Total
5
2
4
11
14
10
8
10
64
1
0
2
3
6
5
6
9
32
0
0
1
0
0
1
0
0
2
6
2
7
14
20
16
14
19
98
7
2
6
15
12
19
10
7
78
3
2
2
5
8
14
11
12
57
3
2
0
2
0
1
4
4
16
13
6
8
22
20
34
25
23
151

Table 64 shows the characteristics of the 5 students who were eligible for
Calculus 1, but were unsuccessful in MTH116. Although student #1272 scored a 22 on
the placement examination and received a 22 on the mathematics section of the ACT,
both her high school GPA and high school mathematics GPA are below 3.0. This student
also took a pre-calculus course during her senior year of high school and received a grade
of C. Student #1250 scored 19 on the placement examination, 18 on the mathematics
section of the ACT, and took a geometry course during his senior year of high school and
received a grade of C-. There appears to be something in all 5 students‟ background that
may explain their enrollment into MTH116 instead of Calculus 1.

160

Table 64
Characteristics of the Students Who Were Eligible for MTH132, but Enrolled in
MTH116 and Were Unsuccessful

ID
1250
1273
1606
1655

MPE
19
22
21
20

1668 20
a

missing value

ACTM
18
22
24

HSGPA
3.475
3.361
2.906

HSMGPA
3.288
3.669
2.375

-a

-a
-a

-a

26
b

LAST (Grade)
Geometry (C-)
Statistics (A)
Pre-calculus (C)
None
b
Pre-calculus

3.483

MTH116
Grade
1.0
1.0
0.0
0.0
0.0

no grade recorded

Survey of Calculus 1 (MTH124)
Table 65 shows the crosstabulation of grades by placement examination score for
the students who were enrolled in survey of Calculus I (MTH124). The placement
examination gave a total of 19 false positives for MTH124 resulting in a false positive
rate of 7.4%. Model 8 above produced a false positive of 6% for MTH124.
Table 65
Crosstabulation of Grades in Survey of Calculus by Placement Scores
Grade in Survey of Calculus 1 (MTH124)
MPE
0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Total
15 – 18
4
3
6 10 25 34 29 27
138
19 +
1
0
5
4 13 17 26 54
120
Total
5
3 11 14 38 51 55 81
258

Table 66 shows the distribution of grades by placement examination scores by
type of examination. Of the 67 students who enrolled in MTH124 under the proctored
161

examination, approximately 32.8% (22/67) of them could have enrolled in Calculus 1
(MTH132) Of the 191 students who enrolled in MTH124 under the unproctored
examination, approximately 51.3% of them could have enrolled in Calculus 1. There was
no significant difference in the percentage of students who were unsuccessful in MTH124
across examination types (z = -2.61, p =.009). Therefore, there was a difference in the
percentage of students who enrolled in MTH124 but could have enrolled in MTH132.
It should be noted, however, that MTH124 is a course that is taken mostly by
students who intend to major in business related courses whereas MTH132 is a course
that is taken mostly by students who intend to major in science, mathematics, or
engineering. Thus, the significant difference that is observed here could be the result of
the differences in intended major and not differences in examination type.

162

Table 66
Crosstabulation of Grades in MTH124 by Placement Scores by Type of Exam

Grades in Survey of Calculus 1 (MTH124)
MPE
0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Total
Proctored
15 - 18
3
1
1
4
6
9 12
9
45
19 +
1
0
1
1
3
3
4
9
22
Total
4
1
2
5
9 12 16 18
67
Unproctored 15 - 18
1
2
5
6 19 25 17 18
93
19 +
0
0
4
3 10 14 22 45
98
Total
1
2
9
9 29 39 39 63
191

Calculus 1 (MTH132)
Table 67 shows the crosstabulation of grades by placement examination scores for
the students who enrolled in Calculus 1. As the placement examination scores get higher,
the number of students who receive high course grades increase. The placement
examination produced a total of 28 false positives for MTH132 resulting in a false
positive rate of 14.7% (28/191). Model 10 above produced a false positive of 11.5% for
MTH132.

163

Table 67
Crosstabulation of Grades in Calculus 1 by Placement Examination Scores
Grade in Calculus 1 (MTH132)
MPE 0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 total
19
7
6
4
4 11 12
5
6
55
20
0
1
1
8
2
5
8
4
29
21
0
1
1
3
2
3
3
8
21
22
2
0
2
1
3
1
6
6
21
23
0
0
0
2
3
3
5
1
14
24
2
0
0
0
1
0
2
6
11
25
0
0
0
2
0
0
4
4
10
26
0
1
0
2
0
0
2
5
10
27
0
0
0
0
0
1
1
2
4
28
0
0
0
0
1
0
0
1
2
total
11
9
8 22 23 25 36 43 177

Table 68 shows the crosstabulation of grades in MTH132 by placement exam
score by exam type. A score of 19 is the minimum score needed to place into MTH132.
Overall, there were a total of 55 students who received a score of 19 on the placement
examination. Thirty percent (17/55) of these students were not successful in MTH132. Of
the 122 students who placed into MTH132 with a placement exam score of more than 19,
9% of them were unsuccessful in MTH132. This could suggest an increase in the
minimum score needed to place into calculus is needed.

164

Table 68
Crosstabulation of Grades in Calculus 1 by Placement Exam Score by Type of
Examination

Grades in Calculus 1 (MTH132)
MPE 0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 total
Proctored
19
3
2
0
1
5
4
2
1
18
20
0
0
0
2
1
3
1
2
9
21
0
0
0
1
0
0
1
3
5
22
0
0
0
0
1
0
0
0
1
24
1
0
0
0
0
0
0
2
3
25
0
0
0
0
0
0
1
0
1
26
0
0
0
1
0
0
0
2
3
27
0
0
0
0
0
0
1
0
1
28
0
0
0
0
1
0
0
1
2
Total
4
2
0
5
8
7
6 11
43
Unproctored 19
4
4
4
3
6
8
3
5
37
20
0
1
1
6
1
2
7
2
20
21
0
1
1
2
2
3
2
5
16
22
2
0
2
1
2
1
6
6
20
23
0
0
0
2
3
3
5
1
14
24
1
0
0
0
1
0
2
4
8
25
0
0
0
2
0
0
3
4
9
26
0
1
0
1
0
0
2
3
7
27
0
0
0
0
0
1
0
2
3
Total
7
7
8 17 15 18 30 32 134

Overall, 21% of the students enrolled in a course lower than the level indicated by
their placement examination score. In addition, there were 598 students placed with the
proctored examination. Of these approximately 19% (113/598) enrolled in a course at a
level lower indicated by their placement examination score. There were 1098 students in
this study who were placed with the unproctored examination. Of these students,
approximately 23% (253/1098) of them enrolled in a course lower than the level

165

indicated by their placement examination score. It is important to the validity of the
placement examination to understand the reasons in which students enroll in a course
lower than that in which they were placed.

166

CHAPTER 5
DISCUSSION
The primary purpose of this study was to compare the results of the proctored
mathematics placement examination to the results of the unproctored mathematics
placement examination. Of particular interest was whether the students tend to score
higher on the unproctored examination than on the proctored examination. Also of
interest was whether the type of examination used for placement produced an effect on
grades in students‟ first college mathematics course or on course performance.
This section begins with a discussion of the conclusions reached as a result of
this study. Then there is a discussion of the threats to the validity of this study. Finally,
there are some recommendations for future research.

Conclusion 1
The unproctored examination has more inappropriate placements than the
proctored examination.
This study concluded that students who were placed with the unproctored
examination received higher average placement examination scores than the students who
were placed with the proctored examination. As a result, students are placed into higher
level mathematics courses with the unproctored examination.
Moreover, of the 598 students who were placed with the proctored examination,
approximately 19% (113/598) enrolled in a lower course. Of the 1098 students who
were placed with the unproctored examination, approximately 23% (253/1098) of them
enrolled in a lower level course. Although a higher proportion of students are placing
167

into higher level math courses with the unproctored examination, a larger proportion of
those students are enrolling in courses that are lower than the level in which they are
placed.

Conclusion 2
The proctored examination and the unproctored examination are not functioning
similarly.
When the unproctored examination is used in the same manner as the proctored
examination, it is important to make sure that the two exams are comparable. In other
words, similar students should receive similar placements regardless of the exam used to
place them. In addition to course placement, the effect on the course outcomes (i.e.
grades) should be similar for similar students. Clearly, this is not happening.
First of all, students are not being placed similarly. The multiple linear regression
model that was constructed with the proctored data predicted that about 35% of the
students who were placed with the unproctored examination would have received similar
placement had they been placed with the proctored examination. Additionally, about
21.5% would have received higher placement and about 43.5% would have received
lower placement.
The multiple linear regression model that was constructed using the unproctored
data predicted that about 41% of the students who were placed with the proctored exam
would have received the similar placement if they would have been placed with the
unproctored examination. Additionally, about 39% would have received higher

168

placement and about 21% would have received lower placement. Either model indicates
that students are placed higher with the unproctored examination.
Second the test of equality of the correlations revealed that the correlation
matrices were not equal. Specifically, the correlation of ACT Mathematics score with
grades in first college mathematics course was not equal across examination types. Also,
the correlation between ACT Mathematics scores and placement examination scores was
marginally significant across examination types. Additionally, ACT Mathematics score
was the variable that was most highly correlated with the grades first college mathematics
course for the proctored data. However, for the unproctored data, high school
mathematics GPA was the variable that was most highly correlated with grades in first
college mathematics course.
Additionally, the unproctored placement examination has a higher reliability than
the proctored examination (rxx = 0.81 for the unproctored examination, rxx = 0.76 for the
proctored examination).

Conclusion 3
The mathematics placement examination alone is not sufficient for deciding
placement.
Currently, students are placed into their first college mathematics course as a
result of the scores on the placement examination. Binary logistic regression models
were constructed to predict the conditional probability that a student would be successful
in their first college mathematics course. When the placement examination score was
considered alone, it was found to be a significant predictor of the success in Intermediate

169

Algebra (MTH1825), College Algebra (MTH103) and Calculus 1 (MTH132). However,
when ACT Mathematics scores, high school GPA, type of exam used for placement,
whether a mathematics course was taken during senior year of high school, and the type
of high school mathematics course taken last was considered in addition to placement
examination score, the prediction of the success in each course was improved. The
classification tables showed an increase in the percentage of correct decisions as well as a
decrease in the false positive rate for each course when the additional variables are
considered. Therefore ACT Mathematics scores, high school GPA, type of exam used
for placement, whether a mathematics course was taken during senior year of high
school, and the type of high school mathematics course taken last should all be
considered when placing students into their first college mathematics course.

Limitations
This study has several limitations. First, the results of this study cannot be
generalized to all college students. The students in this study chose to take mathematics
during their first semester at MSU. However some students do not enroll in their first
mathematics course until their second semester or their second year at MSU.
Random group assignment was not possible. Students chose the type of
examination to take and, therefore, the groups used in this study were predetermined. The
groups however were found to be non-equivalent.
Some of the students at MSU participate in summer “bridge” programs prior to
their enrollment in mathematics. Some of these programs are designed to help improve
students reading, writing, and/or mathematics skills. These students may have had
170

exposure to the type of mathematics that they would be studying the following fall.
Students in this study may have participated in these programs.
Some of the courses in this study may have been special courses designed for
either exceptional students or for “at risk” students. These courses may provide more
(less) rigor to the students enrolled than what a regular course would provide. Thus, there
may be an effect on course grade due to class type. Students in these courses may have
been included in this study.
The students in this study were allowed 3 attempts at the placement examination
but only one attempt at the proctored examination. Students with multiple attempts may
have attempt the proctored examination as one of their attempts or they have not
attempted the proctored examination at all. The placement scores used in this study are
the highest scores received by students on any attempt of the placement examination.
Students who took the examination more than once may have matured between attempts.
The multiple attempts may have better prepared students for their first mathematics
course. Additionally, differences that were found could be due to the fact that the students
may have attempted the unproctored examination more than once.
Grades in first college mathematics course were used as a criterion variable in
this study. There are multiple sections of the same course. Sometimes the assessments
are uniform across these multiple sections while other times the assessments are not
uniform.

171

Future Research
This study has found that students who are placed with the unproctored
examination received higher placement exam scores than students who are placed with
the proctored examination; even after controlling for ability. To help explain this
phenomenon, an investigation of the items may be necessary. Differential item
functioning (DIF) occurs when “examinees from each group with equal knowledge
exhibit different probabilities of success on an item” (Schumacher, 2005, p. 1). Although
guessing is not encouraged it is this researcher‟s belief that students do guess. Therefore,
the three parameter logistic item response theory model could be fitted to both the
proctored and unproctored data and the item difficulty and item discrimination
parameters could be estimated. Item characteristic curves can be constructed for each
item and differences in the parameters can be tested by estimating the area between the
item characteristic curves (Clauser & Mazor, 1998). Items are then flagged for DIF if the
item characteristics curves for each group are different. Items flagged for DIF can be
scrutinized to determine if they require modification or removal from the examination.
Once a student enrolls in a course, there are other variables that could affect their
course performance. As a follow up to this study, consideration should be given to
classroom variables as well as student variables. Classroom variables include but are not
limited to instructor classification (full-time, part-time, graduate student) and class size.
Student variables include, but are not limited to, number of other courses in which the
students are enrolled, whether or not students receive tutoring, students‟ study skills,
students‟ motivation, and choice of major. Understanding how these variables influence

172

students‟ success in mathematics can help university advisors recommend appropriate
courses as well as appropriate academic services for the struggling student.
This study contained students who enrolled in a course at a level lower than the
level in which they were placed. For example, 30.8% of the students who were placed
with the proctored examination and enrolled in MTH103 were eligible to enroll in a
higher level course while 45.9% of the students who were placed with the unproctored
examination and enrolled in MTH103 were eligible to enroll in a higher level course. It
is however, unclear whether students began in the lower level course or “dropped back”
to the lower level course. In either case, it is important to the validity of the placement
examination, as well as the comparability of the proctored and unproctored examination,
to further investigate why students are in course that are lower than the level in which
they were placed.

Summary
This study was motivated by the interest in determining if the MSU unproctored
mathematics placement examination can and should be used to place students into their
first college mathematics course. The short answer is no. Placement into remedial
mathematics was shown to be dependent on the type of test used for placement with the
unproctored examination placing students into remedial mathematics less often than the
proctored examination. Scores on the unproctored examination are significantly higher,
on average, than the scores on the proctored examination and students are placing at a
higher level with the unproctored examination.
Although students are being placed at a higher level with the unproctored
examination, there is a greater proportion of students enrolling into courses lower than
173

the level at which they were placed. It is, however, unclear as to whether students are
“dropping back” to the lower level courses or beginning in the lower level courses.
Whatever the case may be, users of the mathematics placement examination must
consider the unproctored examination differently when using it to place students into
their first college mathematics course.
This study also shows that pre-college variables (e.g. ACT Mathematics scores,
high school grade point average) can be used to help increase the prediction of success in
students‟ first college mathematics course. The models developed in this study increased
the “hit rate” beyond that predicted by placement examination score alone and decrease
the false positive rate. While there are no courses at a level lower than MTH1825, these
pre-college characteristics can be used to help identify students who may benefit from
some of the academic support services offered by Michigan State University.
Mathematics has influenced college graduation rates, choice of college major, and
even earning potential. It can be the difference between receiving a bachelor‟s degree in
4 years or in 7 years (or not at all). It can be the difference between completing a
bachelor‟s degree in a mathematics based field or completing a bachelor‟s degree in a
non-mathematics based field. Whatever the case may be, mathematics is not going away
and it is everyone‟s responsibility to help students overcome their difficulties with
mathematics. One way colleges and universities can help is by examining their
mathematics placement procedures so that it is very likely that a first time college student
begins in the appropriate mathematics course.

174

Appendix

175

Appendix
Description of Mathematics Courses

3
4





MTH110 (Finite Mathematics and Elements of College Algebra)
Description: Functions and graphs. Equations and inequalities. Systems of
equations. Matrices. Linear programming. Simplex algorithm. Probability and
statistics.



Math 112 (Finite Mathematics: Applications of College Algebra)
Description: Combinatorics, probability and statistics, mathematics of finance,
geometry, transition matrices, and linear programming. The course emphasizes
applications and includes work using spreadsheets.



Math 116 (College Algebra and Trigonometry)
Description: Functions and graphs. Equations and inequalities. Exponential and
logarithmic functions. Trigonometric functions. Systems of equations. Binomial
theorem.



MTH124 (Survey of Calculus I)
Description: Study of limits, continuous functions, derivatives, integrals and their
applications.



MTH132 (Calculus I)
Description: Limits, continuous functions, derivatives and their applications.
Integrals and the fundamental theorem of calculus.



STT200 (Statistical Methods)
Description: Data analysis, probability models, random variables, estimation, tests
of hypotheses, confidence intervals, and simple linear regression.



4

MTH103 (College Algebra)
Description: Number systems; functions and relations; exponents and logarithms;
elementary theory of equations; inequalities; and systems of equations.



3

MTH1825 (Intermediate Algebra)
Description: Properties of real numbers. Factoring. Roots and radicals. First and
second degree equations. Linear inequalities. Polynomials. Systems of equations.

STT201 (Statistical Methods)
Description: Probability and statistics with computer applications. Data analysis,
probability models, random variables, tests of hypotheses, confidence intervals,
simple linear regression. Weekly lab using statistical software.

http://www.reg.msu.edu/Courses/Request.asp
This course does not count toward a student‟s degree
176

Bibliography

177

Bibliography
Abraham, Ashley. (1992). College remedial studies: Institutional practices in the SREB
states. Southern Regional Education Board, Atlanta, GA. (ERIC Document
Reproduction Services No. ED 345 617).
ACT. (2008). College retention and graduation rates. Retrieved April 27, 2009 from
http://www.act.org/research/policymakers/reports/graduation.html
ACT. (2002). College Graduation Rates Steady Despite Increase In Enrollment.
Retrieved April 27, 2010 from http://www.act.org/news/releases/2002/11-1502.html
Akst, G & Hirsch, L. (1991). Selected studies on math placement. National Center for
Developmental Education, Reich College of Education, Appalachian State
University, Boone, NC (ERIC Document Reproduction Service No ED 354m
964).
Allen, M.J., & Yen, W. M. (2002). Introduction to Measurement Theory. Long Grove,
IL: Waveland Press.
American Council on Education. (1996). Remedial Education: An undergraduate student
profile. Washington, D.C: Author
American Educational Research Association, American Psychological Association,
National Council on Measurement in Education. (1999). Standards for
educational and psychological testing. Washington DC: American Psychological
Association.
Ang, C. H., & Noble, J. (1993). Incremental validity of ACT assessment scores and high
school course information for freshmen course placement. (Report No. ACT- RR93-5). ACT, Iowa City, IA.
Arifin, M. Z. (2009, May 24). Sample Logic Probit: Session 6: Logistic Regression,
Retrieved from http://www.docstoc.com/docs/docs/6382581/Sample-logit-probit
Armstrong, W, B. (2001). Explaining student course outcomes by analyzing placement
test scores, student background data, and instructor effects. (ERIC Document
Reproduction Service No. ED 454 907).
Bridgeman, B., & Lewis, C. (1995) Gender Differences in College Mathematics Grades
and SAT-M Scores: A Reanalysis of Wainer and Steinberg.( Report No. ETS-RR95-17) Educational Testing Service, Princeton, New Jersey (ERIC Document
Reproduction Services No. ED397074).

178

Bridgeman, B., & Wendler, C. (1991). Gender differences in predictors of college
mathematics performance and in college mathematics course grades. Journal of
Educational Psychology. 83(2), 275-84.
Bridgeman, B., & Wendler, C. (1989) Prediction of grades in college mathematics
courses as a component of the placement validity if SAT mathematics scores.
(Report No. RR 89-9) College Entrance Examination Board, New York, NY.
(ERIC Document Reproduction Services No. ED 395-019).
Beaty, J. C, Fallon, J.B, & Shepard W, J. (2002, April). Unproctored Web-Based
Administration of a Cognitive Ability Test. Paper Presented at the Annual
Conference of the Society of Industrial/Organizational Psychology, Toronto,
Canada.
Chang, J. C. (2002). "Women and Minorities in the Science, Mathematics and
Engineering Pipeline. ERIC Digest. Retrieved from
http://www.ericdigests.org/2003-2/women.html
Center for Institutional Data Exchange and Analysis. (2000). 1999-2000 SMET retention
report. Norman, OK: University of Oklahoma.
Charlesworth, P., Charlesworth, D. & Vician, C. (2006). Students‟ perspectives of the
influence of Web-enhance coursework on incidences of cheating. Journal of
Chemical Education 83(9), 1368 – 1385.
Cohen, Jacob. (1988). Statistical power analysis for the behavioral sciences. New York,
NY: Academic Press, Inc.
Cotter, J. W. (2007). Comparison of three methods of placement and advisement into
freshmen mathematics courses and the effects on eventual degree completion.
Unpublished doctoral dissertation, Georgia State University, Atlanta, Georgia.
Davies, S. A, & Wadlington, P, L. (2006, May). Factor & Parameter Invariance of a
Five Factor Personality Test across Proctored/Unproctored Computerized
Administration. Paper presented at the Annual Conference of the Society of
Industrial and Organizational Psychology, Dallas, Texas.
Davies, S. A, Norris, D, Turner, J, & Waldington P. (2005, April). Cheating, Guessing,
Faking, and Self Presentation in Assessment Responses. Paper presented at the
20th Annual Conference of the Society for Industrial and Organizational
Psychology, Los Angeles, California.
Ebel, R. L. (1986). Essentials of educational measurement (4th Ed.). Englewood Cliffs,
NJ; Princeton-Hall.

179

Frankfort-Nachmia, C., & Leon-Guerrero, A. (2009). Social statistics for a diverse
society (5th ed.). Thousand Oaks, CA: Pine Forge Press.
Flores, R. (2007). Psychometric analysis of a mathematics placement examination.
Unpublished master‟s thesis, Texas Tech University, Lubbock, Texas.
Gabe, L. C. (1989). Relating college – level performance to ASSET placement scores
(Report No. RR 89-22). Fort Lauderdale, FL. Broward Community College.
(ERIC Document Reproduction Service No ED 309 823).
Garson, G. David. (2010). Logistic Regression, from Statnotes: Topics in Multivariate
Analysis. Retrieved from http://faculty.chass.ncsu.edu/garson/PA765/logistic.htm
Gupta, S., Harris, D. E., Carrier, N. M., & Caron, P. (2006). Predictors of Student
Success in Entry-Level Undergraduate Mathematics Courses. College Student
Journal. 40(1), 97-108.
Harmon, O. R., Lambrinos, J., & Kennedy, P., Ed. (2008). Are Online Exams an
Invitation to Cheat? Journal of Economic Education. 39(2), 116-125.
Hill, R. O. (2006). On the transition in mathematics from high school to Michigan State
University, Retrieved from http://www.math.msu.edu/~hill/Transition.pdf
Hudson, J. B. (1989). An analysis of ACT scores, placement tests and academic
performance in reading, English, and mathematics courses. Louisville University.
Ky. Preparatory Division. (ERIC Document Reproduction Service No ED 334
916).
Hunsley, J., & Meyer, G. (2003). The incremental validity of psychological testing and
assessment :conceptual, methodological, and statistical issues. Psychological
Development, 15(4), 446-455
Jacobson, E. (2006). Higher placement standard increase course success but reduce
program completions. The Journal of General Education, 55(2), 139-159.
Kennedy , K., Nowak, S.; Raghuraman, R., Thomas, J.,& Davis, S. F. (2000). Academic
dishonesty and distance learning: Student and faculty views. College Student
Journal 34(2), 309 -314.
Matthews-Lopez, J. L. (1998). Mathematics Placement Testing: Is it really necessary?
(ERIC Document Reproduction Services No. ED 457 165).
McCloy, R. (2008). Unproctored Internet Testing [PowerPoint slides]. Retrieved from
http://www.mppaw.org/slidesarchive.php

180

McDonald, A. D. (1989). Issues in Assessment and Placement for Mathematics. Journal
of Developmental Education. 13(2), 20-23.
Messick, S. (1989). Validity. In R. L. Linn (Ed.). Educational Measurement (pp.13 –
103). New York: Macmillan.
Morante, E. A. (1989). Selecting tests and placing students. Journal of Developmental
Education. 13(2), 2-4,6.
National Center for Education Statistics. (2010). The Condition of Education 2010
(NCES 2010-028), Indicator 21. Retrieved from
http://nces.ed.gov/FastFacts/display.asp?id=40.
Nelson, C. V., & Leganza, K. K. (2006). Is gender a predictor of success in college
mathematics courses? College and University. 81(4), 11-18.
Neill, J. (2008). Why use effect size instead of significance testing in program evaluation?
Retrieved from http://wilderdom.com/research/effectsizes.html
Nolting, P. (2004). Math study skills workbook (3rd ed.). Chicago, Illinois: Houghton
Mifflin.
Osborne, Jason & Elaine Waters (2002). Four assumptions of multiple regression that
researchers should always test. Practical Assessment, Research & Evaluation,
8(2). Retrieved from http://PAREonline.net/getvn.asp?v=8&n=2
Ostrum, C. W. (1990). Time series analysis: Regression techniques. Quantitative
applications in the social sciences (2nd ed., Vols. 07-009). Newbury Park: Sage
Publications.
Parker, M. (2005). Placement, retention, and success: A longitudinal study of
Mathematics and retention. Journal of General Education 54(1), 22-40.
Peck, J. D. (1997). Statistics: The Exploration and Analysis of Data (3 ed.). New York,
NY Duxbury Press.
Peng, C.Y. J., Lee, K. L., & Ingersoll, G. M. (2002). An introduction to logistic
regression analysis and reporting. Journal of Educational Research. 96(1), 3-14.
Pugh, C., & Lowther, S. (2004). College math performance and last high school math
course. Paper presented at the annual conference of the Southern Association for
Institutional Research, Biloxi, Mississippi.
Rodgers, K. V., & Wilding, W. G. (1998). Studying the Placement of Students in the
Entry-Level College Mathematics Courses. Primus. 8(3), 203-08.

181

Rovai, A. P. (2001). Online and traditional assessments: What is the difference? Internet
and Higher Education 3(3), 141-151.
Rubin, L. & Nora, A. (2002, April). Factors impacting the Academic Status of
Undergraduate Students at Four-Year Institutions Paper presented at the annual
meeting of the American Educational Research Association, New Orleans, LA.
Sawyer, R., & Schiel, J. (2000). Posttesting Students to Assess the Effectiveness of
Remedial Instruction in College. ACT Research Report. Retrieved from
http://www.act.org/research/reports/pdf/ACT_RR2000-7.pdf
Sawyer, R. (1996). Decision theory models for validating course placement tests. Journal
of Educational Measurement. 33(3), 271-90.
Schmitz, C. C., & DelMas, R. C. (1990). Determining the validity of placement exams
for developmental college curricula. Applied Measurement in Education. 4(1), 3752.
Schumacher, P. A. & Smith, R. M. (2008). A comparison of placement in first – year
university mathematics course using paper and pencil and online administration of
a placement test. International Electronic Journal of Mathematics Education.
4(3), 193 – 202.
Schumacker, R. (2005). Test bias and differential item functioning. Retrieved from
http://www.appliedmeasurementassociates.com/White%20Papers/TEST%20BIA
S%20AND%20DIFFERENTIAL%20ITEM%20FUNCTIONING.pdf
Seidman, A. (2005) Where do we go from here: A retention formula for student success.
In A. Seidman (Ed.), College student retention (pp. 295 – 316). Westport: Praeger
Publishers.
Shaffer, J. P. (1995). Multiple hypothesis testing. Annual Review of Psychology. 46, 591
– 584.
Shepard, W. J., Do, B. R., Drasgow F. L. (2002, April). Assessing Equivalence of Online
Non-Cognitive Measures. Paper Presented at the Annual Conference of the
Society of Industrial/Organizational Psychology, Orlando, FL
Shevel, L. R., & Whitney, D. R. (1969). Predictive validity of the mathematics placement
examination. Educational and Psychological Measurement. 29(4), 895-901.
Slark, J., et al. (1991). RSC validation of mathematics placement tests. Rancho Santiago
Community College. Santa Ana, Ca. (ERIC Document Reproduction Service No
ED 341 418).

182

Sticha, P. J., & Barber, G. (2003). CAT-ASVAB prototype internet delivery system: final
report (Report #FS-03-46). Retrieved from
http://www.psych.umn.edu/psylabs/catcentral/pdf%20files/st03-01.pdf
Thorndike, R. M., & Dinnel, D. L. (2001). Basic Statistics for the Behavioral Sciences.
Upper Saddle River, NJ: Prentice-Hall.
Tippen, N et al. (2006). Unproctored internet testing in employment settings. Personnel
Psychology, 59(1), 189-225.
Triola, M. F. (2010). Elementary statitics (11th ed.). Boston, MA: Addison Wesley
Wattenbarger, J. L., & McLeod, N. (1989). Placement in the mathematics curriculum:
what are the keys? Community College Review. 16(4), 17-21.
Wheland, E., Konet, R.,& Butler, K. (2003). Perceived inhibitors to mathematics success.
Journal of Developmental Education, 26(3) 19-27.

183