3.-
IV! v-iz‘luha u...
3:. .‘hﬂﬂﬁni 3...

, . 3.1.1... 3 i...
3E3. ﬁnénﬁﬁ 5.392....
. . 1.13.6.
s. .... 3:45..

11L...

3..

4.. 25.1; ‘ 4
I:
‘ a

I) .1!

2...
.vl\nl....illa
r251... .z.
.. .i
.19....91151.
36.11.34... :1
‘42“?‘% VI IAIS
1... . .i z
2!. iii!
2...;

31:5

V... .2) .
, .t!.$.

.3 :9. 11‘1\11‘

1:... l.
VIII-Lu.
5:50.. .i
.3... I. 13.3
.. . 1.28.33:
22“.
£257.43. 9!.va t.
is. ‘ .2 9.3.
.ilL. A}: auét
ail-321,310. o..-‘...J):|:‘.|. s...
{I 03.9.5.1‘2 ..
.9. iiiis
19!...
1 31.13.
1.! ..
V1.4 ‘vawlavioi
i... ll

3;]?! a
.1

r

35“.
i
I
na!.s§§.
. 115,2... .
all.
9.! .
.9141.

In.“
x."
5’31
I§§a=¥tl
\f
.3.“- .
.

 

 

51:3...‘3:
sid
sun".
:3... (I
. 1.: ‘VniI-ﬂr)...‘ so
“”952"? .Jﬂgo \.
Pi 3 it?
. I.
a 2.353%}.
255.9,...)
.13.: 3,
is;

kahuna Inn
#3. .. é...
. ..§..\ss.u.. 3.

Juniﬂ‘xf.

5 1 . a

. , 1.3.2“.

1.1..

. it
111.}.
3..)53-3
i
3‘
if?!
.1 313.1%
1......)

1:1 .1: x:

to
3:,

“In?”
It

tvvtih.
Purina“.

4h!

swa

II}...
..
. z
4- V‘.
LIQ: $951.};
ll. cu“ ii!!!
5.
“(any a
.24...
.. av; 2. Z

3
3.».

‘9 ‘ It .

.2321}... I
.21. .98.

l:

.

.l
I
‘~

.1 [5‘15'151

(5; iii-)3}. . \\

$73.):

a
'13..

. If .IL
54%,.
x

S.

This is to certify that the
dissertation entitled

MEASUREMENT OF STUDENT A'lTlTUDES IN FIRST YEAR
ENGINEERING — A MIXED METHODS APPROACH

presented by

Qaiser Hameed Malik

has been accepted towards fulfillment
of the requirements for the

degree In Electrical Engineering

 

 

me

Major Professor’ 5 Signature

#09. A?! 2.07/0
/ ,

 

Date

MSU is an Afﬁnnative Action/Equal Opportunity Employer

 

 

LlBRARY
Michigan State
University

 

 

 

..-.-.—-u-c-o-a-I-I--I-v-c-I-I---n-o--u-o-v-c-u-IOI-n-I-v-I-c-I---o--o-v-o----n-o-s-v-v-o--I-n--v-u---1-n---v.

PLACE IN RETURN BOX to remove this checkout from your record.
To AVOID FINES return on or before date due.
MAY BE RECALLED with earlier due date if requested.

 

DATE DUE DATE DUE ' DATE DUE

 

 

 

 

 

 

 

 

 

 

 

 

 

 

5/08 KlProj/AccauPrelelRC/Dateoue.indd

 

MEASUREMENT OF STUDENT ATTITUDES IN FIRST YEAR
ENGINEERING — A MIXED METHODS APPROACH

By

Qaiser Hameed Malik

A DISSERTATION

Submitted to
Michigan State University
In partial fulﬁllment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY

ELECTRICAL ENGINEERING

2010

ABSTRACT

MEASUREMENT OF STUDENT ATTITUDES IN FIRST YEAR
ENGINEERING — A MIXED METHODS APPROACH

BY

Qaiser Hameed Malik

This research study focused on freshman attitudes towards engineering in a newly
implemented cornerstone sequence that emphasized holistic design experiences. The
students’ initial attitudes and changes in these attitudes were examined with the
explanatory mixed methods approach that allows a sequential examination of the target
population with two methods, using two sets of data, to investigate the treatment effects.

In the quantitative phase, the study compared changes in freshman attitude
towards engineering, between the new ‘design sequence’ group (composed of ﬁeshmen
in the cornerstone sequence) and the prior ‘traditional sequence’ group (composed of all
other freshmen), over the course of one semester. The data were collected in fall 2008 at
two time intervals and changes in the two groups’ attitudes were examined with repeated
measures analysis of covariance models. The analyses reported here include data from
389 students out of the total population of 722 freshmen. The analyses revealed that
engineering freshmen joined the program with positive or strongly positive attitudes
towards engineering. Those strong attitudes were durable and resistant to change.
Students in the design sequence group had higher ACT scores, enjoyed math and science
the most, and did not believe engineering to be an exact science. However, no

appreciable time- group interaction was observed.

To validate the quantitative results, an interview protocol was developed to
investigate initial freshman attitudes and changes, if any, that took place as a result of the
new cornerstone sequence. One-on-one interviews with a sample of ten students out of
the population of 272 freshmen revealed that freshmen in the cornerstone sequence
entered the program full of enthusiasm and idealism, and with strongly positive attitudes
towards engineering. The strong motivational factors included parental/teacher
inﬂuences, childhood motivations, and high school extra-curricular experiences. The
participants appreciated the team work and problem solving aspects of engineering;
however, they reported negative experiences in the cornerstone sequence. Interestingly,
their overall perception about engineering was not affected by any of the negative
experiences. The qualitative phase substantiated the belief that strong attitudes are harder
to change; they are durable, they have impact, and they are not signiﬁcantly affected by a
short treatment.

The results of this mixed methods study indicate that changing student attitudes
may not be an easy task. One must develop a better understanding of student attitudes in
order to improve understanding of the ﬁne-grained details of curriculum and its
implementation to be able to develop more effective cornerstone design courses. Clearly,
tight and focused quantitative studies complemented with a qualitative component
provide a much broader and deeper insight into the learning that takes place in freshman
courses. This research also documents the use of a longitudinal study to track the design
sequence group and observe their performance in their junior and senior years. This

would provide a better understanding of the long term effects of the new sequence.

Dedicated to the memory of my daughter, Farina.

iv

ACKNOWLEDGEMENTS

First, I would like to thank the Almighty for granting me the honor of
accomplishing this life-long goal.

I would like to speciﬁcally express my deep gratitude and appreciation to my
advisor, Prof. Michael Shanblatt, for being an outstanding mentor and a great friend for
the past sixteen years of our association that began with my Masters degree at MSU. He
has been critical to my professional development as a researcher and as a teacher because
of his guidance, personal motivation, and unﬂagging support during the last four years

Many thanks are due to Dr. Punya Mishra who through his generosity of time and
spirit as well as personal and professional guidance has been the linchpin of my research
at MSU. His contagious enthusiasm, innovativeness, and artistic qualities continue to
amaze me. I would like to thank Dr. Neeraj Buch, the PI of the project that served as the
backbone for my research. He dedicated extra hours for many fruitful and thought
provoking discussions on the statistical results, and provided most of the ﬁnancial
support throughout the course of this research. My thanks are also due to Dr. Matthew
Koehler for his guidance that gave me the essential understanding of the project methods -
and the conﬁdence in my abilities to use them. I would also like to convey my thanks to
Dr. Jon Sticklen for taking extra ordinary interest in my research and especially for
providing the intellectual, ﬁnancial, and moral support towards the completion of this
research. My thanks are also due to Dr. Gregory Weirzba for his guidance and
understanding throughout the course of this research.

My special thanks are due to Steven Pierce for devoting long hours to helping me

develop a better understanding of statistical modeling and diagnostic testing in SPSS©. I

am also thankful to Justin Micomonaco for sharing his expertise in qualitative research.
Additionally, I would like to express my gratitude and appreciation to Dr. Patrick
Dickson for motivating me to take this path in research and for connecting me to the ri ght
people and places at MSU. My thanks are also due to my colleagues and countrymen,
Syed Sajjad Zaidi and Abdul Rehman Tariq, for their comradeship, wittiness, and moral
support that made me feel at home during my stay at MSU.

I would also like to convey my appreciation to the two excellent resource centers
at MSU: the Center for Statistical Training and Consulting (CSTAT) for providing
support in the form of regular workshops and personal consultations with statistics
experts; and the Writing Center for their guidance in improving manuscripts of several
publications including this dissertation.

On a personal note, I would like to express my most sincere gratitude to my
family: my mother who has been behind all my successes in life; my wife Meena for her
sacriﬁces, encouragement and motivation to keep me focused on my goals; her sister
Rubeena for maintaining a close link with daily long distance calls; my son Arslan for
making me the proud father of the best college student and for graduating with a perfect
GPA along with me; and my daughter Aleena for being patient with me. My thanks are
also due to my siblings in Toronto, Zubair Malik and Anjum Tahir, for taking good care
of mother during the course of this research.

The ﬁnancial support from the Michigan State University Graduate School,
College of Engineering, and Department of Electrical and Computer Engineering, as well
as the National University of Sciences and Technology (N UST), Pakistan is also

gratefully acknowledged.

vi

TABLE OF CONTENTS

LIST OF TABLES ................................................................................... ix

LIST OF FIGURES ................................................................................... x

CHAPTER 1

INTRODUCTION ............................................................................................................... 1
1.1 Current Trends and Challenges ................................................................. l
1.2 Institutional Efforts .................................................................................... 4
1.3 Freshman Engineering Experiences at Michigan State University ........... 6
1.4 Context for the Study ............................................................................... 11

CHAPTER 2

DESIGN OF THE STUDY ............................................................................................... 13
2.1 The Mixed Methods Design .................................................................... 13
2.2 The Quantitative Phase ............................................................................ 13
2.3 The Qualitative Phase .............................................................................. 15
2.4 Organization ............................................................................................ 16

CHAPTER 3

THE QUANTITATIVE PHASE ....................................................................................... 18
Part I - Methods .......................................................................................................... 18
3.1 The General Model .................................................................................. 18
3.2 Instrumentation ........................................................................................ 20
3.3 Data Collection ........................................................................................ 30
3.4 Selection of Variables .............................................................................. 33
3.5 The Speciﬁc Model .................................................................................. 39
3.6 Test Assumptions ..................................................................................... 40
3.7 Data Transformation ................................................................................ 42
Part II -—- Analysis/Discussion ..................................................................................... 44
3.8 Career (Prel -Postl) ................................................................................. 47
3.9 Society (Pre3-Post3) ................................................................................ 49
3.10 Perception (Pre4-Post4) ........................................................................... 50
3.11 Math (Pre5-Post5) .................................................................................... 53
3.12 Exact (Pre6-Post6) ................................................................................... 55
3.13 Basic (Pre8-Post8) ................................................................................... 57
3.14 Communication (Pre9-Post9) .................................................................. 60
3.15 Group (Prel l-Postl 1) .............................................................................. 62
3.16 Ability (PrelZ-Post12) ............................................................................. 63
3.17 Compatibility (Pre13-Post13) .................................................................. 65
3.18 Summary of Analysis .............................................................................. 66
3.19 Discussion ................................................................................................ 69

vii

CHAPTER 4

THE QUALITATIVE PHASE .......................................................................................... 72
Part I - Methods .......................................................................................................... 73
4.1 The Context ............................................................................................. 73
4.2 Cornerstone Sequence - A Revisit ........................................................... 74
4.3 Development of Interview Protocol ......................................................... 76
4.4 The Sample .............................................................................................. 79
4.5 Data Collection, Reduction, and Display ................................................ 81
Part II - Analysis/Discussion ...................................................................................... 85
4.6 Data Analysis ........................................................................................... 85
4.7 Discussion .............................................................................................. 105

CHAPTER 5

CONCLUSION ............................................................................................................... 108

APPENDIX A

PITTSBURGH FRESHMAN ENGINEERING ATTITUDES SURVEY (PF EAS) ...... 114

APPENDIX B I

PFEAS FACTOR LOADINGS ....................................................................................... 118

APPENDIX C

ONE-ON-ONE INTERVIEW PROTOCOL ................................................................... 120
Part I — Student Experiences in the Freshman Year ................................................. 120
Part II — Freshman Experiences in the Cornerstone sequence ................................. 122

REFERENCES ................................................................................................................ 124

viii

Table 1.1:

Table 3.1:

Table 3.2:

Table 3.3:

Table 3.4:

Table 3.5:

Table 3.6:

Table 3.7:

Table 3.8:

Table 3.9:

LIST OF TABLES

A presentation of freshman engineering initiatives taken by some of

the CIC institutions to improve retention ......................................................... 6
Deﬁning the PFEAS subscales ....................................................................... 24
Mapping of the PFEAS subscales. ................................................................. 25
Itemized detail of the ten selected PFEAS subscales. .................................... 27
Pre- and post- data collection during 3rd and 14th week of fall

2008, respectively. ......................................................................................... 32
Gender and ethnicity distribution of the sample. ........................................... 33
Deﬁning the three categories of other IVs. .................................................... 35

Distribution of math competency between DS and TS groups
showing low cell count (11 < 10). ................................................................... 37

Independent samples t-test for ACT composite score in the
two groups ...................................................................................................... 38

NS with low variability. ................................................................................ 38

Table 3.10: Box’s test for homoscedasticity for raw and rank-transformed

data. The transformation corrected homoscedasticity problems
apparent in the raw data for the math, basic, groups, ability and
compatibility subscales. ................................................................................. 43

Table 3.11: Repeated measures AN COVA test results for each of the ten

PFEAS subscales based on analyzing both raw and

rank-transformed data. ................................................................................... 45
Table 3.12: Mean scores for ten PFEAS subscales from the ANCOVA analyses,

broken down to illustrate the group *time and gender effects ........................ 47
Table 4.2: Characteristics of the sample vs. the population. ........................................... 81

Table 4.3: Qualitative data display showing presence or absence of themes

for each participant. ....................................................................................... 83

ix

LIST OF FIGURES

Figure 1.1: Engineering students’ retention trends at MSU from 1997 to 2007. .............. 7
Figure 2.1: Explanatory mixed methods design. ............................................................. 13
Figure 3.1: The ACT composite score distribution in the sample ................................... 38
Figure 3.2: Bubble plot for Pre5 math showing three inﬂuential data points. ................ 42

Figure 3.3: Bubble plot for Rank Pre5 math showing no effects of inﬂuential
data. .............................................................................................................. 43

Figure 3.4: Proﬁle plot of the time *group interaction on the career subscale.
The longitudinal trajectories from pre-test to post-test for the two
groups were nearly identical, which was why the interaction effect
was not signiﬁcant. ....................................................................................... 49

Figure 3.5: Proﬁle plot of the time *group interaction on the society subscale.
The longitudinal trajectories from pre-test to post-test for the two
groups were nearly identical, which was why the interaction effect
was not signiﬁcant. ....................................................................................... 50

Figure 3.6: Proﬁle plot of the time *group interaction on the perception subscale.
The longitudinal trajectories from pre-test to post-test for the two
groups differ signiﬁcantly, but by a trivial amount that was due to
the presence of few outliers. ......................................................................... 52

Figure 3.7: Proﬁle plot of the time *group interaction for rank data on the
perception subscale. The longitudinal trajectories ﬁ'om pre-test to
post-test for the two groups were nearly identical, which was why
the interaction effect was not signiﬁcant. ..................................................... 53

Figure 3.8: Proﬁle plot of the time *group interaction on the math subscale.
The longitudinal trajectories from pre-test to post-test for the
two groups were nearly identical, which was why the interaction
effect was not signiﬁcant. ............................................................................. 54

Figure 3.9: Proﬁle points for group marginal means on the math subscale.
The DS group enjoyed math more than the TS group compared
to liberal art courses. .................................................................................... 55

Figure 3.10: Proﬁle plot of the time *group interaction on the exact subscale.
The longitudinal trajectories from pre-test to post-test for the

Figure 3.11:

Figure 3.12:

Figure 3.13:

Figure 3.14:

Figure 3.15:

Figure 3.16:

Figure 3.17:

Figure 3.18:

Figure 3.19:

Figure 3.20:

two groups were nearly identical, which was why the interaction

effect was not signiﬁcant. ..................................................................

Proﬁle points for group marginal means on the exact subscale.
The TS group perceived engineering as an exact science more

than the DS group. .............................................................................

Proﬁle points for gender marginal means on the exact subscale.

Females perceived engineering as an exact science less than males.

Proﬁle plot of the time *group interaction on the basic subscale.
The longitudinal trajectories from pre-test to post-test for the two
groups differ signiﬁcantly, but by a trivial amount that was due to

the presence of few outliers. ..............................................................

Proﬁle plot of the time *group interaction for rank data on the
basic subscale. The longitudinal trajectories ﬁ'om pre-test to
post-test for the two groups were nearly identical, which was

why the interaction effect was not signiﬁcant. ..................................

Proﬁle points for gender marginal means on the basic subscale.
Males had higher conﬁdence level in basic engineering knowledge

and skills than females. .....................................................................

Proﬁle points for gender marginal means for rank data on the
basic subscale. Males had higher conﬁdence level in basic

engineering knowledge and skills than females. ...............................

Proﬁle plot of the time *group interaction on the communication
subscale. The longitudinal trajectories from pre-test to post-test
for the two groups were nearly identical, which was why the

interaction effect was not signiﬁcant .................................................

Proﬁle plot of the time *group interaction on the group subscale.
The longitudinal trajectories from pre-test to post-test for the two
groups were nearly identical, which was why the interaction

effect was not signiﬁcant. ..................................................................

Proﬁle plot of the time *group interaction on the ability subscale.
The longitudinal trajectories from pre-test to post-test for the two
groups were nearly identical, which was why the interaction effect

was not signiﬁcant. ............................................................................

Proﬁle points for gender marginal means on the ability subscale.
Males had higher conﬁdence of problem solving abilities than

females ...............................................................................................

xi

........... 56

........... 57

........... 57

........... 59

........... 59

........... 60

........... 60

........... 61

........... 63

........... 64

........... 64

Figure 3.21: Proﬁle plot of the time *group interaction on the compatibility
subscale. The longitudinal trajectories ﬁ'om pre-test to post-test

for the two groups were nearly identical, which was why the
interaction effect was not signiﬁcant ............................................................ 66

Figure 3.22: Proﬁle points for gender marginal means on the compatibility
subscale. Males had higher engineering abilities than females. ................... 66

xii

CHAPTER 1

INTRODUCTION

This chapter provides the background to the research study. It discusses the
challenges being faced by the evolving ﬁeld of engineering education and how this ﬁeld
is responding to these challenges. In this context, initial freshman attitudes and their
changes during the freshman year, have been found to be highly correlated with retention
rate. This chapter describes the cornerstone sequence, a new initiative launched by the
College of Engineering at Michigan State University to meet the current challenges. It
sets the stage by explaining the context of the study and deﬁning the research questions.
Conducting such a study has far-reaching implications towards molding the future trends

in ﬁeshman engineering cuniculum design.

1.1 Current Trends and Challenges

Engineering education has been experiencing new challenges during the past two
decades [1]. Concerns that there might be something fundamentally wrong with science,
technology, engineering, and mathematics (STEM) education surfaced in the mid-19803.
Studies based on the large national samples of ﬁeshmen at 2- and 4-year institutions,
drew attention to a downtrend in the recruitment and retention of ﬁeshmen in STEM
majors [2, 3]. The greatest losses to STEM majors (34-40%) were found among high
school graduates who abandoned their intentions of entering STEM majors at or before
college enrolment [4]. During college, 53% of the freshmen who started their academic

program in engineering did not graduate with an engineering degree, and at least 50% of

this attrition took place during the ﬁ'eshman year [5]. Clearly, the freshman year is
critical for student success and retention in engineering programs [6]. Retention of
freshmen has been identiﬁed as a nation-wide concern that will affect the strength of the
future engineering workforce and, hence, the role of the United States as a dominant
world player in engineering and technology [1]. This becomes a major challenge as we
address the current global ﬁscal downturn and the technological advances needed to
stimulate the national and world economies. Considerable effort has been directed to
examining the high attrition rates at engineering institutions in order to develop effective
interventions [7].

Research suggests that both cognitive and affective issues contribute to attrition
among engineering students. While cognitive issues in engineering education involve
student knowledge and skills, affective issues relate to their attitudes towards engineering
and conﬁdence in their abilities to succeed [8]. The literature shows a strong evidence
that among all factors studied, attitudes have the highest correlation with retention [9].
The initial attitudes, and changes that take place in these attitudes during the ﬁeshman
year, affect student motivation, performance, and retention in engineering programs [10].
For instance, a multi-institutional, longitudinal study of engineering attitudes and how
they change during the ﬁrst year by Besterﬁeld-Sacre, et al., found that the students who
were most likely to choose engineering majors and completed degree requirements were
those who held positive perceptions of engineering, and had a measurable interest in
science and technology [1 l]. The same study found that students who avoided
engineering majors, or dropped out from engineering, were those who generally had a

negative impression of engineering, lacked conﬁdence in their abilities to complete the

engineering program, and had little or no motivation for studying science and
mathematics. Interestingly, the authors also found that students who left engineering in
good academic standing had signiﬁcantly different attitudes about engineering and
themselves than those who stayed in engineering and those who left engineering in poor
academic standing. Students who left engineering in good standing started their program
liking engineering less and had a lower appreciation of the engineering profession than
the other students. This category of students also liked math and science less and had
lower conﬁdence in their ability to succeed in engineering [9].

Also, in another longitudinal study conducted by Seymour and Hewitt [12] on
seven major 4-year institutions that produce most of the national supply of baccalaureate
scientists, mathematicians, and engineers found that students who left engineering were
not academically different than those who stayed in engineering, and that retention was
better correlated with their attitudes than with academic factors. They also found that
switchers (those who changed to non-engineering majors) and non-switchers had similar
educational experiences, but the non-switchers made more effective use of the resources
and strategies that enabled them to tolerate and overcome their difficulties [12].

These studies substantiate the argument that students’ initial attitudes towards
engineering are key to understanding attrition in engineering programs. Accurately
measuring students’ attitudes and changes in these attitudes over the course of the
freshman year allows us to develop effective means to evaluate the engineering
programs, to reduce attrition, and improve academic success [13, 14]. Attitude strength
may be an important element in this context. Social psychologists have identiﬁed several

aspects of attitude strength, ranging from the depth of knowledge that one possesses

about an issue to the extrerrrity of personal attitude about the issue [15]. Despite so much
variability in the conceptualization of this construct, researchers do agree that strong

attitudes are “resistant to change, persistent over time, and predictive of behavior” [16].

1.2 Institutional Efforts

The literature review indicates that several engineering institutions in the US. and
abroad have conducted attitude related studies to better understand their students, to
develop effective interventions, and to examine to what degree these interventions are
meeting their desired goals and objectives [10, 11, 17, 18]. To affect a positive change in
the students’ initial attitudes, one common and the most talked about intervention that
several institutions have adopted over the past ﬁfteen years is the introduction of design
and computation oriented courses - also called cornerstone or ﬁeshman sequence — in the
freshman year. An early introduction of engineering as a design and computation
oriented discipline is hypothesized to signiﬁcantly enhance student interest and
motivation towards engineering [19].

Sheppard and J enison provide a framework for exposing ﬁeshmen to key design
qualities and give speciﬁc examples of how engineering programs around the US.
revised their ﬁ'eshman curricula to include engineering design [20, 21]. A number of NSF
coalitions have developed valuable information on teaching ﬁ'eshman design courses to
improve the undergraduate engineering curriculum [22]. Research shows that these
courses signiﬁcantly contribute to the progress in academic achievement, create a
stimulating environment for advanced cognitive development, and offer diverse

experiential backgrounds and perspectives [23].

Based on the success of these interventions, many institutions (e. g. Purdue) have
developed regression models to predict attrition and student success even before the
students begin their pro grams [24]. They claim that these models allow academic
advisors to better inform students, especially those at high risk of attrition, of the
opportunities that engineering offers to develop tailor made programs to suit varied
student interests, and to set more realistic retention goals for the institutions [25]. Some
of the institutions have also reported beneﬁts of these changes in terms of signiﬁcant
improvements in retention rates [26]. Other efforts in this direction include: 1) a common
freshman year with integrated cunicula, which has indicated a positive impact on student
retention and learning [27]; 2) integration of residential living-learning communities for
making large campus environments smaller and more personal to connect freshmen more
closely with one another and with faculty that has shown signiﬁcant academic and
environmental gains [28]; and 3) raising of separate ﬁ'eshman engineering entities
(department/school/center) for better control and coordination of the ﬁ'eshman year.
Table 1.1 presents the initiatives taken by some of the CIC (Committee on Institutional
Cooperation) member universities, also named as Big Ten institutions, to improve

retention.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

CIC Institution Cornerstone Courses Common Residential Predictive Engr. Edu.
Freshman Year Programs? Modeling Dept / School
Purdue U. 1—3 courses Yes 3 Yes School
Michigan State U. 2 courses Yes 8 No Center“
U. of Minnesota 2 courses Yes 22 No No
U. of Michigan 2 courses Yes No No No
Northwestern U. 5 courses Yes (2 year) No No No
Ohio State U. 2-3 courses Yes No No No
U. of Iowa 2 courses, 1 seminar No No No No
U. of Wisconsin 2 courses No Females only No No
Penn. State U. 1 seminar No 3 No No
U. of Illinois No No No No No
"‘ Center for Engineering Education Research (CEER) T All residential programs

Table 1.1: A presentation of freshman engineering initiatives taken by some of the CIC
institutions to improve retention.

1.3

Freshman Engineering Experiences at Michigan State University

1.3.1 A Historical Perspective: The College of Engineering (CoE) at Michigan State

University (MSU) offers nine undergraduate degree programs with a yearly intake of

about 600-700 freshmen from all 50 US. states and dozens of other countries around the

world [29]. Until 1978, when the enrollment was ‘open’, a student only had to declare

engineering as his/her ‘major’ to get admitted to the CoE. Enrollments were high

 

' between 1978 and 1997; therefore the College limited admissions to junior year based on
student GPA and class size. In order to save on student time to degree and to improve the
student yearly intake, the Admit When Ready (AWR) initiative was launched in 2004. A
student in the AWR initiative could be admitted any time, between one and four
semesters, as soon as he/she fulﬁlled the requirement of ﬁve core courses and a minimum
GPA. As a result, the students were able to take 200 level courses one semester sooner
and were more focused in selecting their major. Between 2000 and 2005 engineering
enrollments dropped substantially (about 30%; see Figure 1.1). This decline exceeded the

national average largely due to the reduction in the manufacturing industry in Michigan.

 

1400

 

 
 
 

 

 

 

 

 

 

 

\ + Fresh
1200 VA - I» - Soph
x“ - -A- - Junior
I, --.X.-- '
1000 ix 1“ Semot
g r i k ‘- \
0 A’ ‘ x ‘\ -
g 800 ‘_‘a ﬁil‘ Kux
m X\.
500 ‘ f.
‘ a
400 I I I I I I— I I I I
F97 F00 F03 F06
Year

Figure 1.1: Engineering students’ retention trends at MSU from 1997 to 2007.

A need was felt to increase the number of undergraduate engineers by

recruitment, engagement, retention, or a combination of all. In an effort to overcome the

sharp downtrend, the CoE took three major initiatives in a phased program:

1.

2.

Spring 2006: instituted the ofﬁce of the recruitment and K-12 outreach.
Fall 2008: launched the cornerstone sequence and integrated the ﬁrst year into
a common ﬁ'eshman year. The ﬁve core courses requirement was accordingly
changed to include the new sequence.
Fall 2009:
a. Established the Center for Engineering Education Research (CEER)
[30].
b. Launched, “Engaging Early Engineering Students to Expand Numbers
of Degree Recipients (EEES)”, a $2.5 million NSF funded project
aimed at increasing student retention in the CoE from the current 65 %

to 75 % over a ﬁve-year period [31].

c. Upgraded and expanded its existing “Residential Option for Science
and Engineering Students (ROSES)” program to a true living-learning
experience.

d. Launched or continued discipline speciﬁc introductory design courses
at department level.

The new residential option has since been integrated with the cornerstone
sequence and named as “Residential Experience for Spartan Engineering”. It formally
started in fall 2009 with an initial participation of approximately 350 students. Wilson
Hall, one of the many residential halls on campus, has been conﬁgured to provide several
academic opportunities and services under one roof, in addition to holding classes and

laboratories for the two cornerstone design courses. The academic services include

student advisory services, free tutoring in math and science, peer leadersl, and themed
presentations by the College faculty [32]. It is intended to develop a community to bring
another dimension to the common ﬁrst-year curriculum to further enhance student
knowledge of the engineering profession, cultivate their problem solving skills, connect
students with campus and community resources, and enhance their communication skills.

The current admission policy of the College allows a student to remain no-
preference until 56 credits have been completed, i. e., until the beginning of the junior
year. Under this policy there are three categories of engineering freshmen at COR:

1. Engineering-preference: a particular engineering ﬁeld declared as major.

 

' A peer leader is an upper level student who assists freshmen with learning more about engineering
majors, the curriculum, college research, engineering career fairs, engineering student organizations.

2. Engineering-no—preference: engineering declared as major without preference

of a ﬁeld.

3. University-no—preference: no ﬁeld declared as major.

It is too early to comment on the outcomes of the new initiatives because it takes
several years of data to determine the trend. It is worthwhile, however, to take a closer
look at the fall 2008 initiative that forms the basis for this research study.

1.3.2 The New Cornerstone Sequence: The new sequence was designed to provide
freshman engineering students with a broad introduction to engineering design, the
engineering profession and its expectations, engineering problem-solving and teamwork
skills. It was an attempt to introduce engineering as a profession early in the career of the
students to put them on a path of inquiry. It was comprised of two new freshman courses:
EGR 100 (Introduction to Engineering Design) and EGR 102 (Introduction to
Engineering Modeling). EGR 100 was an addition to the existing ﬁve core courses
requirement for admission to the engineering program (Math 132, Math 133, Physics
183, Chemistry 141, CSE231), and was also a prerequisite to EGR 102. EGR 102 was a
replacement to the earlier computing course requirement (CSE 231) for all engineering
disciplines except computer engineering and computer science; computer engineering
and computer science majors continue to take CSE 231.

The College piloted the cornerstone design courses in fall 2007 for a test/trials
period of one year (i.e. two semesters: fall 2007 and spring 2008). During this trial period
the design courses were offered as optional cornses to a limited number of freshmen. The
comerstone sequence was integrated into the freshman curriculum in fall 2008.

Beginning fall 2008, the students were enrolled in the cornerstone sequence courses on a

ﬁrst-come ﬁrst-served basis provided they fulﬁlled the other academic requirements. Fall
2008 was unique in two respects: the cornerstone sequence was running in full swing for
the ﬁrst time; and two distinct groups of freshmen were available. Those enrolled in the
cornerstone sequence course(s) formed the desi gr sequence (DS) group and those unable
to enroll in the cornerstone course(s), either due to non-fulﬁllment of the math placement
requirement or non availability of the classroom space formed the traditional sequence
(TS) sump.

1.3.3 EGR 100 - Introduction to Engineering Design: The objective of this course
was to motivate and engage freshmen in the engineering profession. The course was
focused on structured problem solving and the engineering desigr process, learning to
work in teams and manage projects, ethics, the breadth of the engineering profession, the
interconnection of its disciplines, and its diverse contributions to society. It also
introduced computing tools and basic laboratory equipment used in support of
engineering design. It included three short team design projects and provided an early
gounding in the importance of gaining co-curricular professional experience throughout
one’s undergaduate years.

1.3.4 EGR 102 - Introduction to Engineering Modeling: The course focused on the
use of computational tools to solve technical and engineering problems. It exposed the
students to problem decomposition and identiﬁcation of a solution approach using tools
such as advanced spreadsheet features and MATLAB, data representation, curve ﬁtting
and analysis, mathematical modeling of engineering systems, and application of
principles through team-based engineering projects. Collaborative goup work was the

major operating mode for students in this course, with strong dimensions of both

10

individual and goup accountability underscored for all students. The course was oriented
towards ‘Problem-Based Learning’ (PBL) and ‘Just In Time’ (JIT) introduction of facets
of computational tools to apply to selected engineering problems.

The broad goals of the new initiative were: 1) attracting top students to
engineering progarns and retaining them; 2) better preparing students to adapt to a
quickly and constantly changing global engineering workforce by appreciating the
importance of teamwork, project management, innovation, hands-on experience, ethics,
career preparation and professionalism; 3) seeing engineering as a broad ﬁeld with many
opportunities; 4) positioning engineering as a favored choice for prospective students and
parents; 5) providing an opportunity for an early connection with the CoE and its faculty;
and most importantly, 6) eﬂecting an appreciable and positive change in the freshmen
attitude towards engineering. The cornerstone sequence was aimed at achieving these
objectives by raising the sense of community and interaction centered on design projects
to reap the beneﬁt of long, strong and integated technical education, and social and

professional development [33].

1.4 Context for the Study

This research study focused on the sixth goal discussed above. It sought to
examine the effects of the cornerstone sequence on ﬁ'eshman attitudes towards
engineering, and to establish whether the new sequence produced a signiﬁcant
improvement over the older traditional sequence (comprising students who did not
experience the new sequence) with respect to student attitudes towards engineering.

Whether or not this has been successful is an empirical question and one that this

11

research study sought to answer. A study of this nature could be best designed if two
cohorts were available for a direct comparison. In this respect, fall 2008 was a unique
semester in that students in both the streams, the traditional sequence (T S) and the design
sequence (DS) were available to form a comparison goup and a treatment goup,
respectively. This opportunity set the stage for the study — to examine the key research
question.

1.4.1 Primary Question: Is the cornerstone sequence an improvement over the
traditional sequence in terms of its eﬂects on freshman perceptions and attitudes towards
engineering?

As a corollary to the above, it was worthwhile to investigate the student
expectations and their reactions as they experienced the new sequence to better
understand the outcomes of the primary research question. A secondary research question
was accordingly formulated to reassure and reconﬁrm the ﬁndings of the primary phase.
1.4.2 Secondary Question: What are the freshman expectations of: and experiences
in, the new cornerstone sequence and how do these expectations and experiences
contribute (or not contribute) towards shaping their initial perceptions and attitudes

towards engineering?

12

CHAPTER 2

DESIGN OF THE STUDY

2.1 The Mixed Methods Design

This study employed Explanatory Mixed Methods Design that is characterized by
an initial and extensive quantitative phase followed by a qualitative phase. In this
approach, critical results of the quantitative phase are reviewed with one or more
qualitative methods to build, understand, and validate the earlier quantitative ﬁnding [34-
36]. This dissertation is accordingly divided into three phases: Quantitative , Qualitative ,
and Interpretation. The quantitative phase is based on the quantitative data and an
analysis that sought to answer the primary research question. To build on the results of
the quantitative phase, the qualitative phase was launched sequentially to answer the
secondary research question. The qualitative phase played a supporting role and the two
phases were mixed in the ﬁnal interpretation and discussion phase of the study as shown

in Figure 2.1 [34].

QUANTITATIVE qualitative Interpretation
Phase Phase Phase
QUANTITA TIVE QUANTITATIVE qualitative qualitative Interpretation
Data Collection Data Analysis Data Collection Data Analysis Entire Analysis

Figure 2.1: Explanatory mixed methods design.

 

 

 

 

           

2.2 The Quantitative Phase
To answer the main research question, a repeated measures approach was selected

that includes both ‘between subject factors’ (BSFs) and ‘within subject factors’ (W SFs).

l3

 

In this context, three well known hypotheses are of particular interest: the hypothesis of
no-change-over-time, or the ﬂatness hypothesis; the hypothesis of no-time-by-goup-
interaction, or the parallelism hypothesis; and the hypothesis of no-overall-group-
differences, or the level hypothesis [37]. These three hypotheses are described below.
2.2.1 ﬂatness Hypothesis: Freshman attitudes do not change over time when
disregarding goup membership.
H0,a = upre = ”post (1)
Where, ”are 5 average population mean of the two goups at pre-test, and
upost 5 average population mean of the two goups at post-test.
2.2.2 Parallelism Hypothesis: Freshman attitudes within each goup show similar
patterns of change over time. In other words, there is no time-by-goup interaction or no
treatment effect.
”0,123 #preJ‘S _ ﬂpostrs = #preps — #postpS
or, H0,b 3 A I-‘Ts = A ”as (2)
Where, A u rs 5 pre- to post- difference in the TS population mean, and
A abs 5 pre- to post- difference in the DS population mean.
2.2.3 Level Hypothesis: Freshman attitudes between goups are the same,
disregarding time.
H o,c 1 Hrs = #03 (3)
Where, [1 1'5 5 average TS population mean over time, and
abs 5 average DS population mean over time.

In geometrical terms the parallelism hypothesis states that if we gaphically

connect the means of the dependent variable group across time, then all resulting group

14

 

speciﬁc proﬁles will be parallel. With regard to the main research question, the
parallelism hypothesis is substantively the most interesting one. It asks whether there is a
differential change over time (i. e., treatment, intervention, etc.) on the response variable
for the TS and DS groups, and for this reason it is addressed ﬁrst. If there is no evidence
that the goups’ trajectories have different slopes over time, then the level and ﬂatness
hypotheses become more relevant. Hence, the no-interaction hypothesis (parallelism) was
investigated ﬁrst and only if it was not found to be sigriﬁcant were the ﬂatness and the
level hypotheses tested. The data from the two goups was collected with an existing
survey instrument at two time intervals (pre-, post-) over the course of one semester. A
General Linear Model (GLM) Repeated Measures Analysis of Covariance (AN COVA)

was selected for testing the three hypotheses because it provides estimates of effects for
WSFs and BSFs [37]. For this proﬁle analysis the statistical package SPSS© version 16.0

was used to process the data, represent the models, and analyze the results.

2.3 The Qualitative Phase

The secondary research question was investigated with a qualitative approach.
The aim was to better understand the kind of learning that took place in the new
cornerstone sequence and if it affected the participants’ initial perceptions and attitudes
towards engineering. Since, this was an exploratory question, there was no a priori
hypothesis for this part of the study. Rigorous qualitative research involves its own set of
data collection and analysis methods to ensure the trustworthiness and authenticity of the

ﬁndings [3 8]. One-on-one interviews with a representative sample (from the DS goup)

15

were considered the most suitable qualitative approach for the secondary research
question.

The overall objective was to understand initial freshman attitudes as they enrolled
in engineering progams and whether those attitudes were affected by the new
cornerstone sequence. Improved understanding of attitudinal changes could help in the
formative evaluation of the new sequence. More importantly, it could help develop better
evaluation methods for engineering pro gams by incorporating and integrating students’
attitudes as an addition to the existing feedback system. Setting up a wider scope in the
quantitative phase, using the three research hypotheses, was therefore a deliberate effort
to not only observe the time effects on the attitudes of two student goups, but also study
the changes in those attitudes due to important demogaphic factors (e. g., gender and past
academic performance) that could lead to a better understanding of engineering
freshman. This study is a contribution towards applying mixed methods in the study of
engineering education. The data collected in the empirical method could be built up
further for developing predictive models that could help students make informed

decisions assuring future success.

2.4 Organization

The remaining part of this document is organized in three phases: Quantitative,
Qualitative, and Interpretation/Discussion. Chapter 3 is devoted to the Quantitative phase
of the study; it includes the methodology, collection and analysis of the quantitative data
collected over the course of one semester (fall 2008). Chapter 4 discusses the qualitative

phase; the methodology, development of interview protocol, collection and analysis of

16

 

the data collected in the subsequent semester (spring 2009). Chapter 5 discusses the
overall results and formulates conclusions based on the ﬁndings of the two phases in

accordance with the mixed methods design approach.

17

CHAPTER 3

THE QUANTITATIVE PHASE

This chapter seeks to answer the primary research question with the quantitative
approach. It has two key objectives: to describe the methodology of the quantitative
approach and to analyze the collected data. To meet these objectives this chapter unfolds
into ﬁve sequential processes: 1) development of a general statistical model to address
the primary research question; 2) instrumentation including the process of instrument
selection, transformation, conﬁrmation with a pilot study, and implementation; 3) data
collection and collation; 4) selection of variables for the model and implementation of the
statistical model; and ﬁnally 5) data analysis and discussion. The chapter is divided into

two parts: Methods and Results/Analysis.

Part I - Methods

3.1 The General Model

The General Linear Model (GLM) provides a ﬂexible ﬁamework for the proﬁle
analysis of data in this phase of the study. Repeated measure AN COVA was used for
testing the three hypotheses for the main study. This model allows for comparison of the
two goups on measurements made at the beginning and towards the end of the semester
while controlling for one or more continuous and/or categorical covariates (for example,
ACT scores and gender). The model facilitates testing hypotheses about the effects of the
BSFs, for example goup and gender (level hypothesis), and the WSF, i. e., time (ﬂatness

hypothesis). More importantly, one can investigate interactions between factors as well

18

as the effects of individual factors, including covariates, to answer the main research
question (parallelism hypothesis). Estimated marginal means (EMMs) are the predicted
mean values for the cells in the model after controlling for other variables. They allow us
to interpret the main effects of categorical predictors such as goup and gender, while
proﬁle plots (interaction plots) of EMMs illustrate the nature of interaction effects [3 7].

The general linear model can be represented in vector notation as

Y = XB + e
=[1 x1 X2” .lﬁ:xk][ :ol+£
Y = 30 + 95131 + x232 + + kak + 5- (4)

Where, Y E dependent variables vector,

B 5 unknown coefﬁcients vector,

X :- desi gr matrix comprising independent variables including covariates,

e '=‘ errors vector, and

k 5 number of independent variables.

In case of interactions among the independent variables, equation 4 will have
appropriate interaction terms. For example, for all possible two way interactions
among x1, x2, and x3, equation 4 will have the form

Y = 30 + x1131 + x232 'l' x333 + (x1 * x2W4 + (x2 * x3335 + (x1 * xsme + 5-

For a set of n outcomes, the model (main effects only) takes the form

1 x11 xlk 301 302 3011
{’52:} [1 x21 1:. x2kl [ﬁll 3.12 2." Bl?! WIT-E 3"] (5)

1 x711 "' xnk k1 Bkz

l9

Again, for interaction among the independent variables, the model will have the
interaction vectors added to the desi gr and unknown coefﬁcients matrices. For repeated
measures, the model is augmented with the number of levels or WSFs. For a pre-, post-

design (two levels), the general model can be represented as

553.1% 2 iil+l§§l (6)
Where, Prei E pre- dependent variables vector,
Posti E post- dependent variables vector,

3,- E unknown coefﬁcients matrix for pre-,

yi E unknown coefﬁcients matrix for post-,

X E desigr matrix for independent variables (same for pre- and post-),

St E errors vector for pre-, and

é; E errors vector for post-.

Equation 6 was used to develop me, post- models for the study to examine the
three null hypotheses (i. e., equations 1, 2, and 3). The pre- and post- models in generic
form are represented as equations 7 and 8, respectively. These models inherently
incorporate the time interactions (i. e. pre- and post- or WSFs). Inter-independent
variables interactions (B SFs) can be appropriately added to these models, if needed.

Prei = 30: + xrﬁri + 35232: + + xklgki + 8r . and (7)

POSti = Yo: + 35171: + 75272: + + xkl’ki + éi- (8)

3.2 Instrumentation
The ﬁrst step in such a research study is the identiﬁcation and/or development of

valid and reliable measures to be used to evaluate student learning, attitudes, and

20

experiences within an engineering progam, and whether the new freslunan engineering
experiences were actually succeeding in meeting their declared goals. Given that both
cohorts were available, the study involved comparing attitudes as outcomes of the DS
goup to that of the TS goup. For this purpose a reliable and valid attitude measuring
scale is required [39]. This section describes the selection and modiﬁcation of a suitable
attitude scale and its performance in a pilot study conducted in spring 2008 (prior to the
main study).

3.2.1 Instrument Selection: The relevant literature identiﬁed several types of

 

assessment instruments commonly used by engineering educators: closed form E
questionnaires, open-ended surveys, one-on-one interviews, focus goups, essay
questions, ethnogaphic studies, portfolios, student journals, and verbal protocols. The
closed form questionnaire was selected for the study because: 1) it provides a reliable
assessment of student attitudes; 2) it is commonly used to measure impressions of
engineering, enjoyment of working in goups, and self-assessed competencies; 3) it is
easier to administer; 4) it can be given to a large number of subjects at a minimal cost; 5)
the responses to the questionnaire can be given with a check list, Likert scale, or semantic
differentials; and 6) repeated use of the instrument can measure changes in attitudes over
time or the effect of a particular intervention [39-44].

Developing and validating good survey instruments are tedious and time
consuming tasks. Experts in this ﬁeld strongly recommend using available instruments to
save time in instrument construction and validation [3 9, 45]. An extensive literature
search was undertaken to identify a valid and reliable survey instrument that could

measure attitudes among student cohorts and, particularly, how they were impacted by

21

the cornerstone cornerstone sequence. The search revealed a general scarcity of
standardized instruments in engineering education. Some need-based instruments have
been developed by a few engineering institutions. Astin for example, developed a closed
form survey to measure the attitudes and perceptions of entering freshmen [46]. The
instrument, however, does not have the follow-up component to determine if the
differences among the cohorts persist over the course of a semester. Four commonly used
instruments were examined for this study.
1. Pittsburgh Freshman Engineering Attitudes Survey (PFEAS), developed at the
University of Pittsburgh [19].
2. Freshman Engineering Attitude Survey (FEAS) and Freshman Engineering
Perception Test (F EPT), developed at Texas A&M University [47].
3. Entering Freshman Engineering Survey (EFES), developed at Arizona State
University [48].
4. Cooperative Institutional Research Progam (CIRP) Survey, developed at the
UCLA Higher Education Research Institute (HERI) [49].
3.2.2 Pittsburgh Freshman Engineering Attitude Survey (PFEAS): PF EAS was
selected as the base line instrument for this study because: 1) it was the most relevant
since it was originally developed for a similar study; 2) the student attitude was measured
by gouping the items under thirteen measures or factors, most of which were of interest
to this study; 3) it had been extensively used by various institutions and cited in a number
of refereed publications [50]; and 4) it had an established high degee of validity and
reliability [11]. A recent work has pointed out some weaknesses of the original

instrument [51] and a revised version has been developed [52]. Unfortunately, this

22

revised version is under test and evaluation and was not yet available when the study was
conducted.

The pre-version of the scale was comprised of ﬁfty items. It was desigred to
measure four facets of student attitudes: 1) student deﬁnition of engineering; 2) student
attitude about engineering; 3) student self-assessed conﬁdence; and 4) student self-
assessed skills including working in goups. The post-version was comprised of twenty
additional items that captured student perceptions of their attainment of the eleven
Engineering Criteria (EC) 2000 outcomes as deﬁned by ABET (Accreditation Board for
Engineering and Technology) [53]. Data mapping on EC 2000 outcomes have been used
to demonstrate cross institutional gender and ethnicity differences that parallel those
found for the attitudinal measures [11]. For this research study, the pre- version of the
scale was selected, since data mapping for EC2000 outcomes was not the scope of the
study. For the purpose of this study PFEAS would imply the pre- version of the scale.
The original PFEAS scale is attached as Appendix A.

The scale was originally developed in 1993 by Besterﬁeld-Sacre et al., for a
similar study at the University of Pittsburgh [19]. Since then, it has been adopted by
several institutions to evaluate their freshman pro garns, study attrition and probation
issues related to freshmen, and to measure EC 2000 outcome issues [10, 54, 55]. As part
of the validation process, PFEAS underwent rigorous pilot testing and improvement by
means of item analysis, verbal protocol elicitation, and factor analysis [50]. The scale
was thoroughly tested for reliability and validity [11, 40]. Fifty items of the instrument
statistically cluster into thirteen attitudinal measures or subscales, as listed in Table 3.1

[55]. These subscales deﬁne the domain of the instrument’s main construct: freshman

23

attitude towards engineering. The survey items are rated on either a ﬁve point Likert

scale or an ordinal self-assessed conﬁdence scale.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

No. PFEAS Items Subscale Name Definition of Subscale

1 1-3,4*,5,6*,7,8*,9* Career General impression of engineering

2 10,14,21,23 Jobs Financial inﬂuences for studying engineering

3 11, 20 Society How engineers contribute to society

4 12,17,18,22,25,27,28 Perception Work engineers do and engineering profession

5 13,19"' Math Enjoyment of math and science

6 15,26 Exact Engineering perceived as exact science

7 16,24 Family Family inﬂuence to studying engineering

8 29,30,31,32,35 Basic Conﬁdence in basic engineering knowledge and skills
9 33,34,35 Communication Conﬁdence in communication and computer skills
10 39",46 Study Adequate study habits.

ll 37,43*,45* Groups Working in goups.

12 38,40,42,49,50 Ability Problem solving abilities.

l3 36,44,47,48 Compatibility Engineering abilities.

 

 

 

 

 

*Reverse scored items.

Table 3.1: Deﬁning the PFEAS subscales.

The subscales of the instrrunent have been determined by factor analysis of a
large sum of data collected from several universities and colleges at a national level [19,
40]. These subscales have, over the years, standardized to one common set of values or
loadings [11]. An independent item analyses of the data collected in this study conﬁrmed

that the original thirteen subscales of the instrument hold well. Principle component

method was used with Varimax rotation to extract the factors in SPSS© 16.0. A

comparison of the factor loadings of the instrument using the data for this study and the

one provided by the authors is given in Appendix Bz[56]. A close match between the two
results points towards good reliability. Also the internal consistency as measured by

Cronbach’s alpha (CA) has been reported as 0.8 or better for each of the subscales [18,

 

2 This study data initially extracted fourteen factors that reduced to thirteen, as the participants increased.

24

57], which is good given the acceptable limit of 0.7 [5 8, 59]. The author’s factor loadings

were, therefore, mapped on to the study data (with the permission of the author) to

develop the mathematical expressions for pre- and post- subscales. The mapping would

also enable realistic comparison of results with other similar studies that employ PFEAS.

It may be noted that mapped scales vary between 1 and 5 (with ﬁner step variations) and

were treated as continuous outcome variables. Table 3.2 shows the mapped expressions

for pre- and post- subscales.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Pre- Post- Subscale Name Mathematical Expressions for Subscales

0.124“ 1+0.121* 2+0.120*1nv +0114" 3+0.1 14* 7+0.l 13"
PM PM" cm lnv_Q830.104*Q530.100‘Inv_Q9-?(:090*Inv(3Q6) Q
Pre 2 Post 2 Jobs T (0.301*Q23+O.289*Q14+0.206*QZ1+0.204*Q10)
Pre 3 Post 3 Society (0.519*Q20+0.481‘Q1 1)

t It It i I!
P re4 Post 4 Perception ((31:30Q1B2;%1267‘; Q22+0.163 Q25+0.l43 Q28+0.l35 Ql7+0.128
Pre 5 Post 5 Math (0.525*Inv Q19+0.475*Q13)
Pre 6 Post 6 Exact (0.516*Q15+0.484*Q26)
Pre 7 Post 7 Family 1' (0.586*Q24+0.414*Q16)
Pre 8 Post 8 Basic (0.244’Q30+0.233*Q32+0.224*Q31+0.169*Q29+0.130*Q35)
Pre 9 Post 9 Communication (0.161 *Q35+0.422"'Q33+O.416*Q34)
Pre 10 Post10 Study 1 (0.501 *Q46+0.499*Inv_Q39)
Pre 11 Postll Groups (0.385*Inv_Q43+O.363*Q37+0.252*Inv_Q45)
Pre 12 Post12 Ability (0.247‘Q50+0.244"'Q49+0.177*Q38+0.169*Q42+0.163*Q40)
.Pre l3 Postl3 Compatibility (0.303*Q47+0.294*Q48+0.202*Q36+0.20l*Q44)
1‘ Not relevant to the study.

Table 3.2: Mapping of the PFEAS subscales.

3.2.3 Tuning the Scale: A detailed examination of the instrument revealed that three

subscales, namely jobs (ﬁnancial inﬂuences for studying engineering), family (family

inﬂuence to studying engineering), and study (conﬁdence about study habits) were not

relevant to this study because any likely differences between the DS and the TS goups

on any of these three subscales would not reﬂect a treatment effect since the curriculum

does not address these topics. To ensure scale reliability, the original instrument was used

for data collection and only the relevant ten subscales were examined for data analysis.

25

 

For consistency, the original names and numbers of the subscales were retained as shown
in Tables 3.1 and 3.2. Another important factor for instrument construction is it’s time-
to-complete by an average participant. Participants generally lose interest if the
instruments take longer than 20-25 minutes. To reduce the participants’ time,
demogaphic items were removed ﬁom the instrument as accurate demo gaphic
information was available ﬁom the Associate Deans’ ofﬁce. The relevant subscales for
this study and their associated items are given in Table 3.3.

3.2.4 Institutional Review Board (IRB): PFEAS was submitted to the University’s

IRB for approval prior to its employment, in accordance with federal regulations. The

 

study was approved in the category of ‘Expedited Review ’ which allows the researchers
to retain personal identiﬁers for tracking the subjects in longitudinal studies. It requires
students’ written consent for the release of personal identiﬁers and demogaphics.
Necessary consent forms for the purpose, as required by the state and federal regulations

were prepared and made part of the survey questionnaire accordingly.

26

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Sub-scale Item Item statement
1 I expect that engineering will be a rewarding career.
2 I expect that studying engineering will be rewarding.
3 The advantages of studying engineering outweigh the disadvantages.
1. 4* I don’t care for this career.
Career 5 The future beneﬁts of studying engineering are worth the efforts.
6* I can think of several majors that would be more rewarding than engineering.
7 I have no desire to change to other major (bio, Eng, chem, art, hist., etc.)
8* The rewards of getting an engineering degree are not worth the effort.
9* From what I know, engineering is boring.
3. ll Engineers contribute more to making world better than other occupations.
Society 20 Engineering is more concerned with welfare of society than other professions.
12 Engineers are innovative.
17 Engineering is an occupation that is respected by other people.
4. 18 I like the professionalism that goes with being an engineer.
Perception 22 Engineers have contributed greatly to ﬁxing problems in the world.
25 Engineers are creative.
27 I am studying engineering because I enjoy ﬁguring out how things work.
5. 13 I enjoy the subjects of science and mathematics the most.
Math 19* I enjoy taking liberal art courses more than math and science courses.
6. 15 Engineering is an exact science.
Exact 26 Engineering involves ﬁnding precise answers to problems.
29 How conﬁdent you are of your abilities in chemistry.
7. 30 How conﬁdent you are of your abilities in physics.
Basic 31 How conﬁdent you are of your abilities in calculus.
32 How conﬁdent you are of your abilities in engineering skills
35 How conﬁdent you are of your abilities in computer skill
9 33 How conﬁdent you are of your abilities in writing.
Communication 34 How conﬁdent you are of your abilities in speaking.
35 How conﬁdent you are of your abilities in computer skill
11 37 Studying in goup is better than studying by myself.
Group 43"“ I prefer studying/working alone.
45 "' In the past, I have not enjoyed working in assigned goups.
38 Creative thinking is one of my strengths.
12 40 I have strong problem solving skills.
Ability 42 I feel conﬁdent in my ability to succeed in engineering.
49 I enjoy solving open-ended problems.
50 I enjoy problems that can be solved in different ways.
36 I feel I know what an engineer does.
13 44 I am good at designing things.
Compatibility 47 I consider myself mechanically inclined.
48 I consider myself technically inclined.

 

 

*Reverse scored items.

Table 3.3: Itemized detail of the ten selected PF EAS subscales.

27

: .uﬂ

 

3.2.5 Pilot Study: The CoE piloted the cornerstone sequence courses for two
semesters, prior to launching the full sequence in fall 2008. EGR 100 was piloted ﬁrst in
fall 2007, while in spring 2008, EGR102 was introduced alongside EGR100 as optional
courses to a limited number of freshmen. Spring 2008 was the ﬁrst time the new
cornerstone sequence coexisted with the older traditional sequence forming two cohorts
of students. This provided an opportunity for a pilot study with these goals: 1) to
understand the new cornerstone sequence objectives; 2) to establish the efﬁcacy of
PFEAS and better understand its subscales and performance parameters; 3) to gain
expertise in the collection and collation of data with on-line and paper-pencil surveys
using the Survey Monkey® platform, a commercial survey agency; 4) to gain expertise in
the effective use of SPSS© 16.0, a data processing and analysis package; and 5) to

examine the performance of various demogaphic predictors as covariates for the main
study.

The data of the pilot study was collected once, towards the end of the semester.
The DS goup population was approached in person during their regular lab sessions and
was urged to take the survey via a link. The TS goup population was not available in any
single class so they were sent the link via the university e-mail system. The sample was
comprised of 256 students from a population of 638 freshmen. It formed 73% of the DS
goup and 36% of the TS goup. The high turnout for the DS goup (73 %) was because
the survey was conducted in-class, which obviously makes a marked difference. A host
of dernogaphics were available that included age, gender, ethnicity, high school, high
school GPA, and ACT/SAT scores. The ethnic distribution was 76% Caucasian, 6%

Asian/Paciﬁc Islander, 7% Aﬁican American, 3% Hispanic and 8% Not reported. The

28

"......V. i" 'A

 

Kick. ' .. L." _

ten attitude subscales were examined for the presence of sigriﬁcant differences in the
two goups. The frequency distributions, histogarns, kurtosis, and skewness results
coupled with the Central Limit Theorem (n > 30) conﬁrmed normality of the outcomes
and supported the use of the parametric approach for the data analysis. The internal
consistency for most of the subscales was above the acceptable limits (CA > 0.7) except
math, exact, basic, and communication (CA: 0.5~0.7). The subscales with fewer items
obviously suffer ﬁ'om internal consistency [51], however, the overall scale was known to
hold well on internal consistency and content validity [9, ll, 18].

The statistical modeling or hypotheses testing was not within the scope of the

 

pilot study. However, with the available data, it was important to understand the inter-
relationship of the two goups with the demogaphic data. For the pilot study, gender and
ACT -composite score were selected as predictors. Independent samples t-test (goup vs.
ACT-composite: p = 0.414) and chi-square test (goup vs. gender: Pearson chi-square =
0.367) conﬁrmed that selected variables did not have signiﬁcant relationship with the two
goups; a useful ﬁnding for the main study. The pilot study helped in developing better
understanding of the two cohorts of students. Moreover, it provided a platform to test the

survey instrument and a hands-on experience on data collection and data processing with
SPSS©. More importantly, it helped in selecting good demogaphic covariates for the

main study. Also, the instrument in its original form was found to hold well in

consistency and gouping of the subscales.

29

3.3 Data Collection

As mentioned earlier, data collection for the main study was conducted in fall
2008 when both goups, the TS goup and the DS goup, were running simultaneously —
an ideal situation to examine the key research question and its related three hypotheses.
The data was collected twice: at the beginning (pretest) and towards the end of the fall
semester (post-test). The aim was to capture the WSFs (time changes) for ﬂatness and
parallelism hypotheses, in addition to the BSFs for level hypothesis. Two factors that

could affect the sample size were the time the survey was administered, and the mode of

 

its administration.

3.3.1 Timing of the Survey: Ideally, for a pre- and post- study, the data should be
collected at the beginning and towards the end of a treatment to capture its full effect. For
this study the pre- and post— surveys were conducted during the 3rd and 14th week of the
16 week semester, respectively. This was necessary due to student availability and
commitment issues. Since this was their ﬁrst semester, student availability was low in the
ﬁrst two weeks. Also, towards the end, they were overly busy preparing for the desigr
day presentations and ﬁnal exams scheduled in the 15th and 16th weeks, respectively. This
strategy did yield a moderate to good sample size but raised a question about the partial
treatment effect (more on this in the Discussion section).

3.3.2 Mode of Survey Administration: The two goups, by deﬁnition, were different
in many ways. Those enrolled in EGR 100 and/or EGR102 formed the DS goup. These
freshmen could be approached physically as they shared at least one classroom activity.
Those not enrolled in the cornerstone sequence formed the TS goup. These students had

no common class to share and therefore could not be approached in person as a goup.

30

The data collection methods and incentive offers for the two goups had to be made
different in accordance with their peculiar circumstances. It is important to note that the
‘two goups were not formed by a random selection of subjects as dictated for ideal
statistical analysis. TS goup was comprised of those students who were not admitted to
the DS courses for any reason that ranged from ineligibility due to math deﬁciency to
non-availability of classroom space for DS courses. This limitation was kept in view I
while drawing conclusions. A brief description of mode of survey administration for the
two goups is presented below.

3.3.2.1 DS Group: To ensure maximum participation, the survey for the DS goup was

 

conducted in-class during laboratory sessions using laboratory computers. Students were
sent a link to the survey from SurveyMonkey® and were encouraged to take the survey.
An incentive of entry into a drawing for a free iPod was offered for completing the
survey. Extra credit equal to one homework assignment was also arranged through the
class instructors for taking the survey. There were 450 students registered in EGR 100
and 45 in EGR 102. EGRIOO was further distributed in thirteen laboratory sections while
EGR 102 had two sections. Conduct of the survey was a time-intensive process. Each of
these ﬁfteen sections of the DS goup were personally visited at the beginning of the
class session for a brief introduction to the study to encourage the students, to obtain their
written consent before the survey, and to conduct the survey on the lab computers. It took
approximately one week to complete each survey. The post-participation rate for the DS
goup was 82% and 84% for EGR 100 and EGR 102, respectively.

3.3.2.2 TS Group: Freshmen in the TS goup were composed of 227 students spread

all over the campus and were not physically approachable as an assembled goup.

31

Therefore, an on-line survey was the best option. The TS goup students were sent the
survey link via the university’s secure web mail and urged to respond. They were also
offered the incentive of being entered into the drawing to win an iPod. Several emails
were sent as reminders to persuade these students to take the survey. Despite the
reminders and the incentive, the post-participation rate for the TS goup was only 20%.
The sample size was further reduced because only subjects with both, pre- and '

post-responses could be considered for the study and subjects under the age of 18 years

could not take the survey as per the federal regulations. The effective sample size was

further affected due to “missing values” in the control variables added to the model

 

(more on this later). Table 3.4 shows a summary of the data collected for the two goups.

 

 

 

 

 

 

 

 

 

 

TS Group DS Group
Freshman P P EGR 100 EGR 102 Total
re ost Pre Post Pre Post Pre Post
Population (N) 227 227 450 450 45 45 495 495
Sample (n) 71 46 381 368* 43 38 42 406
Participation rate 31 20 85 82 95 84 86 82

 

 

 

 

 

 

 

*9 incomplete surveys dropped.
Table 3.4: Pre- and post- data collection during 3rd and 14th week of fall 2008,

respectively.
3.3.3 The Sample: An overview of the sample gives a general description of the
ﬁeshman population at MSU. There is a large difference between the two sample sizes:
406 in the post-DS goup (EGR 100 and EGR 102 combined); and, 46 in the post-TS
goup (Table 3.4). It is known that large differences in sample sizes could affect the
robustness of the model [60]. The data ﬁ'om EGR 102 was dropped from the main study
because: 1) it reduced the absolute difference in sample sizes without affecting the TS
goup; 2) it simpliﬁed the DS goup dynamics, now belonging to one course, i. e. EGR

100; and most importantly, 3) it was a computational course offered to a select goup of

32

majors as a replacement to the existing CSE 231 course. EGR 100 was the new addition
to the curriculum. It was the design component of the new sequence and was mandatory
for all majors. The post-sample sizes for the DS and TS goups (alter dropping the data
of EGR 102) were 368 and 46, respectively. The post-sample included 83% males and
17% females. The ethnic distribution was 80% Caucasian, 5% Asian/Paciﬁc Islander

(PI), 8% Aﬁican-American (AA), 3% Other and 4% Not Reported (NR) as given in

 

 

 

 

 

 

 

Table 3.5.
Gender Ethnicity
Group
M F Total Caucasian Asian/PI AA Other NR Total

D3 307 61 368 322 19 11 9 7 368
TS 35 ll 46 10 3 22 3 8 46
Total 342 72 414 332 22 33 12 15 414
Percentage 83 17 80 5 8 3 4

 

 

 

 

 

 

 

 

 

 

 

Table 3.5: Gender and ethnicity distribution of the sample.

3.4 Selection of Variables

Prior to developing the speciﬁc models for the study, the data was examined for
selection of relevant variables for these models. To select the best possible combination
of predictors, covariates, and outcomes, all available dernogaphics were considered. The
rationale for selecting (or excluding) these variables is discussed below.
3.4.1 Dependent or Response Variables (DVs): Thirteen PFEAS subscales formed
the domain of the main construct, that is, freshman attitudes towards engineering. These
subscales were the outcomes or DVs for the statistical model. PFEAS subscales are the
proven outcomes of factor analysis of a large sum of data from multiple longitudinal
studies. The subscales were developed by simple addition of the factored PFEAS items,

duly normalized, with the original factor loadings. Normalization was done for ease of

33

 

reference and standardization. Two sets of thirteen DVs were deﬁned: Pre] -Pre13 for
pre-test scores and, Post1-Post13 for post-test scores, as given in Table 3.2. The ten
selected DVs for this study were examined for treatment effects while controlling for the
confounding effects of carefully selected predictors or independent variables.

3.4.2 Independent or Explanatory Variables (IV s): It is essential for a statistical
model to have a good set of IV s (also called predictors, covariates, and explanatory E
variables) that could explain maximum variations in the response variable(s); the higher

the predictability, the better the IV. Equally important, if not more, is the choice of NS

 

that is contingent upon the research question which dictates the relationship to be tested. 1
IV 3 must be able to represent this relationship. In other words, the NS should have high
correlations with the outcome(s) and should be independent of each other or at least have
low correlations with each other. It was therefore necessary to search for a parsimonious
set of NS that could explain maximum amount of variance in the outcome(s) thus
minimizing the error term (equation 4). A set of seventeen IVs available for this study is
brieﬂy discussed below.

3.4.2.1 Group: A dichotomous variable that distinguished between the two cohorts of
freshmen: the DS (treatment) goup comprising freshmen in the new cornerstone
sequence (EGR 100 only), and the TS (control) goup formed by freshmen not registered
for the cornerstone sequence, in the so called older traditional sequence. It may be
pertinent to re-emphasize that fall 2008 was a unique semester from the stand point of
this study since both cohorts were available for direct comparison. The TS goup size
would considerably decline in the subsequent semesters as the College increased its

capacity to enroll more students for the mandatory courses. “Group” was the most

34

important predictor since it reﬂected the effect of primary interest in the research

question. Other IV s were added to the model to control for their effects while examining

the goup-vs.-DVs relationship over time.

3.4.2.2 Other IVs: To select the best ﬁt for a statistical model, the sixteen other IVs

were grouped into three categories: backgound, past performance, and present

 

 

 

 

 

 

 

 

 

 

 

performance as shown in Table 3.6.
Variable Type 3:21:81 Remarks
Background
- Group" Dichotomous 0, 1 Most relevant
- Age Interval 18-21 Low variability
- Gender“ Dichotomous 0, 1 Most interesting
- Ethnicity Nominal 1-5 Low cell count
- Citizenship Nominal 1-3 Low cell count, low variability, less relevant
- Resident State Nominal l~50 Less relevant, low variability
- Resident County Nominal 1~80 Less relevant
- H.S.]' attended Nominal l~500 Less relevant
Past Performance
- ACT composite“ Interval 11-36 Relevant covariate
- ACT math Interval 11-36 High correlation
- SAT math Interval 200-800 High correlation
- SAT verbal Interval 200-800 High correlation
- H.S.1’ GPA Interval LOO-5.00 Non standard
- Math competency Ordinal 1-3 High correlation, low cell count
Present Performance
- F S07 GPA Interval 1.00-4.00 High correlation
- SSOS GPA Interval LOO-4.00 High correlation
- SS08 CGPA Interval LOO-4.00 High correlation

 

 

"‘ Selected. T High school
Table 3.6: Deﬁning the three categories of other NS.

1. Background: As the name suggests, this category of variables was

associated with the participants’ identity. It included age, gender, ethnicity,

citizenship, resident state, resident county, and high school attended.

2. Past Performance: Past performance deﬁned the subject’s academic

performance before joining the university as a freshman. It included ACT

composite, ACT math, SAT math, SAT verbal, high school GPA and math

35

 

 

competency. Math competency was an ordinal variable; it deﬁned the
subject’s math proﬁciency vis-a-vis a math course prerequisite for the
cornerstone sequence (Math 116 or equivalent). Table 3.7 shows the
distribution of this variable.

3. Present Performance: This category described the subject’s current
academic performance as a freshman. It included fall 2007 GPA, spring 2008 ‘
GPA and spring 2008 CGPA.

3.4.3 Selection Criteria: Each of the sixteen other IVs was examined for relevance

‘I'nl'h J- Wu. Hf:

 

 

.‘""

and usability vis-d-vis the research question. Variables not useful and/or not useable were
dropped based upon the following selection criteria.

3.4.3.1 Correlation: Highly correlated IVs reduce power, pose collinearity issues and
therefore must be avoided. N3 in the past performance and present performance
categories represented the students’ academic capability/performance and were obviously
highly correlated. Only one variable from these two categories could be selected that best
explained (correlated with) most of the response variables and at the same time
represented the freshman’s true academic capability. Present performance category
represented students’ GPAs for one or two semesters and was, therefore, not a true
reﬂection of their academic performance.

Past performance was comprised of three types of scores, high school GPA, math
competency, and ACT/SAT score. High school GPA could have been an ideal candidate
but it suffered from non standard format. High Schools in the US. do not have a standard
gading system; some gade on a 4.00 scale and others on a 5.00 scale. It is difficult to

compare the two scores with reasonable accuracy. Math competency represented only

36

one aspect of the student performance. The variable also suffered from low cell count (11

< 10, see Table 3.7).

 

 

 

 

 

 

 

Level Deﬁnition Math qualification (course name) DS TS Total
1 Under-qualiﬁed MTH-l825/ 103 6 29 35

2 Qualiﬁed MTH—l 16/124/132/152H 255 10 265

3 Over-qualiﬁed MTH-133/153H/234/2541-I/235 95 5 100
Total 356 44 400*

 

 

 

 

 

 

*14 missing values.

Table 3.7: Distribution of math competency between DS and TS goups showing low
cell count (n < 10).

ACT or SAT score is considered a measure of the student academic performance
by most of the engineering institutions. The two scores are highly correlated (r = 0.854).
Most of the applicants in the Midwestern region take the ACT whereas students in the
other US. regions and most of the foreigr students take the SAT exam [61]. ACT scores
were available more often, so SAT scores were converted to equivalent ACT scores
wherever the latter were not available or missing [62]. The ACT score is comprised of
two components: ACT composite and ACT math. The two are known to be highly
correlated. The ACT composite was selected because it is broad based and is commonly
used for admission screening by the engineering institutions. The ACT composite is a
continuous variable with a normal distribution (Figure 3.1). An independent-sarnples t-
test for ACT composite score showed DS goup means were sigriﬁcantly higher than TS
goup means (Table 3.8). ACT composite was found to be mildly positively correlated
with most of the ten outcomes (.015 < r < .276). It should work as an important covariate

and be watched for collinearity issues.

37

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

45-
—+_: Mean =25.73
r \ Std. Dev. =3.835
/ K N =3.
30— '
3 [A
C
s 7
E
15..
c I I I I I I
10 15 2o 25 30 35 40
ACT Composite
Figure 3.1: The ACT composite score distribution in the sample. .
Group Statistics t-Test for Equality of Means
ACT Group N Mean Sig Mean 99.9% Confd.
composite TS goup 38 20.71 t df (2-tail) D'ﬁ' Lower Upper
DS goup 351 26.27 —9.395 387 .000 -5.560 -6.724 -4.397

 

 

 

 

 

 

 

 

 

 

 

Table 3.8: Independent samples t-test for ACT composite score in the two goups.

3.4.3.2 Variability: Low variability is of little utility to the model. Three variables in
the backgound category: age, citizenship, and resident state had low variability as shown

in Table 3.9; therefore, these variables were dropped.

 

 

 

 

Resident State ('/o) Citizenship (%) Age (years)

Michigan 86.5 US Citizen 94.5 Mean 18.75

Non Michigan 13.5 Non Citizen 3.3 Minimum 18.02
Permanent Resident 2.2 Maximum 21.35

 

 

 

 

 

 

 

 

Table 3.9: NS with low variability.

3.4.3.3 Cell Count: It is important to select variables that are reliable and pertinent

predictors of the outcomes. Low cell count (n < 10) affects the reliability of the

38

variable(s) and render them unﬁt for realistic analysis. Ethnicity and math competency
suffered from low cell count (see Tables 3.5 and 3.7). Math competency also had high
correlation with ACT composite (Pearson correlation r = 0.560) and had missing data (14
missing values as seen in Table 3.7), which would have further reduced the effective
sample size. These two variables (ethnicity and math competency) were therefore
dropped. Gender has always been of interest in such like studies. It had acceptable cell
count (Table 3.5) and was found to be uniformly distributed in the two goups (x2 (l) =
1.399, df= l, p = 0.226).

3.4.3.4 Relevance: The backgound category variables including citizenship, resident
state, resident county, high school attended had little relevance to the research question
and were, therefore, dropped from the model. These variables could be useﬁrl for ﬁrture

studies involving such demogaphics.

3.5 The Specific Model
In the light of the above discussion, the variables selected for the model were:
DVs: ten PFEAS sub scales.
IVs: group, gender, ACT -composite (hereafter simply called ACT). The selected
IV 3 along with time shall be italicized from here onwards to distinguish the
covariates for the model.

The model with the selected variables becomes

Prei = B0 + xgroupﬁl + xgenderIBZ + xACT [£3 + g (9)

POSti = 70 + xgroqul + xgenderYZ + xACT Y3 + é- (10)

Where, Prei E Pre-DVs,

39

 

Post,- E Post-DVs,

B E Unknown pre- coefﬁcients,

y E Unknown post— coefﬁcients,

x E IVs; gender, group, ACT,

é E Post- error, and

e E Pre- error.

Before analyzing the data with the above model it was prudent to check it for

parametric test assumptions to ensure the validity of results. Any deviation from the
assumptions must be documented and further investigated before drawing conclusions.

The next section is, therefore, devoted to the parametric test assumptions.

3.6 Test Assumptions

Parametric ANCOVA is a powerful tool for analyzing data, especially if the
underlying assumptions of linearity, normality, and homoscedasticity are not violated
[60, 63]. Moreover, outliers and inﬂuential data points sometimes distort the results and
may have to be resolved [64]. To ensure that repeated measures ANCOVA accurately
summarizes the relationship between the predictors and the outcomes, SPSS© diagrostic
tools were used to check the validity of each assumption [64]. A summary is presented
below.
3.6.1 Linearity: Linearity is fundamental to multivariate statistics because solutions
are based on the general linear model (GLM) [3 7]. Linearity of the relationship between
variables was examined with two kinds of scatter plots: 1) scatter plots of raw residuals

vs. predicted values superimposed with lowess smoothing lines; and, 2) scatter plots of

40

covariate ACT vs. outcomes. No evidence of goss nonlinearity was found between the
pre- and post- measures and the predictors.

3.6.2 Normality: If there is normality, the residuals are normally and independently
distributed [3 7]. To test normality, histogams and Quantile-Quantile (Q-Q) plots of
studentized residuals were examined. All the ten models generally met the normality
assumption except for a few outliers in the data. The Central Limit Theorem also “I
supported the normality assumption because the sample size was larger than the typical
ﬁgure of 30 [65].

3.6.3 Homoscedasticity: The pattern of data spread was examined with scatter plots of

 

studentized residuals vs. predicted values. The data were found to be homogeneous
except for six out of twenty measures (ten pre- and ten post-) where evidence of
heteroscedasticity was found. Box’s tests of equality of covariance matrices supported
this pattern (Table 3.10) [66].

3.6.4 Influential Data: Highly inﬂuential data points can change the ﬁt of the model.
On examination of bubble plots of studentized residuals, four data points were found
highly inﬂuential in most of the outcomes (seventeen out of twenty measures including
the pre- and post-data) [66]. The data points were highly inﬂuential due to the large
Cook’s distance paired with large residuals and large leverage values. For example, the
bubble plot of sub-scale Pre5 math (Table 3.2) is shown in Figure 3.2 with three

inﬂuential data points, that is, 310, 410, and 412 duly highlighted.

41

 

 

 

 

m 4- Cook's
g Distance
. for Pre5
8 2— ----------------------------- a ------- o --------------- Math
E ...... . ... . . . ... . . Means
ﬁg -..".' e ' ‘00 s g ‘ .. ° ° . ° . . 0.12
:2 0— “0.0.0000. ue‘ a o s '- ‘0 o- 0.10
g “'0 I 0 an 0 ° 0 o o o .08
'5 boon-”no.0 . .- .. 006
S -2" """" serv- ------ x ---------------------------------- '
g . . . . 410 O 0 .04
"’ _ ° 310 o 0412 ° '02
-. I I I I I I ' -°°
.00 .01 .02 .03 .04 .05

Uncentered Leverage Value for Pre5 Math

Figure 3.2: Bubble plot for Pre5 math showing three inﬂuential data points.

3.7 Data Transformation

Removal of inﬂuential data points — two out of four belonged to the TS goup —
was neither justiﬁed nor recommended for a relatively small TS goup. Although
parametric AN COVA is generally robust to violations of assumptions, the data was rank-
transformed because rank transformation removes the effects of inﬂuential data, reduces
the importance of normality or homoscedasticity assumptions, and promotes robustness
and power in the analysis of covariance [60, 63, 67]. Rank transforming the outcome
variable effectively converts the AN COVA into a non-parametric procedure that no
longer assumes normality or homoscedasticity (though it has little effect on the linearity
assumption). Bubble plots of the rank-transformed data showed no inﬂuential
observations. For example, Figure 3.3 shows the bubble plot of rank-transformed sub-
scale Rank Pre5 math. Box’s tests also conﬁrmed equality of covariance matrices for all

the ten rank-transformed outcomes (Table 3.10).

42

 

 

 

 

x Cook's
: 4_ Distance
2 for Rank
é Pre5
a 2— ------------------------ O ------ o ------------- Math
3 ".5; -..:u I'T'.’ . - ° 2" g . Means
'm 2 ::.I-' . . no: a a a . . . .
3g 0- :;:.'.:.....,, , L". . 0.04
I— 3.38.8 . . . ‘ 3 ' . 2 ° .03
a _2... ........... .I- ................................. -
g - .01
- .00
:3 —4—
T l j I I I
.00 .01 .02 .03 .04 .05

Uncentered Leverage Value for Rank Pre5 Math
Figure 3.3: Bubble plot for Rank Pre5 math showing no effects of inﬂuential data.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Outcome Raw Data Rank-transformed Data
Box M F dfl/df Sig. Box M F arr/an SiL
Career 10.09 1.076 9/7265 0.377 1.162 0.124 9/7265 0.999
Society 3.831 0.408 9/7265 0.932 5.161 0.55 9/7265 0.839
Perception 13.43 1.431 9/7265 0.169 8.465 0.902 9/7265 0.523
Math 20.71 2.207 9/7265 0.019 1.384 0.147 9/7265 0.998
Exact 5.642 0.601 9/7265 0.797 3.83 0.408 9/7265 0.932
Basic 32.32 3.443 9/7265 <0.001 7.847 0.836 9/7265 0.583
Communication 5.764 0.614 9/7265 0.786 3.14 0.334 9/7265 0.964
, Groups 19.42 2.069 9/7265 0.029 10.87 1.159 9/7265 0.317
Ability 40.1 1 4.274 9/7265 <0.001 14.81 1.578 9/7265 0.1 15
Compatibility 20.79 2.215 9/7265 0.018 14.77 1.574 9/7265 0.1 17

 

 

Table 3.10: Box’s test for homoscedasticity for raw and rank-transformed data.
The transformation corrected homoscedasticity problems apparent in
the raw data for the math, basic, goups, ability and compatibility
subscales.

43

Part II — Analysis/Discussion

Based on equations 9 and 10, ten models were developed in SPSS©, one for each

of the selected PFEAS subscales. These models represent attitude subscales as a ﬁmction
of the two groups of freshmen while controlling for the confounding effects of their
gender and ACT score. This part presents the test results of ten models. Each model is
examined independently, followed by an analysis of the accumulated effect of the ten
models that form the domain of the main construct, i. e. freshman attitude towards
engineering. Two sets of data were used with each model — raw data and rank-
transformed data — to observe the test violations, if any, and to conﬁrm the reliability of
test results.

The analyses reported here are based on raw (untransformed) data, except the
outcomes that showed marginal differences and were examined with rank-transformed
data as well. Table 3.11 shows a summary of the ANCOVA results of ten models with
raw and rank-transformed data. Each model was examined in the light of the three
hypotheses by observing WSFs, and BSFs. The Estimated Marginal Means (EMS) for
the two groups were examined with respect to their proﬁle plots. Table 3.12 provides a
summary of the EMS for time *group and gender. The performances of other predictors
(gender and ACT) were also examined to understand the complete model. Tables 3.11
and 3.12 are exhaustive and shall be frequently referred to for the analysis and discussion

of the ten models presented below.

44

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Source I Raw Data I Rank-transformed Data
Outcome Hypotheses | ss Tail F [ Sig. | n2 | ss [at] F [ Sig. | 112
Time 0.25 1 1.60 0.20 0.00 22365 1 3.64 0.057 0.009
time ‘group 0.07 1 0.46 0. 49 0. 00 9184 I 1.49 0.222 0.004
WSF time‘gender 0.32 1 2.10 0.14 0.00 12506 1 2.03 0.154 0.005
Pre 1_ time‘ACl‘ 0.02 1 0.18 0.66 <00 21 115 1 3.44 0.064 0.009
Pas t 1 error (tr_r;r_e) 60.0 3 236280 38
Career Intercept 318. 1 707. <.00 0.64 830684
Group 0.55 1 1.22 0.26 0.00 30431 1 1.41 0.236 0.004
BSF Gender 0.30 1 0.68 0.40 0.00 3774 1 0.17 0.676 <.00
ACT 2.51 1 5.59 0.01 0.01 172963 1 8.01 0.005 0.020
Error 173. 3 830684 38
Time 0.06 1 0.23 0.63 0.00 5701 1 0.70 0.4 0.002
time *group 0.04 1 0.1 7 0.67 <. 00 I86 1 0. 02 0.879 0. 001
WSF time‘gender 0.10 1 0.37 0.53 0.00 11036 1 1.37 0.242 0.004
Pre3- time*ACT 0.00 l 0.01 0.91 <.00 1170 1 0.14 0.703 <.00
Pos t 3 error (tr_r_r_re) 107. 3 309734 38
Society Intercept 319. 1 444. <.00 0.53 796215 1 425. <.00 0.525
Group 1.90 1 2.65 0.10 0.00 48084 1 2.56 0.110 0.007
BSF Gender 1.06 1 1.49 0.22 0.00 26032 1 1.39 0.239 0.004
ACT 15.8 1 22.0 <.00 0.05 327624 1 17.4 <.00 0.043
Error 276. 3 720995 38
Time . 0.1 1 1 1.57 0.21 0.00 19692 1 3.03 0.082 0.008
time‘group 0.29 1 3.96 0. 04 0.01 19811 1 3.05 0.081 0. 008
WSF time‘gender 0.02 1 0.26 0.60 0.00 19.73 1 0.00 0.956 <.00
Pre4- time‘ACT 0.08 1 1.09 0.29 0.00 3897 1 0.60 0.439 0.002
Post4 error (ti_r_r_re) 29.0 3 249751 38
Peree- Intercept 328. 1 123 <.00 0.76 742303 1 341. <.00 0.470
ption group 0.21 1 0.81 0.36 0.00 1 1432 1 0.52 0.469 0.001
BSF gender <00 1 <.00 0.98 <.00 4229 1 0.19 0.659 0.001
ACT 2.04 1 7.68 0.00 0.02 120823 1 5.56 0.019 0.014
error 102. 3 836454 38
time 0.49 1 2.29 0.13 0.00 5688 1 1.20 0.273 0.003
time *group 0.16 1 0. 75 0.38 0. 00 521.18 1 0.1 1 0. 740 <. 00
WSF time'gender 0.05 1 0.24 0.62 0.00 1111.6 1 0.23 0.627 0.001
Pre5- time*ACT 0.18 1 0.87 0.35 0.00 8592 1 1.82 0.178 0.005
P0315 error (tr_me) 83.3 3 181384 38
Math Intercept 217. 1 217. <.00 0.36 537166 1 239. <.00 0.383
group 12.2 1 12.2 0.00 0.03 258674 1 1 1.5 0.001 0.029
BSF gender 0.03 1 0.03 0.84 <.00 8911 1 0.39 0.529 0.001
ACT 0.00 1 0.00 0.93 <.00 22066 1 0.98 0.322 0.003
error 385 3 863666 38
time 0.47 1 1.69 0.19 0.00 4819 1 0.76 0.382 0.002
time *group 0.58 1 2.08 0.14 0. 00 1 1592 1 1 . 84 0.1 76 0. 005
WSF time‘gender 0.75 1 2.68 0.10 0.00 11616 1 1.84 0.175 0.005
Pr e 6- time‘ACT 0.95 1 3.42 0.06 0.00 2899 1 0.46 0.498 0.001
Pos t 6 error (age) 107. 3 242615 38
Exact Intercept 268. 1 308. <.00 0.44 905051 1 456. <.00 0.542
group 4.80 1 5.51 0.01 0.01 99915 1 5.03 0.025 0.013
BSF gender 4.48 1 5.14 0.02 0.01 87800 1 4.42 0.036 0.011
ACT 12.7 1 14.6 <.00 0.03 265448 1 13.3 <.00 0.034
error 335. 3 763419 38

 

 

Table 3.11: Repeated measures ANCOVA test results for each of the ten PFEAS
subscales based on analyzing both raw and rank-transformed data.

45

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Source Raw Data I Rank-transformed Data
Outcome Hypotheses ss lail F—I Sig. | '12 | ss Idfi F rSig. | n2
time 0.10 1 1.05 0.30 0.00 813 1 0.16 0.687 <.001
wsr: time‘group 0.63 1 6.22 0.01 0.01 18143 1 3.61 0.058 0.009
time*gender 0.04 1 0.44 0.50 0.00 483.94 1 0.09 0.756 <.001
”98' time*ACT 0.16 1 1.62 0.20 0.00 7529.9 1 1.50 0.221 0.004
£038 error (time) 38.9 3 193200 38
“”0 intercept 132. 1 285 0 0.42 280019 1 134. <.00 0.258
BSF group 1.12 1 2.40 0.12 0.00 40685 1 1.95 0.163 0.005
gender 7.34 r 15.7 <.00 0.03 334881 1 16.0 <.00 0.040
ACT 8.55 1 18.3 <.00 0.04 398137 1 19.0 <.00 0.047
error 179. 3 803382458
time 0.15 1 0.99 0.31 0.00 1165 1 0.30 0.581 0.001
wsp time’group 0. 04 1 0.28 0.59 0.00 67 1 0.01 0.895 <.00
time*gender 0.16 1 1.10 0.29 0.00 2132 1 0.30 0.581 0.001
Pre9— time‘ACT 0.01 1 0.08 0.76 <.00 32.277 1 0.00 0.927 <.001
Post9 error (time) 58.7 3 147269 38
Commu- Intercept 208 1 209. <.00 0.35 630393 1 255. <.00 0.399
nication BSF group 0.54 1 0.55 0.45 0.00 21523 1 0.87 0.351 0.002
gender 0.58 1 0.58 0.44 0.00 14458 1 0.58 0.445 0.002
ACT 0.00 1 0.00 0.96 <.00 25928 1 1.04 0.306 0.003
error 382. 3 951211 38
time <00 1 <.00 0.99 <.00 6579 1 1.15 0.284 0.003
WSF time‘group 0.37 1 1.66 0.19 0.00 8944 1 1.56 0.212 0.004
time*gender 0.06 1 0.28 0.59 0.00 453.7 1 0.07 0.778 <.001
Pre, 1_ time*ACT 0.08 1 0.35 0.54 0.00 1460 1 0.25 0.614 0.001
Post“ error (timer 86.6 3 220278 38
Groups Intercept 291. 1 318. <.00 0.45 859360 1 398 <.00 0.508
BSF group 2.06 1 2.25 0.13 0.00 48962 1 2.26 0.133 0.006
gender 0.82 1 0.90 0.34 0.00 21179 1 0.98 0.323 0.003
ACT 19.9 1 21.7 <.00 0.05 548878 1 25.4 <.00 0.062
error 352. 3 831269 38
time 0.23 1 2.49 0.11 0.00 797 1 0.15 0.690 <.001
WSF time *group 0. 04 1 0. 46 0. 49 0. 00 3552 1 0. 71 0.400 0. 002
time‘gender 0.02 1 0.21 0.64 0.00 3381 1 0.67 0.412 0.002
Pre 12_ time‘ACT 0.08 1 0.88 0.34 0.00 302 1 0.06 0.806 <.001
Pom, moraine) 35.8 3 192658 38
{tbility Intercept 214. 1 459. <.00 0.54 530725 1 230. <.00 0.375
BSF group 0.23 1 0.51 0.47 0.00 348 1 0.01 0.902 <.001
gender 1.78 1 3.82 0.05 0.01 128247 1 5.57 0.019 0.014
ACT 0.35 1 0.76 0.38 0.00 8859 1 0.38 0.535 0.001
error 179. 3 885021 38
time 0.07 1 0.53 0.46 0.00 3776 1 0.68 0.408 0.002
wsp time‘group 0.41 1 3.11 0.07 0.00 12911 1 2.34 0.127 0.006
time‘gender 0.06 1 0.49 0.48 0.00 4673 1 0.84 0.357 0.002
Pre13- time‘ACI‘ 0.32 1 2.44 0.11 0.00 2693 1 0.48 0.485 0.001
Post13 error (time) 50.9 3 212003 38
Compat- Intercept 216. 1 387. <.00 0.50 525491 1 240. <.00 0.384
ibility 35]: group 0.82 1 1.47 0.22 0.00 35587 1 1.62 0.203 0.004
gender 10.1 1 18.2 <.00 0.04 360574 1 16.4 <.00 0.041
ACT 0.40 1 0.72 0.39 0.00 21772 1 0.99 0.319 0.003
error 214. 3 842348 38

 

Table 3.11: Continued.

46

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Group*Time Effect
Outcome DS TS Gender Effect
Pre-test Post-test Pre-test Post-test Males Females
Career 4.11 3.99 4.25 4.05 4.07 4.13
Society 3 .49 3.62 3.28 3.47 3.51 3.42
Perception 4.27 4.22 4.14 4.23 4.21 4.21
Math 4.09 3.91 3.57 3.49 3.77 3.77
Exact 3.19 2.91 3.38 3.31 3.30 3.10
Basic 3.80 3.75 3.55 3.71 3.83 3.57
Comm. 3.50 3.73 3.57 3.86 3.63 3.70
Groups 3.35 3.12 3.08 3.01 3.18 3.10
Ability 3.93 3.87 3.90 3.78 3.93 3.81
Compatibility 3.58 3.66 3.37 3.63 3.71 3.41

 

 

 

 

 

 

 

 

 

Table 3.12: Mean scores for ten PFEAS subscales ﬁom the ANCOVA analyses,
broken down to illustrate the group *time and gender effects.

3.8 Career (Prel-Postl)

This subscale measured the subject’s general impression of engineering as a
profession, career, and a ﬁeld of study. The subscale is comprised of nine items of the
scale as listed in Table 3.3 (lterns 1-9). Four items of this subscale (item 4, 6, 8, and 9)
were reverse coded to retain positive sense of the Likert scale. This was the biggest and
the most reliable of the PFEAS subscales.

3.8.1 Tests of WSFs: These tests check for signiﬁcant 'changes in the outcome (career)
with respect to time (pre- and post-). They include tests for the ﬂatness and the
parallelism hypotheses for time, and time *group interaction. These tests failed to reject
both hypotheses (H0321) = 0.206, and Ho,b :p = 0.497). In other words, the subscale
did not Signiﬁcantly change at the two time intervals irrespective of the group
membership; and it did not Show Si gniﬁcant time *group interaction. The two groups
were not signiﬁcantly different on their general impression of engineering over the span
of one semester; they had parallel trajectories over time, meaning no treatment effect on

this subscale. Apart from these two hypotheses, other predictors in the model, that is,

47

 

gender and ACT, did not Show signiﬁcant interaction with time. The model, as such,

explained very little of the variations as indicated by the values of Partial Eta Squared (172

S 0.5%).

3.8.2 Tests of BSFs: These tests looked for signiﬁcant changes in the outcome due to
IV s disregarding the time factor (averaging over time) and included tests for the level
hypothesis. The tests failed to reject the level hypothesis (Ho; : p = 0.269); the group E
did not inﬂuence the subscale. In other words, the two groups had similar impressions

about engineering while disregarding or averaging the time factor. In addition, the tests

 

15“- l a

showed that gender did not inﬂuence whereas ACT signiﬁcantly inﬂuenced the outcome
(ﬁpre=-0.018, ﬁpost = —0.015, p =0.019). Students with higher ACT scores had lower
impressions about engineering when controlling for gender and group afﬁliations and
disregarding the time effects.

3.8.3 EMMs: The proﬁle plots for time *group interaction indicate that while the
interaction was insigniﬁcant, the overall EMMs for both groups remained above 3.99 on
a scale of 1-5 as shown in Figure 3.4 and given in Table 3.12. This indicates that the
initial impression of engineering in both the groups was strongly positive and it did not

change signiﬁcantly over the course of one semester.

48

 

 

 

 

 

 

 

... 5.0—
c
8 ----- '13 Group
(E. 4.25 — DS Group
.E O ----------------- 4.05
M *— -----------------
ta 4 o— 4
E
N
E
'5
In
“‘ 3.0—
l I

Figure 3.4: Proﬁle plot of the time *group interaction on the career subscale. The
longitudinal trajectories from pre-test to post-test for the two groups were
nearly identical, which was why the interaction effect was not signiﬁcant.

3.9 Society (Pre3-Post3)

 

This subscale measured the freshman perception of how engineers contribute
towards the welfare of society compared to other professionals. The subscale was
comprised of two items as listed in Table 3.3 (items 11, 20) and was among the smallest
of the PFEAS subscales.

3.9.1 Tests of WSFs: The tests failed to reject the ﬂatness and the parallelism
hypotheses (”on :p = 0.631, and Ho), :11 = 0.676). The rejection of the two
hypotheses means the time and time *group interactions were not signiﬁcant for this
subscale. The two groups’ perception of how engineers contribute to the society did not
change over the course of one semester, meaning no treatment effect for this subscale
could be observed. The model also did not show interaction of other predictors as no

time *gender and time *A CT interactions were observed. As such, the model explained

very little of the variations in the subscale (172 < 0.1%).

3.9.2 Tests of BSFs: The tests failed to reject the level hypothesis (Ho; : p = 0.104);

the group did not inﬂuence the subscale. The two groups had a similar perception of how

49

engineers contribute to the society while disregarding the time factor. In addition, the

tests showed that gender did not inﬂuence whereas ACT signiﬁcantly inﬂuenced the
outcome (Bpre=-0.042, Bpost=-0.041, p < 0.001). Students with higher ACT scores had

a lower perception of how engineers contribute towards the welfare of the society when
controlling for gender and group afﬁliations and disregarding the time effects.

3.9.3 EMMs: The proﬁle plots for time *group interaction indicate that though the
interaction was insigniﬁcant, the overall EMMs for both groups remained above 3.28 on
the raw scale of 1-5 as shown in Figure 3.5 and Table 3.12. This indicates that freshman
perception of how engineers contribute towards the welfare of the society in both the
groups was mildly high in the beginning and it did not change signiﬁcantly over the

course of one semester.

 

 

 

 

 

e. 5.04
i
2 ----- TS Group
.75. — DS Group
E
a 4.0‘
E 3 49 3'62
2 . .
to k ,,,,,,,,,,,,, ...... 3.47
.E a” """""""""""
E 3.28

3.0-

l T
Pre- Po st-

Time
Figure 3.5: Proﬁle plot of the time *group interaction on the society subscale. The
longitudinal trajectories from pre-test to post-test for the two groups were
nearly identical, which was why the interaction effect was not Signiﬁcant.
3.10 Perception (Pre4-Post4)
This subscale measured the freshman perception of the work engineers do in

terms of innovation, creation, problem solving, use of technology, and the

professionalism and respect that go with it. It was comprised of seven items as listed in

50

Table 3.3 (Items 12, 17, 18 22, 25, 27, 28) and was among the most reliable of the

PF EAS subscales.

3.10.1 Tests of WSFs: The parallelism hypothesis was rejected with the raw data,
however, the corresponding test on the rank data failed to reject the parallelism
hypothesis(Ho,b :p = 0.047 for raw data, ”0.0 :p = 0.081 for rank data). The
difference between the two tests was due to the presence of a few outliers in the raw data,
and therefore the rank-transformed results were more credible. The two groups
effectively had parallel trajectories in how their perceptions of engineering work and the
engineering profession changed over the span of one semester. In other words, there was
no treatment effect on this subscale. The tests also failed to reject the ﬂatness hypothesis
for both data sets (He’d : p = 0.21 for raw data, HM : p = 0.082 for rank data)
implying there was no time effect — the trajectories were not only parallel, they were also
ﬂat. Additionally, gender and ACT did not signiﬁcantly interact with time in both the

data sets. The partial eta squared measures Show that these WSF effects explained very

little of the variance in this subscale (172 S 1% for raw data and 172 _<_ 0.1% for rank data).

3.10.2 Tests of BSFs: These tests failed to reject the level hypothesis in both data sets
(Ho; : p = 0.369 for raw data, HM : p = 0.469 for rank data) ; group had no
inﬂuence on the subscale when disregarding time effects. In addition, gender did not

inﬂuence it (p = 0.982), whereas ACT signiﬁcantly inﬂuenced the subscale in both data
sets (Bpre=-0.012, 8pm = -0.018, p=0.006 for raw data and ﬁpre=-0.099, [3pm =

-0.142, p = 0.019 for rank data). In other words, students with higher ACT scores had
lower perception about this outcome when controlling for gender and group afﬁliations

and disregarding the time effects.

51

3.10.3 EMMs: The proﬁle plots for time *group interaction for raw and rank data are
shown in Figures 3.6 and 3.7, respectively. There is only a trivial difference between the
slopes for the two trajectories in both the ﬁgures. The failure of the rank data to conﬁrm
the presence of an interaction that is detected in the raw data suggests that it is only
signiﬁcant in the raw data due to the excessive inﬂuence of a few outliers. However, the
overall EMMs for both groups remained above 4.14 on the raw scale of 1-5 and above
177 on a rank scale of 1-3 89. This indicates that the freshmen in both the groups joined
the semester with strongly positive impression of engineering and this impression did not

change signiﬁcantly over the course of one semester.

 

 

 

 

 

 

u. 5.
C
E ..... TS Group
3 4.27 4.23 _ 05 Group
.E F a
3" F 4.22
«I 4 o— 4.14
2
E
('0
E
'3:
.3
3.0—
l 1
Pre— 11m e Po st-

Figure 3.6: Proﬁle plot of the time *group interaction on the perception subscale. The
longitudinal trajectories from pre-test to post-test for the two groups differ
signiﬁcantly, but by a trivial amount that was due to the presence of few
outliers.

52

 

 

 

 

 

g 400—
$ M-TS Group
72‘ 300—- —DS Group
C
E, 209 ‘ . 210
('0 ........
E 200'— 177 ._ ................... 206
'0
.9
(U
.E 100-
E
l 1
Pre- Time Post- ‘-

Figure 3.7: Proﬁle plot of the time *group interaction for rank data on the perception
subscale. The longitudinal trajectories ﬁ'om pre-test to post-test for the two
groups were nearly identical, which was why the interaction effect was not .
signiﬁcant. !

 

3.11 Math (Pre5-Post5) 3'"
This subscale measured how much a freshman enjoyed the subjects of math and

science compared to liberal arts courses. It was comprised of two items (items 13, 19) as

listed in Table 3.3. The scale was balanced with one positive and one negative (reverse

scored) item.

3.11.1 Tests of WSFs: The tests failed to reject the ﬂatness and parallelism

hypotheses (H04 : p = 0.131, and ”0.0 :p = 0.386). The two groups were not different

in their enjoyment for math and science subjects over the span of one semester, hence, no

treatment effect was seen on this subscale. Additionally, the other two explanatory

variables, gender and ACT, did not show Signiﬁcant interaction over time. The partial eta

squared measures show that these WSF effects explained very little of the variance in this

subscale (”250.6%).

53

3.11.2 Tests of BSFs: The tests rejected the level hypothesis(Ho'c : p = 0.031);
meaning that group signiﬁcantly inﬂuenced the subscale. Other predictors in the model
namely gender and ACT, did not signiﬁcantly inﬂuence the outcome.

3.11.3 EMMs: Estimated marginal means for the time *group interactions, as shown in
Figure 3.8, indicate that the interaction was not signiﬁcant. The DS group means,
however, were signiﬁcantly higher than the TS group means. The DS group enjoyed
math and science more than the TS group compared to liberal art courses disregarding
the time effects, or averaging over time as indicated in Figure 3.9. Interestingly, the

overall marginal means remained above 3.49 on a scale of 1-5 as shown in Table 3.12.

 

Irrespective of the group distinction, freshmen enjoyed math and science subjects more

 

 

 

 

 

than liberal arts courses.

g 5.0-
§ HTS Group
Tu -- 05 Group
.5, 4.09
a 4 0— N31
2 3 S7
'8 - 3.49
a ’ '4
E
'1:
(n
“" 3.0—

l I

Pre- Post-

Time

Figure 3.8: Proﬁle plot of the time *group interaction on the math subscale. The
longitudinal trajectories from pro-test to post-test for the two groups were
nearly identical, which was why the interaction effect was not signiﬁcant.

54

 

 

 

 

In 5.0-
g Math: 15:0, 05:1
2
Tu
“Edi 4.00
g 4 0- C
.8 3.53
g Q
E
‘5
U!
“" 3.0—1

l 1

TS Group 05 Group

Figure 3.9: Proﬁle points for group marginal means on the math subscale. The DS group
enjoyed math more than the TS group compared to liberal art courses.
3.12 Exact (Pre6-Post6)

This subscale measured the freshman perception of engineering as an “exact”
science. This is deﬁned as how much they liked ﬁnding precise answers to problems vs.
open ended problems, or multiple solutions to a given problem. This subscale, like the
previous one, was asked in two different ways; hence it had two items (items 15, 26) as
given in Table 3.3. Though a small scale, it signiﬁed an important constituent of the
construct because engineering strongly encourages ﬁnding multiple or alternative
solutions to a given problem.

3.12.1 Tests of WSFs: The model showed no interaction over time for any of the
predictors. It failed to reject the ﬂatness and parallelism hypotheses (Hon : p = 0.194,
and ”0,1; : p = 0.149). The two groups were not different in their perception of
engineering as an exact science, meaning no treatment effect on this subscale. The partial

eta squared measures show that these WSF effects explained very little of the variance in
this subscale (712 S 0.9%).

3.12.2 Tests of BSFs: The model showed Si gniﬁcant contributions by all the

predictors. It rejected the level hypothesis meaning that group signiﬁcantly inﬂuenced

55

the subscale (H0,c :p = 0.019). Gender showed signiﬁcant contributions

(ﬂpre=-0.118, Bpost = -0.281, p = 0.024 for females). Also ACT showed signiﬁcant
effect on the outcome (Bpre=-0.027, Bpost = -0.047, p = 0.037). The partial eta squared
values indicated the model was able to explain some variance in the subscale (n2 _<_ 2.4%).

3.12.3 EMMs: Estimated marginal means for time *group interactions show that
although the interaction was insigniﬁcant the overall group means remained above 3.5 on
a scale of 1-5 as shown in Figure 3.10. More importantly, the TS group means remained
signiﬁcantly higher than the DS group means as also shown in Figure 3.11 and Table
3.12. It implies that the TS group perceived engineering as an exact science more than
the DS group. Also, males had signiﬁcantly higher perception of engineering as an exact

science than females as shown in Table 3.12.

 

 

 

 

 

is 5.0—
'4': ----- TS Group
E — 05 Group
I‘D
.E
E?
m 4 0"
E
'8
.. 3.38
to 3.31
.§ “ -
11? 3 0_ 3.19 2.91
1
Pre— T. m e Post-

Figure 3.10: Proﬁle plot of the time *group interaction on the exact subscale. The
longitudinal trajectories from pre-test to post-test for the two groups were
nearly identical, which was why the interaction effect was not signiﬁcant.

56

 

i

Exact: TS=0, DS=1

Estimated Marginal Means
A

 

 

 

3.35
. 3.05
3.0— °
TS Glroup 05 Group

Figure 3.11: Proﬁle points for group marginal means on the exact subscale. The TS
group perceived engineering as an exact science more than the DS

 

 

 

 

group.
5.0—
E Exact: F=0, M=1
Q)
2
Ta
.5
E 4 0—
(I
2
E
«I 3.30
g 3.10 O
‘“ 3.0— '
l T
Females Males

Figure 3.12: Proﬁle points for gender marginal means on the exact subscale.
Females perceived engineering as an exact science less than males.
3.13 Basic (Pre8-Post8)

This subscale measured the students’ conﬁdence level on basic engineering
subjects (chemistry, physics and math), engineering Skills and computer skills. The scale
is comprised of ﬁve items, one for each of its ingredients (items 29-32, 35), as listed in
Table 3.3.

3.13.] Tests of WSFs: The parallelism hypothesis was rejected with the raw data,
however, the corresponding test on the rank data failed to reject the parallelism

hypothesis (H0,b :p = 0.013 for raw data, H01, :13 = 0.058 for rank data). The

57

difference between the two tests was due to the presence of a few outliers in the raw data,
and therefore the rank-transformed results were more credible. The two groups
effectively had parallel trajectories in how their conﬁdence in basic engineering
knowledge and skills changed over the span of one semester. In other words, there was
no treatment effect on this subscale. The tests also failed to reject the ﬂatness hypothesis
for both data sets (HM :p = 0.304 for raw data, H0,a :p = 0.687 for rank data)
implying there was no time effect — the trajectories were not only parallel, they were also
ﬂat. Additionally, gender and ACT did not signiﬁcantly interact with time in both the

data sets. The partial eta squared measures show that these WSF effects explained very
little of the variance in this subscale (n2 _<_ 1.6% for raw data and 172 S 0.1% for rank data).

3.13.2 Tests of BSFs: These tests failed to reject ﬂre level hypothesis in both data sets
(Ho; : p = 0.122 for raw data, H05 : p = 0.163 for rank data); group had no
inﬂuence on the subscale when disregarding time effects. However, gender and ACT
Signiﬁcantly inﬂuenced the subscale in both the data sets (gender: p < 0.001; ACT:
Bpre=0.026, 8pm = 0.035, p<0.001 for raw data, and Bpre=0.188, 8,00“ =
0.248, p<0.001 for rank data). In other words, students with higher ACT scores had
higher conﬁdence in their engineering knowledge and skills when controlling for gender
and group afﬁliations and disregarding the time effects.
3.13.3 EMMs: Estimated marginal means for time *group interaction indicates that
though the interaction is not signiﬁcant. The overall means remain above 3.55 on a scale
of 1-5 for raw data and above 150 on a rank scale of 1-3 89 thus indicating that freshman
generally had higher conﬁdence levels in basic engineering knowledge and Skills in both

the groups (see Figures 3.13, 3.14 for raw and rank data proﬁle plots, respectively, and

58

Table 3.12). Moreover, males were found to have a signiﬁcantly higher conﬁdence level
in the basic engineering knowledge and skills than females when controlling for group
and ACT and disregarding time effects, as shown in Figures 3.15 and 3.16 for raw and

rank data, respectively, and Table 3.12.

 

 

 

 

 

g 5.0-
% ..... TS Group .
_ — DS Group
I"
.5
2°
to 4.0"
5 3 3° % ”"4. 3 71
E. 3 55 o ----------------------- 3 75
N
.E
a
“4 3.0-
l I

 

Pre- Time Post-

Figure 3.13: Proﬁle plot of the time *group interaction on the basic subscale. The
longitudinal trajectories from pre-test to post-test for the two groups
differ signiﬁcantly, but by a trivial amount that was due to the
presence of few outliers.

 

 

 

 

 

400-
In
5 "”15 Group
g _ DS grol'p
.E 300—
no
3
E
B 200— 194 184
g r a 1,,
"3 rso gum-3......"
“J
100 I I
Pre. Post-

Time

Figure 3.14: Proﬁle plot of the time *group interaction for rank data on the basic
subscale. The longitudinal trajectories from pre-test to post-test for
the two groups were nearly identical, which was why the interaction

effect was not signiﬁcant.

59

 

 

 

 

 

 

 

 

 

g 5'0— Basic: F=0. M=1

8

2

a

.E

9 4 0— 3 83

g .-

1: 3.58

8 O

G

E

'5

“‘ 3.0—

T l
Females Males E
Figure 3.15: Proﬁle points for gender marginal means on the basic subscale. Males had

higher conﬁdence level in basic engineering knowledge and skills than
females.

in 40

g 01 Basic: F=0, M=1

d)

E

_g 300-4

E

N

E 207

{-3-}, 200‘ 153 .

.5 .

100 I I
Females Males

Figure 3.16: Proﬁle points for gender marginal means for rank data on the basic
subscale. Males had higher conﬁdence level in basic engineering
knowledge and skills than females.

3.14 Communication (Pre9-Post9)

This subscale measures the student conﬁdence level in their communication skills
for three categories: writing, speaking and computer usage. The scale is comprised of
three items, one for each of the categories as listed in Table 3.3.

3.14.1 Tests of WSFs: The model showed no signiﬁcant interaction of any of the

predictors over time. It failed to reject the ﬂatness and parallelism hypotheses (HM : p =

0.318, and Hm, : p = 0.593). The two groups were not different in their conﬁdence

6O

levels on communication skills, hence, no treatment effect was noticed on this subscale.
Additionally, gender and ACT did not signiﬁcantly interact with time. The partial eta
squared measures show that these WSF effects explained very little of the variance in this
subscale (n2 s 0.3%).

3.14.2 Tests of BSFs: These tests failed to reject the level hypothesis (Hog : p =
0.459); group had no inﬂuence on the subscale when disregarding time effects. In
addition, gender and ACT did not inﬂuence the subscale. The model explained very little
of the variance in the outcome (772 S 0.2%).

3.14.3 EMMs: Estimated marginal means for time *group interaction indicates that
though the interaction was not signiﬁcant, the overall means remained above 3.5 on a
scale of 1-5 as shown in Figure 3.17. This indicates that initial conﬁdence in

communication skills was positive and it remained positive over the semester for both

 

 

 

 

 

groups.
is 5.
N
g «~- TS Group
7, — DS Group
.5
no
ta 4.o~ 3-86
2
E 3.57
g 3.50
'8
“4 3.0—
l l
Pre- Post-

Time
Figure 3.17: Proﬁle plot of the time *group interaction on the communication subscale.
The longitudinal trajectories from pre-test to post-test for the two groups
were nearly identical, which was why the interaction effect was not
signiﬁcant.

61

3.15 Group (Prell-Postl 1)

This subscale measured the student enjoyment of working in a group. It was one
of the important subscales of engineering attitudes because a majority of engineering
work involves group activity. The scale was comprised of three items (items 37, 43, 45)
as listed in Table 3.3. Two of the items were reverse coded to retain a positive sense of
the Likert scale. l"

3.15.1 Tests of WSFs: The tests failed to reject the ﬂatness and parallelism

"2} "l‘k”. _l

hypotheses (HM :p = 0.992, and Hub :1? = 0.198). The two groups were not different

in their perception about group work over the span of one semester. Hence, no treatment

 

 

effect was observed on this subscale. Additionally, gender and ACT, did not show

signiﬁcant interaction over time. The partial eta squared measures show that these WSF
effects explained very little of the variance in this subscale (212 S 0.4%).

3.15.2 Tests of BSFs: The model was unable to reject the level hypothesis (H0; :1) =
0.134); meaning that group did not signiﬁcantly inﬂuence the subscale. Gender did not
inﬂuence whereas ACT signiﬁcantly inﬂuenced the subscale (ﬂpre=-0.043, Bpost =
-0.049, p<0.001). In other words, students with higher ACT scores had lower
preference of working in groups when controlling for gender and group afﬁliations and
disregarding the time effects. This was due to the ACT, that the model was able to explain
some variance in the outcome as shown by the value of partial eta squared (172 S 5.3%).
3.15.3 EMMs: Estimated marginal means for time *group interaction indicates that
though the interaction was not signiﬁcant, the overall means remained above 3.10 on a

scale of 1-5 thus indicating that freshmen had neutral impression about the group activity

(see Figure 3.18 and Table 3.12).

62

 

 

 

 

 

g 5.0‘
g ----- TS Group
E -— DS Group
.E
9
m 4 0-
2
8
‘g 3.35
. 3.12
E 10... 3'08 :\‘ 3.01
l l
Pre- Post—

Figure 3.18: Proﬁle plot of the time *group intergtteion on the group subscale. The
longitudinal trajectories from pre-test to post-test for the two groups were
nearly identical, which was why the interaction effect was not signiﬁcant.

3.16 Ability (PrelZ-PostlZ)

This subscale measured the students’ problem solving abilities through creative
thinking, conﬁdence in success, and enjoyment of solving open ended problems. The
scale was comprised of ﬁve items (Items 38, 40, 42, 49, 50) and was one of the important
subscales of the main construct (Table 3.3).

3.16.1 Tests of WSFs: The model showed no signiﬁcant interaction of any of the

predictors over time. It failed to reject the ﬂatness and parallelism hypotheses (Ho,a : p =

0.115, and H0,b : p = 0.497). The two groups were not different in their perception of

problem solving abilities. Hence, no treatment effect was observed on this subscale. The

model as such explained very little of the variations in the outcome.

3.16.2 Tests of BSFs: The model was unable to reject the level hypothesis (H0531? =

0.475) meaning that group did not signiﬁcantly inﬂuence the subscale. In addition,

gender signiﬁcantly inﬂuenced (p = 0.05) and ACT did not inﬂuence the subscale. The

model did not explain much of the variance in the subscale (772 S 1%).

63

3.16.3 EMMs: Estimated marginal means for time *group interaction indicates that
though the interaction was not signiﬁcant, the overall means remained above 3.8 on a
scale of 1-5 thus indicating that freshmen had a positive impression of their problem
solving abilities (see Figure 3.19 and Table 3.12). Moreover, males had signiﬁcantly

higher perception of problem solving abilities than females as shown in Figure 3.20 and

 

 

 

 

 

 

 

 

 

Table 3.12.
2 5.0—
8 ----TS Group
E —DS Group
N
.5
3" 3.93
a 4 0— 3.87
5 3 90 f» T.
B 3.78
‘5
,E
a
“4 3.0!
l I

Figure 3.19: Proﬁle plot of the time *group interaction on the ability subscale. The
longitudinal trajectories from pre-test to post-test for the two groups were
nearly identical, which was why the interaction effect was not signiﬁcant.

 

 

 

 

2 5'0‘ Ability: r=o, M=1

8

2

E

.‘5

an 3.93

'2' 40" 3.81 g
0

'0

d:

‘5

_E

E

3.0-
l l
Females Males

Figure 3.20: Proﬁle points for gender marginal means on the ability subscale. Males
had higher conﬁdence of problem solving abilities than females.

64

3.17 Compatibility (Pre13-Postl3)

This subscale measured the students’ engineering abilities through conﬁdence in
engineering knowledge, design capabilities and inclination towards mechanical and
technical abilities. The subscale was comprised of four items (Items 36, 44, 47, 48) as
listed in Table 3.3. This was also considered an important subscale of the main construct.
3.17.1 Tests of WSFs: The model showed no signiﬁcant interaction of any of the
predictors over time. It failed to reject the ﬂatness and parallelism hypotheses ("ma :1) =
0.465, and Ho), : p = 0.078). The two groups were not different in their perception of
engineering abilities hence, no treatment effect was observed on this subscale. The model

as such explained very little of the variance in the outcome (r12 5 0.8%).

3.17.2 Tests of BSFs: The model was unable to reject the level hypothesis meaning
thereby that group did not signiﬁcantly inﬂuence the subscale. ACT also did not

inﬂuence the model but gender signiﬁcantly affected the outcome (p < 0.001).The model
explained some variance due to gender as noted by the values of eta squared (r72 5 4.5%).

3.17.3 EMMs: Estimated marginal means for time *group interaction indicates that
though the interaction was not signiﬁcant, the overall means remained above 3.37 on a
scale of 1-5 thus indicating that freshmen had a mildly positive impression about their
engineering abilities in both the groups (see Figure 3.21 and Table 3.12). Moreover, the
EMMs indicate that males had signiﬁcantly higher engineering abilities than females as

shown in Figure 3.22 and Table 3.12.

65

Figure 3.21:

Figure 3.22: Proﬁle points for gender marginal means on the compatibility subscale.

3.18

 

"i

 

 

 

 

U!
C
8 ----- TS Group
E — DS Group
N
.5
3°
to 4 0-
2 3 58 3.66
a -------------------- 3.63
E 9"”
u: 3.37
II!
"J 3.0—
l l
Pre- Time Post-

Proﬁle plot of the time *group interaction on the compatibility subscale.
The longitudinal trajectories from pre-test to post-test for the two groups
were nearly identical, which was why the interaction effect was not

 

 

 

 

signiﬁcant.

g 5'0— Compatibility: F=0, M=1
d)

E

To

.E

3°

g 4-0‘ 3.71
E 3.41 .

N

.E ‘

E

3.0“
I I
Females Males

Males had higher engineering abilities than females.

Summary of Analysis

A summary of the analyses on the ten repeated measures ANCOVA models is

presented below [68-70].

The parallelism hypothesis was not rejected in any of the ten outcomes; the time *
group interactions were not statistically signiﬁcant for any of the subscales. The
signiﬁcance noted in the raw data for two of the subscales — perception and basic
— was due to four highly inﬂuential data points and therefore disregarded in favor

of the rank-transformed results. There was no treatment effect on the DS group in

66

 

terms of changes in freshman attitudes towards engineering compared to the TS
group.

The ﬂatness hypothesis was not rejected in any of the ten outcomes; the student
attitudes were not affected over time (pre-, post-) when disregarding the group

membership. The partial eta squared measures show that the WSF effects

explained very little of the variance in all the models (42 S 2%).

The EMS for all the ten outcomes showed positive initial attitudes and these
attitudes did not change during the course of one semester. The attitudes were
measured on a raw scale of 1-5 (with scores of 1-2 meaning negative perceptions,
3 meaning neutral perceptions, and scores of 4—5 meaning positive perceptions).
Career and perception, the most reliable PFEAS subscales, were strongly positive
(M > 4.0). Freshmen had joined the program with strong perceptions about
engineering in relation to innovation, creation, problem solving, and
professionalism and respect for the career. Math, basic, communication, ability,
and compatibility were positive (3.5 < M > 4.0), and society, exact, and group
were mildly positive (3.0 < M > 3.5). The mildly positive subscales were either
those that required the knowledge participants were not exposed to, e. g. working
in groups, ﬁnding multiple solutions to a problem, or the subscales were low in
reliability because of only two items. Notwithstanding the reason, none of the
scales were negative. Most scales were positive in the beginning of the course and
did not change during the semester.

The level hypothesis was rejected in two subscales: math and exact. The DS

group had higher EMM for math and lower EMM for exact compared to the TS

67

group. This means that DS group enjoyed math and science subjects more and
was less likely to believe that engineering was an exact science. The signiﬁcantly
more positive attitudes in the DS group could be due to their stronger background
in math and science (the DS group must meet a higher math requirement imposed
by the CoE), higher ACT scores, and better understanding that engineering was
not an exact science.

In addition to the three hypotheses, the models were examined for two covariates,

gender and ACT, for their response in relation to other similar studies. Some interesting

ﬁndings are outlined below.

Deﬁnite gender differences were found in four of the subscales. Females rated
lower than males in all the four attitude measures: basic; ability; compatibility;
and exact. In other words females had lower conﬁdence levels in basic
engineering knowledge and skills, problem solving abilities, and engineering
abilities. These ﬁndings are consistent with the literature in the area [11].
Interestingly, females perceived engineering as being an exact science less than
their male counterparts. This means males liked ﬁnding precise answers to
problems more than females. In other words, females showed a better inclination
towards ﬁnding solutions to open ended problems and are, perhaps, more akin to
engineering.

The covariate, ACT signiﬁcantly affected six of the subscales in both data sets:
career, society, perception, exact, basic, and group. ACT was also related to the

explanatory variable group — the DS group had higher ACT scores than the TS

68

group - which was why it was important to adjust for this covariate in the models.

The literature also supports the strength of this predictor [9, 71].

3.19 Discussion

This part of the study was aimed at ﬁnding an answer to the primary research
question whether the new cornerstone sequence was more effective than the older
traditional sequence at positively inﬂuencing ﬁeshman attitudes about engineering over
the course of one semester. A comparison of freshmen in the new DS group to ﬁ'eshmen
in the previous TS group was done by collecting attitude data twice over the course of a
semester. The data was examined for changes in the two groups’ attitudes with repeated
measures ANCOVA models. It was found that freshmen join the program with positive
(EMMs > 3.00) and strongly positive (EMMs > 4.00) attitudes toward engineering that
could be resistant to change. Students in the DS had higher ACT scores, enjoyed math
and science more, and did not believe engineering to be an exact science. Some
interesting results were found in how gender and ACT performed as covariates in the
model. The DS group, however, had a similar longitudinal trajectory to the TS group, so
there was no evidence of differential inﬂuence on student attitudes. In other words there
was no treatment effect in the DS group.

The lack of treatment effect could be because of one or all of the factors given
below that could have affected the models’ ability (power) to detect differences.

0 The DS was a set of newly designed courses that would certainly require a break-

in period and constant feedback to reach a level of maturity before this could fully

69

 

meet the course objectives. It may not be realistic to expect a new course to meet
all of its goals and objectives at the outset.

The short time between pre- and post- surveys - eleven weeks of experience —
may simply not be enough to affect an appreciable change in the two groups’
attitudes.

The study only looked at one of the two DS courses that were designed for
sequential treatment over the ﬁeshman year. The compound effect of taking the
two courses in the intended sequence could be signiﬁcantly larger than that of
taking only the ﬁrst part alone.

The different data collection methods and their associated incentives for the TS
and DS groups could have contributed to large differences in participation rates
due to selection bias. There may have been a selection bias for higher achieving
or more motivated students to complete the online surveys, which may have
skewed the data for TS group.

PFEAS subscales are being revised to further improve the internal reliability and
structural validity of the scale [51]. The revised scale, whenever available, could
improve the models’ ability to detect intervention effects.

The structure of the two groups was not based on the random selection. The TS
group was comprised of students who could not be admitted to the DS courses for
a number of reasons, and therefore could be skewed. There could be some
strongly motivated students in the TS group who could not get admitted due to

the shortage of classroom space or lack of pre requisites for the DS courses.

70

Based on the above, it was prudent to test the results of this phase with some
other method that could validate the quantitative results, especially the lack of treatment
effect. Qualitative methods are one way to probe deeper into the quantitative results, to
better understand the construct, and bring out the subtle changes that are not possible
with a quantitative approach [72]. A one-on-one interview with a representative sample,
a qualitative approach, was adopted with further collection of data and its analysis [3 8].

The next chapter is devoted to the qualitative approach.

71

CHAPTER 4
THE QUALITATIVE PHASE

The most important ﬁndings of the quantitative phase of this study were that the
tests of within subject factors (W SFs) had not been signiﬁcant in any of the ten PFEAS
sub-scales that collectively deﬁned the main construct of the primary research question,
namely, freshman attitudes towards engineering. These tests failed to reject the ﬂatness
and parallelism hypotheses for all the ten models. This implied that neither the new
cornerstone sequence nor the older traditional sequence signiﬁcantly affected the
freshman attitudes over the course of a semester. It was hypothesized in the quantitative
phase that perhaps a time frame of one semester of engineering experience (eleven
weeks, to be precise - time between pre- and post- surveys) may not be enough to affect
an appreciable change in freshman attitudes towards engineering. To probe deeper into
the results of the quantitative phase, especially the lack of the treatment effect, qualitative
methods are more suitable because they allow for the generation of rich, contextual
description of data, also called a “thick” description [36].

As many researchers have pointed out, rigorous qualitative research involves its
own set of data collection and analysis methods to ensure trustworthiness of the ﬁndings
[73]. Qualitative methodology could generate rich and detailed descriptions of the
student expectations and experiences for a deeper understanding of their attitudes [74]. It
was therefore worthwhile to examine the student experiences and their expectations of
the new sequence using qualitative methods to seek a possible relationship and to
validate the earlier results. For this part of the study, the secondary research question was

formulated (in Chapter 1): What were the freshman expectations of and experiences in

72

the new cornerstone sequence and how did they contribute (or not contribute) towards
shaping their initial perceptions and attitudes about engineering?

This chapter presents an investigation of the secondary research question by
probing a sample of the new cornerstone sequence population with a qualitative
approach. It has two key objectives: to describe the methodology of the approach
(instrument development, data collection, reduction and display, conclusions and
veriﬁcation) and analysis of the data. The chapter is accordingly divided into two parts:

Methods and Data Analysis/Discussion.

Part I - Methods

4.1 The Context

When the quantitative results are inadequate to provide explanation of outcomes,
the problem may be better understood by using qualitative data to enrich and explain the
quantitative results. In the quantitative phase we were unable to reject the parallelism
hypothesis implying no-treatment—effect on the DS group insofar as ﬁ'eshman attitude
towards engineering was concerned. A need was felt for further interpretation of the
quantitative results, perhaps with more detailed views of select participants. For such
situations mixed methods design is considered the preferred approach [75]. g

The qualitative approach for this study involved one-on-one interviews with a
sample of freshman volunteers ﬁ'om the DS group. This part of the study was conducted
in spring 2009, after the completion of the quantitative phase in fall 2008. A majority of
the freshmen who took EGRIOO in fall 2008, and hence would have participated in the

quantitative phase, registered for EGR102 in spring 2009 (computer engineering and

73

computer science majors take CSE 231 instead of EGR102). It was, therefore, a good
opportunity to sample the same cohort of students from the DS group using one-on-one
interviews for a deeper understanding of their attitudes. The participants were asked a
number of open-ended questions regarding their experiences in the cornerstone sequence
and whether these experiences affected their perceptions about engineering. It is pertinent
to mention that this part of the study did not require data ﬁ'om the TS group because the
secondary research question did not address it. The qualitative study is equivalent to
testing of the ﬂatness hypothesis in a quantitative study.

When considering the responses of the student participants, two factors are
important. First, the student participants (freshmen) were young adults between 18-20
years of age, they were interviewed in a relaxed atmosphere, and they were assured that
their views would be kept conﬁdential and would not affect their grades. Second, an
understanding of the cornerstone courses provides the context of the study, especially
because many students commented on the peripheral details of these courses. It may,
therefore, be worthwhile to revisit the cornerstone sequence before delving into the

qualitative data.

4.2 Cornerstone Sequence - A Revisit

4.2.1 EGR 100: EGR 100 introduced the discipline of engineering to freshmen in their
very ﬁrst semester at MSU. These students had presumably joined the engineering
program with varied and mixed initial perceptions. As it provides the ﬁrst impression to
new engineering students, it is argued that EGR 100 could play an important role in

shaping their initial perceptions about engineering. The purpose of EGR 100 was to

74

engage the freshmen with the engineering profession. At the end of this two credit course
the students were expected to be able to: 1) implement an organized methodology in
solving new and unfamiliar engineering design problems and effectively communicate
the solutions to others; 2) function effectively on cross-ﬁmctional design teams; and 3)
understand the unique aspects of the engineering profession, its signiﬁcance to society,
its ethical framework, and career opportunities. The course was delivered in lecture-cum-
recitation format (one SO-minute lecture followed by one llO-minute lab session per
week). Lectures were delivered in a large lecture hall by the course instructor and subject
specialists from different engineering disciplines. There were weekly clicker quizzes and
lab assignments. No textbook was speciﬁed for the course.

The lab sessions were conducted by graduate teaching assistants (TAs). The
laboratories were comprised of three team-based design projects conducted in groups of
4-5 students. The aim of these projects was to introduce the methodology for solving
simple engineering problems in an integrated team setting without commitment to a
speciﬁc engineering discipline. Several projects were experimented during the two pilot
offerings of EGR 100 in fall 2007 and spring 2008. The projects for fall 2008 were
selected on the basis of their relevance to the course objectives, positive student feedback
ﬁom the pilot runs, and faculty discussions and recommendations. These projects were:
1) design, construct and test an edible car that meets project requirements, that is, to
travel down a given track while minimizing the elapsed time; 2) determine the most cost-
effective, eco-ﬁiendly means for a team to travel to a meeting site while minimizing total
trip cost, carbon emissions, and elapsed time; and 3) design and prototype a product or

process to remove a frequent inconvenience in your daily life.

75

4.2.2 EGR 102: EGR 102 was focused on the use of computational tools to solve
engineering problems. The competencies the students were expected to demonstrate at
the completion of this two credit course were: 1) strategy to solve engineering problems;
2) write programs to model systems using the MATLAB environment; and 3) portray
two and three dimensional data in a meaningful manner, draw logical conclusions, and
make appropriate recommendations using the MATLAB. The course was delivered in
lecture-cmn-recitation format (one SO-minute lecture followed by two 80-minute lab
sessions per week). Unlike EGR 100, its weekly laboratories consisted of mostly
MATLAB oriented individual assignments. It had one group project towards the end of
the semester. The project involved a team of four students tasked with developing a
MATLAB model for a water system to determine the necessity of a secondary water
supply. EGR 102 was a fast paced course with weekly lecture and lab assignments as
well as class quizzes. It required a strong mathematical base to fully understand the

course content. No textbook was prescribed for this course.

4.3 Development of Interview Protocol

Qualitative research is characterized by two aspects: instrumentation, and context
of the research question. Instrumentation involves development of qualitative instrument
(surveys, interviews, focus group protocols etc.), and collection and examination of
textual data. One-on-one interviews were the most suitable method for collecting
qualitative data for this study because: 1) interviews can give the most detailed feedback;

2) interviews are ﬂexible because they allow researchers to gauge initial reactions and

76

probe for more in-depth responses; and 3) in a private encounter people tend to be more
willing to speak their minds than they are in group settings.

The ﬁrst step in the qualitative phase of the study was developing an instrument
or the interview protocol. As mentioned earlier, Engineering Education is relatively new;
it is rapidly developing to become an independent discipline of engineering [36, 76].
There is, understandably, a scarcity of reliable instruments especially in the realm of
qualitative research because engineers do not often use qualitative methods [36]. There
were no existing protocols for this kind of study in the literature. To economize on the

efforts and time required for a valid and reliable instrument, the interview protocol was

written in collaboration with the EEES research group3 working on student expectations
and experiences in the freshman year.

The interview protocol was developed in two parts in a semi-structured format to
allow for ﬂexibility in the order and precise content depending on the participants’
responses [3 8, 39]. Part 1, comprising seven items, addressed the general freshman
experiences and expectations. This part was developed by the EEES research group; it
focused on issues such as academic and social integration, motivation to pursue degree,
classroom experiences and faculty integrations. Part I broadly covered the PF EAS
subscales, and could be mapped on to the quantitative part of this study. The data from
Part I was categorized into three subcategories: engineering perceptions, engineering

abilities, and basic skills. This part prepared the participants to focus on their initial

 

3 The BEES (Engaging Early Engineering Students to Expand Numbers of Degree Recipients) research
group was engaged in a pilot study to collect preliminary data in Spring 2009. This group works under the
Center for Engineering Education Research (CEEE) that was established at MSU in August 2009.

77

perceptions and relate them to their experiences in the ﬁrst two semesters of the
engineering program.

Part II of the protocol was written by this researcher. It was speciﬁc to the
cornerstone sequence and addressed the secondary research question. It focused on
freshman experiences and expectations in the cornerstone sequence and how those
experiences and expectations changed or did not change students’ initial perceptions and
attitudes towards engineering. This part of the protocol was comprised of six items that
inquired into student experiences in EGRIOO and EGR102 with regard to content
learning and conduct of the courses, and their effects on participants’ initial perceptions
and attitudes towards engineering. For example, it asked the participants to: 1) describe
their experiences in the DS courses; 2) compare their learning in the DS courses with
learning in other courses; 3) how their thinking about themselves changed as a result of
the DS courses; and 4) how their perception about engineering changed since they joined
the DS courses. Two subcategories emerged ﬁom this part: DS experiences and DS
effects on perceptions. Probing questions were used to prompt participants to explore
their experiences in the DS courses and to keep them focused on how those experiences
affected their initial perceptions about engineering.

To ensure good validity and reliability, the protocol was reviewed, pilot tested,
and ﬁnally used on one common sample for data collection. It was reviewed by four
experts from two colleges, the College of Education and the College of Engineering, for
content, selection of probes, sequence of questions, and wording of questions that may
resonate better with undergraduate freshmen. It was then tested on three engineering

graduate students to conﬁrm that the item statements were clear, concise, and conveyed

78

what they were meant to convey without confusion or ambiguity. The protocol was pilot
tested on a sophomore in the College of Engineering who had recently experienced the
ﬁrst-year transition into engineering. This student provided feedback on both form and
content and suggested additional probes based on his experience. The instrument and its
associated consent form were approved by the university’s IRB before they were used for

this study in spring 2009. The interview protocol is attached as Appendix C [77].

4.4 The Sample

A total of 533 freshmen were registered for spring 2009. All of these freshmen,
except l4 transfer cases and 8 ﬁrst semester students, registered for one or both of the DS
courses. In other words, the TS group was almost nonexistent in the population. This was
not a disadvantage because this part of the study required data from the DS group only —
the secondary research question addressed the experiences and expectations of the DS
group. The population for this part of the study was comprised of 272 ﬁeshmen — those
who had experienced EGR 100 in fall 2008 and were enrolled for EGR 102 in spring
2009. It is important to note that a portion of the population that opted for computer
engineering or computer science majors did not take EGR 102 and was therefore not a
part of the population for the qualitative phase. The population was approached through a
recruitment e-mail to participate in one-on-one interviews and earn $25 as compensation.
A good-to-moderate response was expected (20~25%). The intent was to form a stratiﬁed
sample of approximately 10-15 students from various demographic backgrounds and
levels of academic achievement. Unfortunately, fewer students responded than expected.

Ten students volunteered as a result of six e-mail attempts to 113 students. All ten

79

respondents were interviewed towards the end of the semester to ensure they had
sufficient exposure to EGR102. Table 4.1 presents the breakdown of the sample’s

characteristics, demographics, and their pseudonyms.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Student Ethnicity* Gender ACT Declared EGR100 EGR100 Current
ID Score Engineeriﬂ Major Term Grade Semester
Sara African Female 19.00 Electrical F808 3.0 8809
Lora Caucasian Female 30.00 Bio-systems F 808 3.0 8809
Erin Caucasian Female 23.00 Civil F808 3.5 8809
Jane Caucasian Female 23.00 Bio-systems F808 4.0 8809
Pat Caucasian Male 25.00 Civil F808 3.5 8809
Matt African Male 17.00 Electrical F808 3.5 8809
Don Caucasian Male 25.00 Mechanical F 808 4.0 8809
Brad Caucasian Male 34.00 Mechanical F808 4.0 8809
Mike Caucasian Male 28.00 Applied. Sciences F808 4.0 8809
Jon Aﬁican’r Male 20.00 Electrical F808 4.0 8809
‘Ethnic designation based on self-identiﬁcation. [Not African American.

Table 4.1: The sample characteristics showing pseudonyms, demographics and
academic perforrnmance.
The demographics of the sample were not fully representative of the College

student population in terms of ethnicity and gender. The fall 2008 statistics for the

College4 was not truly reﬂected in the sample insofar as the ethnic minorities were
concerned. This is not sm'prising given the sample size. The majority ethnic group,
Caucasian, was adequately represented but minority groups such as Hispanics and Asians
were not represented at all. It may be noted that three participants self identiﬁed as
minorities. The characteristics of the sample roughly matched the target population in
terms of ACT scores and EGR 100 course grades. The participants’ gender distribution

was skewed (40% females) since females actually made up 20% of the target population.

 

’ Caucasians: 77.8%, African Americans: 5.8%, Asians: 5.4%, Hispanic: 1.8%, American Indians: 0.6%,
not reported: 8.5%, Females: 16.9%, Males: 83.1%, ACT lowest 25m percentile: < 23, ACT middle 50"“
percentile: 23-27, ACT highest 25th percentile: > 27 — Ofﬁce of the Associate Dean, Sep 9, 2008.

80

Over-sampling here helped to examine possible diversity of views among female
students because success of women in engineering is an important issue [4, 78]. Table 4.2
presents a comparison of the characteristics of the participants in the sample with the
targeted freshman population in terms of their EGR 100 course grades, ACT-composite

score, and gender.

 

EGR 100 GPA Fall 08 ACT-Composite Scores* Gender*

 

 

 

 

 

 

 

 

 

G N/
”up n s 3.0 > 3.0 Lowest Middle 50% Highest 25% (Females)
Population 272 14% 85% 13% 53% 34% 20%
Sample 10 20% 80% 30% 40% 30% 40%

 

 

 

 

"' see footnote 4 for CoB statistics for fall 2008.

Table 4.2: Characteristics of the sample vs. the population.

An interesting aspect of the sample was that all the ten participants belonged to
Category-1, engineering-preference (see Chapter 1); they could remain in the program
for two years to take this decision. It points towards their strong preference for the
engineering major. The demographic peculiarities of the sample and their relationship
with the target population were important and kept in view while drawing conclusions

ﬁom the results.

4.5 Data Collection, Reduction, and Display

The interviews with the participants were conducted one-on-one by the researcher
from the BEES research group, and observed by this researcher, during the 13th-15th
weeks of the semester. The interviews took 60-75 minutes per participant. The purpose of
the research was explained to the interviewees and their written consent was obtained
before each interview in accordance with the IRB regulations. These interviews were

audio recorded in addition to taking of ﬁeld notes by the two researchers. At the

81

conclusion of each interview session, the two researchers reviewed, elaborated, and
consolidated the ﬁeld notes to ensure the data were complete. The interviews were
transcribed by a professional transcriptionist. The transcriptions were substantiated with
the consolidated ﬁeld notes of the interviewer and the observer.

The raw data was simpliﬁed, reduced, and transformed by coding statements into
categories for various themes [72]. The raw transcribed ﬁle of the ten participants was
large. It had redundancies, repetitions and errors/omissions due to voice/dialect
recognition issues. The redundancies, repetitions, errors/omissions were carefully
examined in the context of the subject and removed/corrected with the mutual consent of
the two researchers. Bach transcript was studied in depth to identify the relevant quotes
and to group them into the selected themes in case of Part I, and emergent themes in case
of Part 11. Following the coding process, the ideas were grouped into themes,
subcategories and groups and ﬁnally converted into a matrix of key themes. The data was
then revisited to note the presence or absence of these themes in each of the interviews
and coded for each theme as ‘+’ (positive response), ‘-’ (negative response), ‘0’ (neutral
response), and ‘blank’ (absence of a response) [38, 72]. In this manner the data was
reduced to the display in Table 4.3 and converted into a primary data ﬁle for further
analysis. The qualitative display was intended to present the data in an organized,
compressed format that could permit illustration of certain patterns and help in forming
logical and key conclusions [3 8]. The data reduction and display was independently
developed by the two researchers, and was later compared, veriﬁed and validated for

inter-coder reliability check before drawing conclusions [79].

82

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Interview Protocol Part I Interview Protocol Part 11
Student - - . . DS Experiences
n) “”2““ Egg?“ Ba?" EGR100 EGR102 pus Effms
Perceptions ties Skills A B C 1) B C D Perceptions

Sara + + 1 1 - _ _ 0 _ _f 0 T
Lora + + - .. _ + +

Erin O + - + + + - 0

Jane + 0 0 0 + 0 0 0

Pat + + 0 - - - + - + 0

Matt + + + 0 + - - - + 0

Don + + 0 0 - - - - 0

Brad + + + - - - - + 0 + 0

Mike + + + 0 - + + +

Jon + + + + - 0

 

 

 

 

 

 

 

 

 

 

 

 

 

" A-Projects experiences, B-Class experiences, C-Time per credit, D-Conduct of course,
1' ‘+ ’ Positive response, ‘-’ Negative response, ‘0’ Neutral response, ‘blank’ No response

 

 

Table 4.3: Qualitative data display showing presence or absence of themes for each
participant.

Before analyzing the qualitative data, a brief explanation of various themes,
subcategories and their relevance to the research question is given below.
4.5.1 Part I - General Engineering Perceptions: The data in this part broadly
covered the PFEAS subscales and was grouped into three subcategories:
4.5.1.1 Engineering Perceptions: This category was broad in meaning; it
encompassed several themes to include participants’ perceptions about engineering as a
discipline, major, career, job, and profession. It also included their perception of
engineering as an art, engineering as a science, its impact on society, and parent] teacher
inﬂuences on the formation of such perceptions. With regard to the PFEAS sub-scales,
the category included themes that broadly covered six of the thirteen sub-scales: sub-
scale 1, 2, 3, 4, 6, and 7 as given in Table 3.1.
4.5.1.2 Engineering Abilities: This category included themes on the participants’

conﬁdence in problem solving abilities, technical inclination, designing abilities, ability

83

to work in groups, and their study habits. In terms of PFEAS sub-scales, the themes
broadly covered four of the thirteen sub-scales: sub-scale 10, ll, 12, and 13 as shown in
Table 3.1.

4.5.1.3 Basic Skills: This category included themes on the participants’ science skills
including math, physics, chemistry, and computers, and communication skills including
writing and speaking. In terms of PF EAS sub-scales, the themes broadly covered three of
the thirteen sub-scales: sub-scale 5, 8, and 9 as shown in Table 3.1.

4.5.2 Part II - DS Related Perceptions: Part II of the protocol was speciﬁc to the
cornerstone sequence. It was divided into two subcategories: DS experiences and DS
effects on perceptions.

4.5.2.1 DS Experiences: Participants talked about their expectations of and
experiences in the two courses in four themes: 1) Project experiences in three group
project activities during lab sessions; 2) Class experiences in lectures and Powerpoint
presentations; 3) Time per credit expended by the students; and 4) Conduct of the
courses. The data for this subcategory was primarily collected with the ﬁrst four items of
Part 11.

4.5.2.2 DS Effects on Perceptions: This subcategory was the most important with
regard to the research question. It focused on one theme: did the participants’ perception
about engineering change because of their experiences in the new ﬁ'eshman sequence
courses. The focus of the analysis for this theme was on the last two items of the

instrument.

84

Part H - Analysis/Discussion

4.6 Data Analysis

The data was analyzed by examining the pattern of each theme with supporting
quotes as evidence. The aim was to capture the participants’ true feelings in all possible
dimensions without prejudice to any one speciﬁc opinion. As mentioned earlier, Part I of
the protocol was a reﬂection of the PFEAS subscales and was primarily developed by the
BEES research group. The data in this part was mapped onto the results of the
quantitative phase. Part II was speciﬁcally developed for this study and was treated at
length to explore initial perceptions of freshmen and why the initial perceptions were not
affected by the treatment. The subsections below provide a detailed description of the
data analysis with relevant quotes. Some parts of the quotes are italicized by the author to
add emphasis.
4.6.1 Part I - General Engineering Perceptions: A general view of the data showed
the majority of participants responded positively to Part I of the Protocol. Themes in two
of the subcategories, engineering perceptions and engineering abilities, were
overwhelmingly positive whereas themes in the third subcategory, basic skills, had mixed
responses. The data showed that the participants joined the freshman year with strongly
positive perceptions about engineering and had conﬁdence about their engineering
abilities. However, they had mixed feelings about their basic skills, i. e. basic sciences,
computer and communication skills. The three subcategories are brieﬂy discussed below.
4.6.1.1 Engineering Perceptions: Positive engineering perceptions of the participants
were deeply rooted and had been developed over time, often since childhood. Parents and

high school teachers played an important role in developing these perceptions especially

85

if they themselves were engineers. Talking of positive engineering perceptions, Dons,
like most of the other participants, viewed parents and high school teachers as a source of
inspiration. For him, selection of mechanical engineering was a natural instinct since he
grew up around automobiles as was apparent ﬁom this statement:

Well my dad actually works for BMW of North America. He
previously was a master mechanic in a shop, certiﬁed many times
and won a lot of awards. He’s an absolute genius when it comes to
automobiles. So growing up it was an inspiration there. Also, one
of my favorite teachers in high school was an engineer... She
always said that I was capable of it because of my math skills so it
was kind of an inspiration... I kind of grew up with an engineering
background.

The participants deﬁned three aspects of engineering that interested them, in
addition to their general interest in math and physics: 1) engineering as a practical,
hands-on profession; 2) problem solving nature of the job with multiple solutions; and 3)
working on projects in a team setting. They appreciated engineering in different roles, i. e.
as a discipline, as a job, as a profession, etc. and they liked all of these roles. Pat was
highly motivated to be a mechanical engineer. He deﬁned and described what he liked
about engineering in these words:

I always liked the problem-solving and working in teams. In
engineering, there is a lot of different ways to get to that one
solution and it makes you think about all those different
possibilities. So you kind of see yourself as a kind of problem-
solving, hands-on type. I loved to see how things worked, whether

it was a reﬁigerator or how a house was put together, anything. So
I always wanted to be a mechanical engineer.

 

5 All names in this document are ﬁctitious and used only for illustrative purposes.

86

Most of the participants enjoyed math and physics. They understood the
requirement of the basic science subjects for engineering and took engineering as a
natural consequence. Brad, for example described engineering in these words:

Since I was junior high, maybe even earlier, I’ve known I was
going to do engineering of some type... I haven’t really had to
make a decision on what ﬁeld to study, to pursue, because math
always came easy, physics always came easy, and those are what I
enjoyed. I wanted to get into design of some kind with a math and
physics background. And that’s basically describing engineering.

The participants were also aware that engineering was not math and science
alone. Mike, for example, talked about his perception of engineers in these words:
“Engineers can do more than just math.” Lora, a Bio-systems major, did not like math
and physics, yet she was persistent to continue with her choice because she knew she
could still be successful in her choice of engineering if she could survive the initial
challenges of math and physics. She said:

I am not a math major and I am not really interested in physics but
I got to take them. And it kind of turned me off engineering a little
bit... So I was actually considering switching out of engineering at
the beginning of the semester.

When asked what kept her in, she said:

I was looking at the classes, I was like , if I can just get through
these, you know, two more classes of calculus, one more class of

physics and then I can start taking the stuff I’ m actually interested
in. So I gave it a second chance.

The participants considered engineering as a rewarding career that could earn
them a good job with decent salary despite the economic crunch these days. Mike, for

example, said:

87

I’m willing to work hard, and I think engineering will make me
work hard... 80, I just want to get basically a solid degree that will
be able to give me a job and make money.

Interestingly, Don termed engineering as “artistic” because he believed every
engineer is unique and innovative. His elaboration of this notion shows a strong
comrrritrnent and desire to become an engineer:

It’s more rewarding just because it’s something that I would
actually appreciate doing, working on a design team, getting to
actually express your own opinions and kind of being artistic at the
same time that you’re getting paid to do work and crunch
numbers... The idea of being in a group project and having your
own opinion to solve a problem in a way that nobody else saw that
it could be solved,. . . to have that intricacy. And everybody has
that capability, every engineer at least, and to have a whole bunch
of those people together it’s just an awesome, awesome
experience.

Positive engineering perceptions were deeply rooted and had developed over time
since childhood. Parents and teachers played an important role in the development of
these perceptions. Participants deﬁned what it meant to be an engineer with practical
hands-on nature; they appreciated working in teams and problem solving with multiple
solutions. They mostly liked math and science subjects, those few who did not like them
understood their requirement for engineering and took them as challenges. Some termed
engineering as artistic and rewarding and commented that they “really enjoy design and
engineering”.
4.6.1.2 Engineering Abilities: The participants commented on their conﬁdence and
understanding of their engineering abilities speciﬁcally problem solving, designing,

leadership, and working in groups. Sara, for example, elaborated on different engineering

skills and expressed her conﬁdence of these skills in these words:

88

Team working skills, problem solving skills, are able to work
under little supervision, and be able to communicate when you
need help. You got to analyze things and review things and do
compare and contrast; you don’t really get the opportunity to do
that as much as in engineering... I like learning new stuff; I like
not always knowing the answer... Coming up with a new way to
get the answer.

Lora added to Sara’s description of engineering abilities, though she did not
express her explicit fondness for these abilities. She believed creativity and knowledge,
in addition to good communication and team work, were important for engineers.

Deﬁnitely you got to be creative about thinking of solutions to
your problems. You got to know about pretty much all aspects of
engineering... In a group working on the same project, you got to
be able to know what other people are talking about. You have to
be able to work well with others, like good communication.

Two aspects of engineering abilities were new to the participants because they
were not fully exposed to these activities in their high school years: working in groups,
and ﬁnding multiple solutions to a given problem. Irrespective of whether they liked
these activities or not, most students appreciated their importance in engineering. Pat said
while comparing engineering with other degrees:

I always liked the problem-solving and working in teams... Other
degrees it’s like, ‘leam this, learn this, here’s the equation.” In
engineering, it’s like there is a lot of different ways to get to that
one solution and it makes you think about all those different
possibilities.

Jon on the other hand was not happy with the team work aspect though he
appreciated its importance. He expressed his views in these words:

It is all part of the experience. Not just getting the project done but

ﬁguring out the way to adapt to what is going on to the group of
people working on the project. Some people are satisﬁed; some

89

 

people ﬁnd ways to come to an agreement, some people just don’t
work out. And we were the one team with the people that
somewhat didn’t work out... That part was the hardest part of the
projects.
4.6.1.3 Basic Skills: Participants cited the match between basic skills and the demands
of engineering as a motivating force for selecting the engineering major. Mike, for
example, based his decision on his math and writing skills:
I’ve been strong in math and my writing was okay. And I
discovered the applied engineering sciences major, which kind of
implemented both and decided that’s what I was going to do.
Some participants did not like basic science subjects. They, however, considered
them as initial challenges and expressed their conﬁdence to overcome those in due course
to achieve proﬁciency in the basic skills. Lora, as noted earlier, was not happy with her
performance in math and physics but was persistent in continuing with the major and
meeting these challenges:
I had to repeat my calculus class because I did not do so great the
ﬁrst semester... math is not my favorite subject and neither is
physics, and not really sure why I have to take so many classes of
it. I know it’s like the general college requirement, so I’m pretty
sure that mechanical engineering needs to know that stuff. But
that’s deﬁnitely been a challenge But I know I have to take
them.
Part I of the protocol inquired about participants’ initial perceptions and
expectations as they joined the program, and their experiences and challenges during the
freshman year. The three subcategories and their related themes from this part were

considered an extension of the PFEAS subscales. Participants’ responses were aligned

with the quantitative results in so far as the ﬂatness hypothesis was concerned. The DS

90

group had joined the program with strongly positive initial perceptions about
engineering. They could deﬁne engineering in many roles and expressed conﬁdence in
their engineering abilities and basic skills. They appreciated their new experiences of
team work and problem solving with multiple solutions. The majority understood that
engineering was much more than math and physics. Those who liked these basic science
subjects correlated them to their preference for engineering; those who did not like them
still persisted to continue, considering them as challenges.

4.6.2 Part II - DS Related Perceptions: This part of the protocol was speciﬁc to the
cornerstone sequence and directly addressed the secondary research question. It focused
on freshman experiences and expectations in the cornerstone sequence and how those
experiences affected their initial perceptions and attitudes about engineering. Two major
subcategories emerged: DS experiences and DS effects on perceptions, each of which
had themes.

4.6.2.1 DS Experiences: The two courses in the cornerstone sequence were distinctly
different in their objectives. EGR 100 was an introductory level course aimed at
engaging the students with the profession of engineering emphasizing the importance of
organized methodology, team work, good communication, work ethics and multi-
dimensional problern solving. The course involved extensive team work on small
projects. EGR 102, on the other hand, was focused on the use of computational tools to
solve engineering problems, mathematical modeling with speciﬁc solutions, and much
more individual work. Understandably, student reactions were different for the two

courses, as discussed below.

91

 

EGR 100: The participants’ expectations were not very well aligned with the
course objectives. They expected the course to be more of an advanced exposure
to engineering design with real world engineering projects. Their responses could,
therefore, be misleading if not seen in the context of the course objectives. The
data emerged into four themes: project experiences, class experiences (other than
projects), time per credit (appropriateness of time spent per credit hour), and
course conduct. The participants’ responses were mostly negative but some were
positive in all of the four themes.
Project Experiences: The projects for this course were designed to get
students to appreciate the importance of implementing an organized a
methodology for solving a given design problem. The emphasis was on
the process, not on the problem itself. Participants seemed to have
somehow missed out on the projects’ objectives. Seven out of ten
participants were not happy with the projects; they considered them
irrelevant, boring, and out of the context of engineering. Some did like
projects but complained of logistic problems and less-learning-more-ﬁrn.
Sara conveyed her disappointment in these words: “Not one project I did
in EGR100 that I felt like helped me engineering wise, not one...” Pat
ﬁrrthered Sara’s point of view. To him the projects had no relevance to
engineering; they were there just to keep students busy:
I wasn’t a huge fan of a lot of the projects... They were boring. I
thought they had nothing to do with engineering. They were just
kind of mundane tasks. It almost seemed like busy work. That’s

what I deﬁnitely felt like and talking with other students in the
class, they deﬁnitely felt like as well.

92

Matt, like few others, enjoyed the project experience and called them ‘ﬁrn
time’, but desired more science and math content.
I think EGR 100 was like the best experience for me... pretty
much what I expected, come here and learn something and still
have fun at the same time... fun thing is to just work in groups,
ﬁgure out those things... I think if they would have threw [sic] a
little more math - I mean calculus and chemistry... in Engineer100
and kind of prepared me for 102.
Erin, liked projects but had trouble in the implementation phase. She
expressed her views:
I enjoyed EGR100 better just because of the projects. The projects
were kind of cool... it at time seemed like it was kind of a pain in
the butt because we were ﬁ'eshmen. And it was hard to be able to
go get stuff. But then once we ﬁgured that out that the bus went
right to Meijer [grocery store], it was all right.
The students’ expectations were different from the project objectives.
They were more interested in the problem solving aspect than its process.
As this was their ﬁrst exposure to a design project, they expected to see in
it everything they knew about engineering. The responses of Pat and Sara
should, therefore, not be surprising. They found the projects overly simple
with no direct relation to engineering, for example, the food car project,
and were obviously disappointed. There was a mismatch between student
expectations and the learning objectives; hence the participants called the
activities “boring” and “pointless”. However, they appreciated the beneﬁts

of team work and the intricacies of logistics involved in managing the

work together.

93

 

Class Experiences: The class lectures were organized in an attempt to
cause the student to appreciate the unique aspects of engineering
profession, its signiﬁcance to society, its ethical framework, and career
opportunities in this ﬁeld. In addition to the class instructor, lectures were
delivered by the experts in different engineering disciplines from
academia and industry. The participants had more negative than positive T
feelings about the class lectures. Four out of seven respondents (three did
not respond) thought the lectures were not interesting and they attended

only for clicker quizzes. Pat did not like the class lectures, he called them

 

boring and pointless in these words:

It was a class that you went to every week and you just did the
quiz and then you just kind of sat there because it was pointless. I
think a lot of other people felt the same way. You didn’t gain
anything from it... It wasn’t interesting compared with some of
my other classes. It didn’t really intrigue me in any way... I’ve
seriously considered not being an engineer before because of the
classes. Because I’ve thought, “Man, these classes are so boring.
This is what the whole Engineering School is going to be like”.

Pat’s statement was strong with some serious ramiﬁcations, and may
require study of the type of instructional material and the instructor’s
teaching style - the two aspects out of the scope of this study. It, however,
was apparent that lectures in large lecture halls were not generating
enough interest. Don had similar sentiments about the lectures:
You could’ve just stayed home, read the Power Point that he
[instructor] posted on Angel and you would’ve gotten the exact

same education. The only reason that you went to class for
EGR100 was for the clicker quizzes.

94

Erin felt differently. To her the class was helpful in understanding the
engineering profession and its different disciplines:

I thought EGR100 was really helpful because I was able to meet
with people and get an introduction to engineering. Hear from the
professors from all different majors so... it would really help you
decide what you wanted to do before you got three years into it
and decided that wasn’t what you wanted. It helps you be familiar
with some policies and things that happened and how stuff
worked. E

Time per Credit: Unlike the class lectures which some students liked

and others didn’t, they unanimously agreed that the work load for the

 

project portion was too high. Students were new to group work and faced t
difﬁculties in communication and decision making in a group activity.
They suggested the credit hours be adjusted to reﬂect actual and realistic
time requirements. Pat indicated his frustration about how much time he
spent on the projects.
There would be weeks where we spent twenty hours on a project
for EGR 100... and this is a two-credit class. So that was deﬁnitely
frustrating. It’s a lot of work for a two-credit class.
Don supported Pat’s views:
I think we would’ve really appreciated it if it was either a reduced
workload or if it was at least a 3-credit course. There was a lot of
work... I didn’t spend that much time studying for my 4-credit
classes.
Brad summed up student ﬁ'ustration on time spent on projects in these

words:

They’re two-credit courses, but the amount of time I was expected
to put into them was way beyond what I feel a two-credit course

95

should be. EGR100, I spent hours and hours and hours doing that
Food Car project. I don’t know how many trips to Meijer I made,
and it ended up being a two-credit. ..

Talking about the timing of the projects, Mike thought it was too early to
do group projects. He said:

I was surprised at the physical aspects of it, where we had to make,
within the ﬁrst few weeks, a food car with a group of random kids.
And it was a little weird because I didn’t really know where
anything was and they expected us to go out and buy food and
stuff and I had really no idea how to even get places... they just
gave the project really soon..

 

The participants touched on two issues with regard to the work load for
this course: projects were time consuming and they were given too early
in the course. They admitted though that the projects were simple and did
not require much of engineering knowledge. It may be pertinent to point
out that for many of the freshmen this was the ﬁrst time they were
exposed to an independent environment away from their homes. They did
not have a personal transport (ﬁ'eshmen are not allowed to keep
automobiles on campus), so their travel was restricted to foot, bicycle or
public transport. They were passing through a life changing phase. They
were grouped in teams of students whom they were meeting for the ﬁrst
time and they had to learn to adjust to each other’s personality and yet
produce a joint work within the time constraint. All of these issues take
time to settle and this time varies from individual to individual. Some of
the students took more time to adjust than others, especially in the group

work, hence the complaint about projects given too soon.

96

Conduct of the Course: Fall 2008 was the ﬁrst semester when the
cornerstone sequence was in ﬁrll swing. It was expected to encounter
problems and difﬁculties in the beginning. All participants opined that the
course lacked in preparation and execution, and that it had no prescribed
syllabus and textbooks. They unanimously agreed that lectures and
projects were easy, and required little or no preparation to earn a good
grade. Sara expressed her concern about the course execution. She
reﬂected:

Well, I have that easier BGR100 course which is a total waste of

time in my opinion... They had really pointless assignments. It
didn’t have a topic... I just don’t feel like it was executed well.

 

Brad, like most others, knew EGR 100 was a new course and that new
courses do not go as well as the established ones. He recounted that:
It seemed like they were still trying to work some kinks out. And
that just didn’t really feel that good, being like the guinea pigs. I
know it’s a new course, so that’s probably to be expected — a little
bit of a sense of disorganization...
Brad went on to say that course was not difﬁcult ﬁ'om the material
standpoint:
It didn’t seem like there was any real testable material that we
talked about... And they weren’t too hard. I mean, I feel like
someone off the street could walk in and take EGR100 exam and
pass it on common sense alone...

Don supported Brad on the course difﬁculty level and claimed his 4.0

came with little effort:

97

Most of EGR 100 was just kind of straightforward. It was pretty
easy and I can’t really say anything that was really difficult in
lecture or lab really. It’s all prior knowledge, it’s as long as you
understand human beings you will pass the class. To be completely
honest, I didn’t study once for the midterm exam or ﬁnal exam,
and I 4-pointed the class.

The course was considered easy in terms of its content by most of the
participants. They did not ﬁnd enough challenge in it; lectures were easy, F
projects were easy, and there was not much testable material to prepare to

get a good grade. Students knew it was a new course and were somewhat

aware of the problems with new courses.

 

are

Unfortunately, all but few of the participants were dissatisﬁed with EGR 100 in
terms of their class experiences. Although this introductory level course was
aimed at motivating and engaging students to the discipline of engineering, the
majority of participants criticized the course in all the areas: projects, lectures,
conduct, and workload. The main cause for general dissatisfaction was that their
expectations did not match the course objectives. They joined the course with
strongly positive perceptions about engineering. They expected to learn and
implement engineering design techniques in some real world design projects.
Their experiences fell short of their expectations. They found projects boring and
lectures as pointless activities. They appreciated the new experience of working
in teams but reported excessive time consumed in team activities due to

communication, coordination and logistics issues. Although the majority

98

participants expressed excitement about studying engineering, this ﬁrst
introductory course failed to build upon that enthusiasm.
EGR 102: This two credit course was primarily designed to learn problem
solving with the MATLAB. The typical class session would begin with a lecture
on a MATLAB aspect followed by a pro grarnnring assignment and a quiz at the
end. It was a well structured course with deﬁnite boundaries and mostly
individual assignments except one project activity towards the end of the course.
The course contents were not new; they had been taught in other courses for a
long time and had well organized lecture plans with focused activity. The
students’ responses emerged into three themes: course experiences, course
credits, and conduct of the course. Unlike EGR 100, participants mostly enjoyed
the learning experience of this course mainly because their expectations were
aligned with the course objectives. Some participants, however, reported a
disproportionately high level of difﬁculty.
Course Experiences: Participants had mostly positive experiences in this
course. Those who liked math liked EGR 102 because MATLAB is based
on math concepts. Unlike EGR 100, this course had a more focused
approach that participants appreciated. The participants’ expectations
seemed to align with the course objectives. Talking of positive learning
experiences, Lora compared the two courses in these words: “I learned
way more in EGR 102 than I did in 100.” Erin followed Lora’s sentiments

about the positive learning experience. She said:

99

I think I learned a lot in MATLAB. I learned a lot in Excel too. I
probably learned more in EGR102. I mean stuff I didn’t really
want to learn but I did.

Brad talked about his achievements and the course outcomes. He was
happy with the course learning though he had to spend more time on it
than his other courses:

I’ve enjoyed EGR102 a lot better because that’s more of a speciﬁc
goal of the class. You’re learning MATLAB and learning
programming methods... And it’s been interesting. I’ve learned
new techniques... and 102 has been a lot more enjoyable... I’ve
put a lot of time into it, but it’s felt a lot more worth it, because
there’s actually a ﬁnished product from the class. I know how to
use these techniques, and I know how to run MATLAB now. And
that’s what I took from the class.

Mike liked the course because he thought it was more relevant and
rewarding:
And in 102, it’s more systematic, and a little more relevant, it
gives you an idea of more technical ability feel. . .I enjoyed 102, I
think, because of the relevance. I felt like it was more rewarding. It
was a little more work, though. But I think I enjoyed 102 more
than 100.
Don was one, not too happy with the course experience because he did not
enjoy math. He summed up his sentiments in the following way:
It’s basically just applied calculus. What was the class covering? I

couldn’t even tell you. It was basically, go to class, learn a new
formula and take a quiz on it...

The positive learning experience was one common theme by the majority

hence terms like “systematic”, “relevant , rewarding”, were commonly

used. They enjoyed the course despite the fact that they had to put in more

100

effort for this course. EGR 102 was based on math concepts. Those who
enjoyed math, or realized its relevance to engineering enjoyed the learning
experience in the course. They termed it much more rewarding than EGR
100. However for those like Don who did not enjoy math, EGR 102 did

not make much sense to them.

 

Course Credits: Most of the participants felt the course was much more '-
demanding than a two credit course should be; they had to spend extra 5
time to ﬁnish the assigned work. Matt, for example, called it a demanding i
class in these words: a
E

Engineering 102; it was a handﬁd. It was a lot— way more work
that time pretty much... I’d say for it to be a two credit class is
pretty tough. It’s more time consuming than any other class and
it’s more demanding.
They however admitted to have learned the most in this course. As an
example, Pat remarked:
It’s deﬁnitely a lot of work for a two-credit class. I would say it’s
deﬁnitely an interesting class. You do a lot more with the actual
engineering... 1 would have to say the MAT LAB that I learned the
most.
It may be interesting to note that participants claimed both DS courses
demanded more time than a normal two credit course, but, for different
reasons. For EGR 100, students faced problems in logistics issues and
working in groups; the course contents were easy. For EGR 102, it was a

tough and demanding course, “a handful”, that required extra time to

complete the assigned work.

101

Conduct of the Course: Not all the participants talked about the conduct
of EGR102. Those who did, had mixed feelings. Mike thought the course
was better conducted than EGR100, in these words:
100 was new... it just felt may be not completely professionally
organized... And in 102, it seems like they kind of had it down
more, where it’s more systematic...

Don complained of excessive homework assignments and quizzes in class:

The only reason that you went to class for EGR102 was for the
homework due and the quizzes at the end of class.

 

I'.‘ "-

Sara was the only participant who did not like the way this course was
conducted. She reﬂected:
But, that class is just a waste of time. It’s not really like a
structure. . .it jumps around it doesn’t have like a certain straight
forward outline course.
EGR 102 was considered better than EGR 100 in terms of the learning
experience, maturity of curriculum, and alignment of student expectations with
the course objectives. They had to spend extra time on the assignments but
appreciated the enjoyment and learning they experienced during the course. The
interviewed participants speciﬁcally enjoyed learning MATLAB as a useful tool
for engineers. Those who did not enjoy math and MATLAB did not enjoy the
course.
4.6.2.2 DS Effects on Perceptions: This segment of the study was the most directly
related to the research question. None of the participants gave a negative response in this

category (see Table 4.3). Eight out of ten said loud and clear that the cornerstone

102

sequence courses did not affect their initial perceptions about engineering. Interestingly,
the majority of the participants admitted to strong negative experiences in at least one of
the two courses, yet when asked to reﬂect on the effects of the D8 courses on their
perceptions, the participants clearly denied that negative experiences affected their
perceptions. When Pat was asked if the DS courses changed his initial opinion about
engineering, he said:
It didn’t really change my perception at all. I would say I kind of
expected this from [pause]. MAT LAB has been useful, but other
than that, neither of the courses, if I was undecided, have guided
me, would have guided me in any way or have even, I’m not one

hundred percent sure I want to be a Civil engineer. They haven’t
swayed me to want to be a Mechanical Engineer in any way. 15..

 

To Don, it didn’t matter either. He had clear objectives set forth no matter what happened

in the initial courses. He said:
Not really. I would say basically just kind of broadened my
horizons. I’m just excited for engineering. It’s kind of a thing
where I just want to be an engineer now and it really doesn’t

matter. I’m just really excited and I think I’m doing what I should
be doing and I’m really excited to have a career.

For two participants, Lora and Don, it did make a difference — a strengthening of their

already positive perception of engineering. Lora said:
I respect engineers a lot more now that I know the difﬁculty that
goes into it. And how much work you have to put into it. That’s
deﬁnitely changed my viewpoint a lot; they’re smart kids.

Sara’s comments were unique; she reiterated that her perception of engineering remained

unchanged, but pointed towards another aspect that could make a marked difference in

103

 

student attitude, namely, instructor attitude. This aspect was, however, out of the scope of
this study.

No, that wouldn’t change, you’re not going to always have people
in the real world that, are going to be nice all the time so,
whatever, I would have to stick it out and deal with it... Teachers’
attitudes are just bad sometimes and they shouldn’t be. I feel the
teachers sometimes make it seem like they’re doing us a favor.

g—
Brad had strong views about his self but was concerned about his roommate who he
thought might switch to a different major because of the negative experiences in BGR
100. Brad reﬂected:
Not really, because I’ve been able to move past, maybe not such a _:_

 

good experience in EGR100, because I know that engineers aren’t
going to sit around and learn about other engineers. That’s not
what it is. But that’s kind of what it seemed like you did in
EGR100. I’m trying to get my roommate to understand that, you
know, like, “I know that you have hated these intro classes, but
this isn’t what you’re going to do... So if you just get past these
intro courses, I think you’re going to really like the stuff you start
to learn.” And so I hope that he doesn’t change his major based on
these.

These comments are particularly concerning because negative experiences could be

effecting the perceptions of students with not-so-positive (mild) initial perceptions about

engineering.

Students like Pat, Don, and Brad had strong initial perceptions about engineering.

These students had clear objectives set forth for their future goals. They did not care

about negative experiences in D8 courses because they could see the reward four years

down the road. They had joined the program with a deﬁnitive aim of becoming an

engineer. These students had identiﬁed several issues about the DS courses (speciﬁcally

for EGR 100). The most important ones were: 1) student expectations were not aligned

104

 

with the course objectives; 2) they appreciated team work but it consumed excessive
time; 3) lectures in big halls were not rewarding; 4) EGR 100 lacked coherence and
coordination, course contents lacked testable material, and a textbook was not assigned;
5) excessive quizzes and assignments to turn in for EGR 102 that took a lot more time.
Despite these negative experiences in the cornerstone sequence, the participants’ initial
perception about engineering remained unchanged. They were ﬁrm in their commitment
to continue with their engineering major and join the profession of their choice. In fact,
for two of the participants the DS experience had a positive impact on them. It further

strengthened their perception and concept of engineering.

4.7 Discussion

The qualitative phase of this study provided a mrique opportunity to probe deeper
into the kind of learning that took place in the newly introduced cornerstone sequence. A
sample of ten freshmen ﬁ'om the DS group was interviewed one-on-one. The sample was
comprised of students who had undergone EGR100 in the previous semester and were in
the ﬁnal stages of EGR102 in the current semester. In other words these students had
experienced the full cornerstone sequence. All the participants belonged to Category 1,
which could point towards a strong preference for the ﬁeld.

Part I of the protocol showed the participants had joined the program with
strongly positive initial perceptions about engineering. These perceptions had developed
over a long period since childhood and were mostly inﬂuenced by parents and teachers,
especially if they belonged to this profession. The preference for math and science

subjects also contributed towards strong preference for the engineering ﬁeld. The

105

 

participants appreciated working in group activities, and on open-ended problems with
multiple solutions. They radiated conﬁdence in their abilities to succeed as engineers.
Part II of the protocol focused on the participants’ experiences in the new
cornerstone sequence. They generally had negative experiences in EGR100 because the
course objectives fell short of their expectations. They considered it to be an easy course;
it offered little academic challenge and did not add much to their already acquired r-
knowledge. They appreciated the concepts of team work and problem solving in 1
engineering. They expressed their understanding of engineering to be of a much wider

scope than just math and science. Participants were disappointed with almost every

 

aspect of EGR 100 including content and conduct. They did not like projects and
lectures, and were dissatisﬁed with the way it was conducted mainly because their
perceptions were not aligned with the course objectives. On the other hand they liked
EGR102 more because their expectations more or less were aligned with the course
objectives. They complained of excessive work load for both courses but for different
reasons; EGR 100 had more administrative issues whereas EGR 102 was full of lengthy
MATLAB assignments.

Despite negative experiences in the DS courses that might have dampened their
enthusiasm, the participants expressed their ﬁrm commitment to becoming an engineer of
their choice. When asked if their initial perception about engineering had changed in any
way because of the negative experiences, all but two of the participants unanimously
disagreed. Those, two who agreed, had more positive perceptions about engineering.

Students in the DS group had joined the program with strongly positive attitudes towards

106

engineering. The negative DS experiences did not signiﬁcantly change these strong
perceptions [77].

Before drawing conclusions and recommendations it may be pertinent to review
the limitations of this qualitative framework. First and foremost, as with all qualitative
research, the issue of generalizability needs to be considered with regard to the sarnple-
size and representativeness of the target population before making informed decisions ‘-
about the new sequence [34]. The sample did not include a portion of the population that
took CSE 231 instead of EGR 102. Moreover, the sample could have been skewed

because of the type of incentive offered, $25, perhaps by not attracting those who did not

 

‘3

need the incentive as much. Researcher bias in a qualitative research also could sway the
results of the analysis [80]. Maximum efforts were made to remove the researcher bias by
engaging in rechecking of the coding by another researcher and repeating the coding

several times before drawing conclusions.

107

CHAPTER 5
CONCLUSION

This study involved an explanatory mixed methods design that is characterized by
an initial and extensive quantitative phase built upon by a subsequent qualitative phase.
The quantitative phase was designed to answer whether the new cornerstone sequence
was more effective than the older traditional sequence at positively inﬂuencing freshman
attitudes about engineering over the course of one semester. Students in the new
cornerstone sequence were compared to students in the previous traditional sequence by
collecting attitude data twice over the course of one semester and exarrrining changes in
the two groups’ attitudes with repeated measures ANCOVA models. It was found that
ﬁ'eshmen join the program with positive and strongly positive attitudes towards
engineering. Students in the cornerstone sequence had higher ACT scores, enjoyed math
and science more, and did not believe engineering to be an exact science. Females had
lower conﬁdence levels in basic engineering knowledge and skills, problem solving
abilities, and engineering abilities. The DS group, however, had a similar longitudinal
trajectory to the TS group, so there was no evidence of differential inﬂuence on student
attitudes. In other words there was a lack of treatment effect in the DS group [68, 69].

To probe deeper into the quantitative results, especially the lack of the treatment
effect, the qualitative approach was adopted with further collection of data and its
analysis. One-on-one interviews were conducted with a representative sample ﬁom the
DS group. The analysis of textual data revealed that freshmen joined the engineering
program with strongly positive initial perceptions about engineering. These perceptions

were based on their childhood and high school experiences and were inﬂuenced by

108

 

parents and teachers, more so if their parents and/or teachers had stressed STEM-based
education. The preference for math and science subjects also contributed towards a
strong preference for engineering. The two cornerstone sequence courses impacted the
students’ attitudes about engineering in several ways. The participants appreciated: 1) the
signiﬁcance and challenges associated with team work; 2) that engineering was not an
exact science; 3) that engineering was not only math and science, team work,
communication and ethics were equally important for success; 4) that the usual high
school formula for credit load does not work with design courses because of intangible
factors like group activity [77].

The above notwithstanding, most of the freshmen reported negative experiences
in the DS courses, speciﬁcally in EGR 100. Some of the major impediments were: 1)
their expectations were not aligned with the course objectives; 2) the course being a new
course lacked coherence and coordination, giving a sense of disorganization; 3) excessive
workload (projects in EGR 100, and class quizzes and home assignments in EGR 102);
4) a sudden shift in students’ academic and living environments. These negative
experiences were dampening their enthusiasm for engineering and need to be addressed.
This criticism is consistent with previous research on why students leave the STEM
majors [4]. A review of the course curriculum, especially the course projects, could help
in matching up the student expectations with the course objectives. Interestingly, despite
- the impediments, the participants’ initial attitude and perception towards engineering did
not change signiﬁcantly. When asked if their initial perception about engineering had
changed negatively because of the negative experiences, the participants unanimously

disagreed. Students in the DS group had joined the program with strongly positive

109

 

attitudes about engineering. The negative DS experiences did not signiﬁcantly change
these strong perceptions.

The qualitative phase validated and supported the quantitative ﬁndings. Students
in the DS group were found to have joined the program with strongly positive attitudes
about engineering that were deeply rooted. These attitudes were not affected by the short
(one semester) and predominantly negative treatment effects of DS courses as indicated
by the participants’ responses in the qualitative phase of the study. All of the participants
were ﬁrm in their commitment to continue with their engineering major and join the
profession of their choice. It substantiated the belief that strong attitudes are harder to
change: they are durable, and they have impact [15, 16, 81]. They are not much affected
by a short treatment, especially if it is not very well organized and focused.

The cornerstone sequence was new; it was passing through a transient phase.
Many of its conduct and execution issues would be resolved as the course is ﬁne tuned
and reaches maturity. At this time it may not appear to be as effective a treatment as it
was initially perceived. It would therefore be prudent to allow at least 2-3 iterations for
the sequence to reach a steady state. This study could meanwhile continue to collect data
to build upon the quality analysis.

It was pointed out in the beginning that the new cornerstone sequence required
two semesters or one complete freshman year for implementation. To capture the
opportunity window of one semester in which both the groups were available for a
comparative study, the quantitative phase of this study was launched concurrently for one
semester. In the quantitative phase, data for EGR 100 was used for statistical modeling,

thus capturing only partial effects of the sequence. Similarly, for the qualitative phase,

110

 

one-on-one interviews were conducted when the semester was still in progress thus
captming full effects of EGR 100 but losing some effects of EGR 102. Also, a portion of
the freshman population that opted for computer engineering or computer science majors
was lost because they did not take EGR 102. It is argued that compound effects of the
two courses in the intended sequence could be signiﬁcantly larger than these partial
effects. It therefore suggests that the study be continued to capture the full effects of the
sequence.

In the quantitative phase the time between pre- and post- surveys was only eleven
weeks. Due to administrative reasons the surveys could not be conducted nearer the
beginning and closer to the end of the semester. This could have resulted in the loss of
vital experience given the fact that the initial few weeks of the semester are important,
and also the fact that the learning curve is typically not linear. It was argued then that a
short treatment of a partial semester of eleven weeks may not be enough to affect a
signiﬁcant change in student perceptions, more so when the perceptions are durable and
deep rooted [69]. It has been found that treatment effect can capture the non-linear
changes in the learning cycle if the samples are taken more ﬁequently [82]. It is
suggested that the study may be spread over two semesters to capture the full effects of
the sequence by taking more frequent samples throughout the year with a gap of 4-6
weeks.

The study also identiﬁed several issues with the two samples due to selection bias
in both the phases. Different data collection methods and their associated incentives
could have skewed the samples towards higher achieving, more motivated, and/or lower

socioeconomic groups. In qualitative research the issues of researcher bias and

111

 

generalizability also need to be considered with regard to the sample size and its
representation of the target population. A larger and more homogenous sample size is
therefore desirable and could be achieved with concerted efforts in the subsequent data
collections.

PFEAS subscales are being revised to further improve the internal reliability and
structural validity of the scale [51]. The upgraded version of PFEAS may become
available shortly, which could improve its power to detect intervention effects. It is also
pertinent to build up our own data and develop our own norms to improve the reliability
of the scale. A short, compact, and more powerful scale would be able to precisely
measure and detect the differences with accuracy.

This mixed methods study also brought to light some other variables out of the
scope and context of this study that could impinge upon the student attitudes and may be
good covariates for the future studies. These include course content, instructor attitude,
and teaching style. It may also be interesting to add other forms of qualitative data to
further look into the construct for a broader view, for example, classroom observations,
focus groups, student portfolios, and interviews with class instructors. Adding more
variables may make the model more complex, but at the same time it would be more
realistic and complete and may throw more light on the time *group interactions.

The results of this mixed methods study indicate that changing student attitudes
may not be an easy task. We need to develop better understanding of the ﬁne-grained
details of the actual implementation of such cornerstone engineering courses. Such
research is critical for the formative evaluation of these programs in order to improve

curriculum development efforts. Clearly tight and focused quantitative studies

112

complemented with a qualitative component provide a much broader and deeper insight
into the learning that takes place in freshman courses. We must engage in a broader and
larger research program that should include improved instrumentation with a larger and
homogeneous sample size, with same collection methods, and with frequent samples at
4-6 weeks intervals (instead of pre- and post-) to capture the subtle, non linear and full
effects of the cornerstone sequence. The accuracy of the model could be further enhanced
by the addition of carefully selected covariates, for example, student GPA, ethnicity, high
school. attended, and socioeconomic status. Other forms of qualitative methods including
classroom observations, focus groups, interviews with the class instructors, consideration
of instructor attitude and teaching styles would certainly provide further depth to our
understanding of student attitudes.

The mixed methods approach is a useful tool for developing a better
understanding of students’ life experiences with freshman courses [36]. To measure the
longitudinal effects of the new sequence, the fall 2008 cohort of students should be
observed during their junior and senior years in a progressive manner and with a larger
and diversiﬁed sample to represent the population in a true sense with little or no
limitations and preferably with the revised version of the PFEAS, if possible. This would
provide a clearer and better understanding of the freshman engineering experience so

vital to improve the quality of our pro gram and reduce attrition.

113

 

APPENDIX A

PITTSBURGH FRESHMAN ENGINEERING ATTITUDES SURVEY

(PFEAS) [19]

This is a survey to elicit Freshman Engineers’ opinions and feelings about engineering.

Please do not spend more than 25 minutes to complete the questionnaire, so work as

quickly as you can. Remember these are your own personal attitudes, not your ﬁiend’s.

Your ﬁrst response is usually the most accurate for you.

For each statement about engineering, please ﬁll in the number that corresponds to how

strongly you disagree or agree with the statement.

1.

2.

8.

9.

I expect that engineering will be a rewarding career.

I expect that studying engineering will be rewarding.

The advantages of studying engineering outweigh the disadvantages.

I don’t care for this career.

The future beneﬁts of studying engineering are worth the efforts.

I can think of several other majors that would be more rewarding than engineering.

1 have no desire to change to another major (biology, English, chemistry, art, history,
etc.).

The rewards of getting an engineering degree are not worth the effort.

From what I know, engineering is boring.

. . 6
10. Engineers are well paid.

 

6 Items 10, 14, 16, 21, 23, 24, 39, and 46 in grey shade, were not used for this study.

114

 

DJ“ J

 

ll.

12.

l3.

14.

15.

16.

l7.

l8.

19.

20.

24.

25.

26.

27.

28

Engineers contribute more to making the world a better place than people in most
other occupations.

Engineers are innovative.

I enjoy the subjects of science and mathematics the most.

I will have no problem ﬁnding a job when I have obtained an engineering degree.
Engineering is an exact science. m
My parents are making me study engineering.

Engineering is an occupation that is respected by other people.

I like the professionalism that goes with being an engineer.

 

I enjoy taking liberal art courses more than math and science courses.

Engineering is more concerned with improving the welfare of society than most other

professions.

. I am studying engineering because it will provide me with a lot of money; and I
cannot do this in other professions.

. Engineers have contributed greatly to. ﬁxing problems in the world.

. An engineering degree will guarantee me a job when I graduate.

My parents want me to be an engineer.

Engineers are creative.

Engineering involves ﬁnding precise answers to problems.

I am studying engineering because I enjoy ﬁguring out how things work.

. Technology plays an important role in solving society’s problems.

115

 

For the following subjects and skills, please ﬁll in the number corresponding to the
response that describes how conﬁdent you are of your abilities in the subject or skill.
29. Chemistry.

30. Physics.

31. Calculus.

32. Engineering. H.—
33. Writing. ?
34. Speaking.

35. Computer Skills.

 

For the following statements about studying, working in groups and personal abilities,
please ﬁll in the number corresponding to the response that best describes how strongly
you disagree or agree with the statement.

36. I feel I know what an engineer does.

37. Studying in group is better than studying by myself.

38. Creative thinking is one of my strengths.

39. 1 need to spend more time studying than I currently do.

40. I have strong problem solving skills.

41. Most of my ﬁiends I “hang out” with are studying engineering.

42. I feel conﬁdent in my ability to succeed in engineering.

43. I prefer studying/working alone.

44. I am good at designing things.

45. In the past, I have not enjoyed working in assigned groups.

 

ll6

46. I am conﬁdent about my current study habits or routine.

47. I consider myself mechanically inclined.
48. I consider myself technically inclined.
49. I enjoy solving open-ended problems.

50. I enjoy problems that can be solved in different ways.

117

 

 

APPENDIX B

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PFEAS FACTOR LOADINGS
Statements 1-28
uthor supplied Measures [56] / Study based Measures© Normalized
No Perception Career Jobs Society Math Exact Family Weight
12 0.68/0.47 0.166
22 0.67/0.48 0.164
25 0.67/0.47 0.163
28 0.58 /0.46 0.143
17 0.55/0. 74 0.135
18 0.53/0.64 0.128
27 0.42 /0.45 0.101
1 0.66/0.84 0.124
2 0.64/0.85 0.121
4 -0.64/-0.65 0.120
3 061/0. 75 0.114
7 0.61/0.69 0.114
8 -0.60/-0.47 0.113
5 0.55/0. 72 0.104
9 -0.53/-0.58 0.100
6 -0.48/-0.67 0.090
23 0.74/0. 75 0.301
14 0.7 1/0. 64 0.289
21 0.51/0.61 0.206
10 0.50/0.50 0.204
20 083/0. 70 0.519
11 0.77/0. 70 0.481
-0.83/
19 0. 4 I (Comm. ) 0525
0.753/

13 0.65 (Basic) 0'475
15 0819/08] 0.516
26 0.7670. 74 0.484
24 0.87/0.83 0.586
16 0.62/0.63 0.414

 

 

 

 

 

 

 

 

 

© Study based measures are italicized.

118

r h

 

l'u". _

 

 

 

Statements 29—35

 

 

 

 

 

 

 

 

 

 

Author supplied Measures/Study based Measures© Normalized
Number Basic Communication Weight
30 074/056 0.244
32 0.71/ 0.233
31 0.68/0. 65 0.224
29 051/065 0.169
35* 0.395/0.6 7 (Compatibility) 0.317 0.130/0.l61
33 083/0. 70 0.422
34 0.82/0. 73 0.416

 

 

 

 

 

’Statement 35 is shared by both components due to relative weak relationship to both measures.
© Study based measures are italicized.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Statements 36—50
Author supplied Measures/Study based Measures© Normalized
Number Ability Compatibility Groups Study Weights
50 0.79/0. 7 0.247
49 078/067 0.244
38 0.56/0.4 7 (Comm) 0.177
42 0.54/0. 53 (Career) 0.169
40 0.52/0.46 0.163
47 079/0. 72 0.303
48 077/081 0.294
36 0.53/0.53 (Ability) 0.202
44 0.52/0.59 0.201
41 0.37/0.58 ’r
43 0.85/-0.86 0.385
37 -0.81/0. 79 0.363
45 0.56/-0. 62 0.252
46 0.84/0. 80 0.501
39 -0.84/-0. 82 0.499

 

 

 

 

 

 

 

 

© Study based measures are italicized. T Statement 41 too weak to be considered.

119

APPENDIX C

ONE-ON-ONE INTERVIEW PROTOCOL
First-Year Engineering Students’ Learning, Attitudes, and Experiences

Institution: Michigan State University
Interviewee: (Title and Name)

Interviewer:

 

Part I - Student Experiences in the Freshman Year

1. Tell me why you chose to pursue an engineering degree at MSU?

a) What motivated you to pursue an engineering degree?

b) What other options did you consider? Dual major what would you choose?

c) In what ways were you inﬂuenced by others (e. g. family, ﬁiends, high school
teachers)?

2. Now I want to ask you about your transition to MSU. How has your transition to
MSU been?

a) In terms of academic preparation and ﬁt?

b) In terms of developing ﬁiendships and peer groups? Within engineering?

c) To what extent do you feel part of an academic community at MSU? Where is
it based?

d) To what extent do you feel part of a social community at MSU? Where is it

based?

120

 

3. Now I want to spend some time discussing your expectations for your undergraduate
experience here.
a) What expectations did you have for your classroom experiences (e. g.
instruction, projects, etc.)?
To what extent have these expectations been met?
Important to your experience?
b) What expectations did you have for the workload or effort required of you?
To what extent have these expectations been met?
Important to your experience?
c) What expectations did you have for interactions with faculty?
To what extent have these expectations been met?
Important to your experience?
(1) What expectations did you have for interactions with peers in the engineering
college?
To what extent have these expectations been met?
Important to your experience?
e) What did you expect the College of Engineering to be like in general?
To what extent have these expectations been met?
Important to your experience?
f) Any additional expectations that I may not have asked about speciﬁcally?
4. What have been the most signiﬁcant challenges or obstacles to your success here?
5. What types of support have been important to your success? Where did you ﬁnd

them?

121

 

a) For advising?
b) For help with classes?
c) For personal support/friendship?
d) For social opportunities?
6. What ways have you sought support but were not satisﬁed with the result?
7. Looking back on your ﬁrst-year here, describe any changes you would make to the
engineering program? In what ways, could the engineering experience at MSU be

improved to facilitate your success?

Part II — Freshman Experiences in the Cornerstone sequence

Now we would like to ask you a few questions speciﬁcally about the EGR sequence.
1. Describe your experience in EGR 100/102?
2. What are some of the topics you have covered in EGR 100/102?
3. Which of those topics you think are most difficult, and why?
a. Which of those topics did you enjoy the most, and why?
b. From which of those topics did you learn the most, and why?
4. How do you compare your learning in BGR 100/102 with the learning in other
courses?
a. How is this different ﬁ'om what you were expecting before joining the course?
b. Which learning approach do you prefer and why?
5. How has your thinking about yourself changed as a result ofEGR100/102?
a. About how you learn.

b. About your future plans.

122

6. How has your perception about engineering changed since you joined EGR 100/102?
a. About what you think makes a good engineer
b. About what are the advantages/disadvantages of studying engineering

0. About your likes/dislikes about engineering

Thank you for your time.

Would you like to comment on any part or aspect of this interview or add anything that I

might have missed?

123

 

 

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

 

REFERENCES

N. A. 0. Engineering, The engineer of 2020: visions of engineering in the new
century: National Academies Press, 2004.

P. D. Galloway, The ﬁst-century engineer: a proposal for engineering education
reform: American Society of Civil Engineers, 2007.

J. L. Melsa, "Trends in engineering education in the USA," Computing & Control
Engineering Journal, vol. 8, pp. 209-214, 1997.

ﬂash.
B. Seymour and N. M. Hewitt, Talking about leaving: why undergraduates leave
the sciences: Westview Press, 2000.

A. W. Astin, "What matters in college," Liberal Education, vol. 79, pp. 4-15,
1993.

 

M. Besterﬁeld-Sacre, C. J. Atrnan, and L. Shuman, "How ﬁ'eshman attitudes
change in the ﬁrst year," in ASEE Annual Conference, Anaheim, CA, 1995, pp.
1 57-163.

1. B. S. Leonhard E. Bernold, Chris M. Anson, "Understanding Our Students: A
Longitudinal Study of Success and Failure in Engineering With Implications for
Increased Retention," Journal of Engineering education, vol. 96, pp. 263-274,
2007.

B. 8. Bloom, Taxonomy of educational objectives, handbook I : cognitive domain:
Addison Wesley Publishing Company, 1956.

M. Besterﬁeld-Sacre, C. J. Atrnan, and L. J. Shuman, "Characteristics of
ﬁ'eshman engineering students: Models for determining student attrition in
engineering," Journal of Engineering Education, vol. 86, pp. 139-149, 1997.

Q. Li, D. B. McCoach, H. Swarninathan, and J. Tang, "Development of an
instrument to measure perspectives of engineering education among college
students," Journal of Engineering Education, vol. 97, pp. 47-56, 2008.

M. Besterﬁeld—Sacre, M. Moreno, L. J. Shuman, and C. J. Atrnan, "Gender and
ethnicity differences in freshmen engineering student attitudes: A cross-
institutional study," Journal of Engineering Education, vol. 90, pp. 477-489,
2001.

E. Seymour, "Tracking the processes of change in US undergraduate education in
science, mathematics, engineering, and technology," Science Education, vol. 86,
pp. 79-105, 2001.

124

 

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

R. M. Felder, G. N. Felder, M. Mauney, C. B. Hamrin, and Dietz, "A longitudinal
study of engineering student performance and retention. Part III - gender
differences in student performance and attitudes," Journal of Engineering
Education, vol. 84, pp. 151-163, 1995.

A. W. Astin, Green, K. C., Korn,W. S., Schalit, M., "The American freshman:
twenty-year trends, 1966-1985," Higher Education Research Institute, UCLA.,
Los Angeles, CA 1987,

J. A. Krosnick, Boninger, D. 8., Chuang, Y. C., Berent, M. K., Carnot, C. G.,
"Attitude strength: one construct or many related constructs?," Journal of
Personality and Social Psychology, vol. 65, pp. 1132-1151, 1993.

E. M. Pomerantz, S. Chaiken, and R. S. Tordesillas, "Attitude strength and
resistance processes," Journal of Personality and Social Psychology, vol. 69, pp.
408-419, 1995.

M. A. Hutchison-Green, D. K. Follrnan, and G. M. Bodner, "Providing a voice:
qualitative investigation of the impact of a ﬁrst-year engineering experience on

students' efﬁcacy beliefs " Journal of Engineering Education vol. 97, pp. 177-
190, 2008.

J. Burtrrer, "The use of discriminant analysis to investigate the inﬂuence of non-
cognitive factors on engineering school persistence," Journal of Engineering
Education, vol. 94, pp. 335-338, 2005.

M. Besterﬁeld-Sacre and C. J. Atrnan, "Survey design methodology: measuring
ﬁeshman attitudes about engineering," in ASEE Annual Conference, Admonton,
Canada, 1994, pp. 236-243.

S. Sheppard, J enison, Rollie, "Freshmen engineering design experiences and
organizational framework," International Journal of Engineering Education, vol.
13, pp. 190-197, 1997.

S. Sheppard, J enison, Rollie, "Examples of freshman design education,"
International Journal of Engineering Education, vol. 13, pp. 248-261, 1997.

J. D. Burton, White, Daniel M., "Selecting a model for freshman engineering
design," Journal of Engineering Education, vol. 87, pp. 327-332, 1997.

J. W. Dally, Zhang, G. M., "A freshman engineering design course," Journal of
Engineering Education, vol. 82, pp. 83-91, 1993.

"Colleges mine data to predict dropouts," The Chronicle of Higher Education 38,
http://chronicle.conr/ﬁee/v54/i38/38a00103.htm#mining

125

 

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

A. Scalise, Besterﬁeld-Sacre, M., Shuman, L., Wolfe, H., "First term probation:
models for identifying high risk students," in 30th ASEE/IEEE Frontiers in
Education Conference, Kansas City, MO, 2000, pp. ll-16.

M. Besterﬁeld-Sacre, C. J. Atrnan, L. J. Shuman, R. L. Porter, R. M. Felder, and
H. Fuller, "Changes in freshman engineers' attitudes-a cross institutional
comparison: what makes a difference?," in 26th ASEE/IEEE Frontiers in
Education Conference, Salt Lake City, Utah, 1996, pp. 78-82.

N. Al-Holou, Bilgutay, Nihat M., Corleto, Carlos, Demel, John T., Felder,
Richard, Frair, Karen, Froyd, J efferey B., Hoit, Marc, Morgan, Jim, Wells, David
L., "First-year integrated curricula: design alternatives and examples," Journal of
Engineering Education, vol. 87, pp. 435-448, 1999.

G. R. Pike, "The effects of residential learning communities and traditional
residential living arrangements on educational gains during the ﬁrst year of
college," Journal of College Student Development, 1999.

"College of Engineering, Michigan State University",
http://www.egr.msu.edu/w;/about

J. Sticklen, "Center for Engineering Education Research Established at MSU,"
MSU, East Lansing 2009, http://www.emsu.edu/news/2009/08/25/center-
engineering-education-resquch-established-msu

L. L. Seely, "NSF grant to Increase MSU engineering graduates," MSU, East
Lansing 2008, Md/www.egr.msu.edu/news/2008/09/10/nsf-grant-increase-mgr;
ergineering-gadugggs

M. S. University, "The engineering residential experience,
http://www.egr.msu.edu/residential/exmrience," MSU, East Lansing 2008,

T. F. Wolff, "Residential initiative / cornerstone engineering program," MSU,
East Lansing, Pro gram Overview Nov. 27 2006,

J. W. Creswell, "Research design: qualitative, quantitative, and mixed methods
approaches," 3rd. ed LA: SAGE Publications inc., 2009, pp. 203-225.

B. R. Johnson, Onwuegbuzie, Anthony J, "Mixed methods research: a research
paradigm whose time has come," Educational Researcher, vol. 33, pp. 14-26.

M. Borrego, Douglas, E. P., Amelink, C. T. , "Quantitative, qualitative, and
mixed research methods in engineering education," Journal of Engineering
Education, vol. 98, pp. 53-66, 2009.

B. G. Tabachnick and L. S. Fidell, Using multivariate statistics, 2nd ed.
California State University, Northridge: Harper Collins Publishers, Inc., 1989.

126

 

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[57-]

B. L. Berg, Qualitative research methods for the social sciences, 6 ed. Long
Beach CA: Pearson, 2007.

S. Sudman and N. M. Bradburn, Asking questions: a practical guide to
questionnaire design: Jossey-Bass, 1982.

M. Besterﬁeld-Sacre, C. J. Atrnan, and L. J. Shuman, "Engineering student
attitudes assessment," Journal of Engineering Education, vol. 87, pp. 133-141,
1998.

M. Besterﬁeld-Sacre, L. J. Shuman, H. Wolfe, and J. McGourty, "Triangulating
assessments: multi-source feedback systems and closed form survey," in ASEE
Annual Conference, St. Louis, Missouri, 2000.

A. Fink, How to design surveys. CA: Sage Publications, Inc. , 1995.
A. Fink, How to report on surveys. CA: Sage Publication, Inc., 1995.
A. Fink, How to analyze survey data. CA: Sage Publication, Inc., 1995.

R. F. DeVellis, Scale development: theory and application, 2nd ed. CA: Sage
Publications, Inc., 2003.

A. W. Astin, "How " good" is your institutions retention rate?," Research in
Higher Education, vol. 38, pp. 647-658, 1997.

"Assessment tools for attitudes and skills ",
http://www.founclagioncoalition.org/publications/brochures/2002-Feb-
27_Assessment_Attitudes and Skillspdf

M. R. Anderson-Rowland, "A ﬁrst year engineering student survey to assist
recruitment and retention," in 26th ASEE/[BEE Frontiers in Education
Conference, Salt Lake City, Utah, 1996, pp. 372-376

"The Cooperative Institutional Research Program (CIRP) freshman survey,"
2008, http://www.gseis.ucla.edu/heri/PDFs/survexAdmin/tfs/tst8.lnstrument.pdf

M. Besterﬁeld-Sacre, N. Y. Amaya, L. J. Shuman, C. J. Atrnan, and R. L. Porter,
"Understanding student conﬁdence as it relates to ﬁrst year achievement," in 28th
ASEE/IEEE Frontiers in Education Conference, Tempe, AZ, 1998, pp. 258-263.

J. Hilpert, G. Stump, J. Husman, and W. Kim, "An exploratory factor analysis of
the Pittsburgh Freshman Engineering Attitudes Survey," in 38th ASEE/IEEE
Frontiers in Education Conference, Saratoga Springs, NY, 2008, pp. 1-6.

J. Hilpert, G. Stump, J. Husman, W. Kim, W.-T. Chung, and J. Lee, "Steps
toward a sound measure of engineering student attitudes: Pittsburgh Engineering

127

 

[53]

[54]

[55]

[56]

[57]

[58]

[59]

[60]

[611

[62]

[63]

[64]

Attitudes Scale - Revised," in 39th ASEE/IEEE Frontiers in Education
Conference, San Antonio, TX, 2009, pp. 1-6.

"Engineering Criteria 2000",
http://www.ele.uri.edu/facultv/dalv/criteria.2000.html

M. Besterﬁeld-Sacre, L. J. Shuman, H. Wolfe, C. J. Atrnan, J. McGourty, R. L.
Miller, B. M. Olds, and G. M. Rogers, "Deﬁning the outcomes: a ﬁamework for
EC-2000," IEEE Transactions on Education, vol. 43, pp. 100-110, 2000.

M. Besterﬁeld-Sacre, M. Moreno, L. J. Shuman, and C. J. Atrnan, "Comparing
entering freshman engineers: institutional differences in student attitudes," in
ASEE Annual Conference Charlotte NC, 1999.

M. Besterﬁeld Sacre, "Personal e-mail communication dated May 30, 2008 on
factor loadings with author of PFEAS ", Q. Malik, Ed. E. Lansing, 2008.

M. Besterﬁeld Sacre, "Personal e-mail communication dated Oct 9, 2008 on
Cronbach Alpha (CA) with author of PF EAS," Q. Malik, Ed. East Lansing, MI,
2008.

J. A. Gliem, Gliem, Rosemary R. , "Calculating, Interpreting, and Reporting
Cronbach’s Alpha Reliability Coefﬁcient for Likert-Type Scales," in Midwest
Research to Practice Conference in Adult, Continuing, and Community Education
The Ohio State University, Columbus, OH, , 2003.

D. G. Garson, "Scales and standard measures," in Quantitative Research in Public
Administration, 2008.

S. F. Olejnik and J. Algina, "Parametric ANCOVA and the rank transform
ANCOVA when the data are conditionally non-normal and heteroscedastic,"
Journal of Educational Statistics, vol. 9, pp. 129-149, 1984.

E. Cohn, S. Cohn, D. C. Balch, and J. J. Bradley, "Determinants of undergraduate
GPAs: SAT scores, high-school GPA and high-school rank," Economics of
Education Review vol. 23 pp. 577-586, 2004.

"SAT-ACT conversion chart," http://www.princetonreview.com/, 2008.

W. J. Conover and R. L. Irnan, "Analysis of covariance using the rank
transformation," Biometrics, vol. 38, pp. 715-724, Sep. 1982.

C. Collins and S. J. Pierce, "Evaluating regression models: understanding and
using regression diagnostics to improve your analyses," in Evaluation 2008:
Evaluation Policy and Evaluation Practice, the annual conference of the
American Evaluation Association Denver, CO, 2008.

128

 

[65]

[66]

[67]

[68]

[69]

[70]

[71]

[72]

[73]

[74]

[75]

[76]

[77]

R. V. Hogg and E. A. Tanis, "Sampling distribution theory," in Probability and
statistical inference, 6 ed, 8. Yagan, Ed. Upper Saddle river, NJ: Prentice Hall,
Inc., 2001, p. 704.

J. Fox, Applied regression analysis, linear models, and related models: SAGE
Publications, Inc., 1997.

S. F. Olejnik and J. Algina, "A review of nonparametric alternatives to analysis of
covariance," Evaluation Review, vol. 9, pp. 51-83, 1985.

Q. H. Malik, Koehler, Mathew J ., Mishra, Punya, Buch, Neeraj, Shanblatt,
Michael A, Pierce, Steven J. , "Participation in a freshman design sequence and
its inﬂuence on students’ attitudes towards engineering," in 39th ASEE/IEEE
Frontiers in Education Conference, San Antonio, TX, 2009.

Q. H. Malik, Koehler, Mathew J ., Mishra, Punya, Buch, Neeraj, Shanblatt,
Michael A, Pierce, Steven J ., "Understanding student attitudes in a ﬁeshman
design sequence," International Journal of Engineering Education (accepted for
publication), 2010.

Q. H. Malik, Koehler, Mathew J ., Mishra, Punya, Buch, Neeraj, Shanblatt,
Michael A., "Understanding freshman perceptions about engineering," in ASEE
North Central Section Conference, Grand Rapids, MI, 2009.

M. S. Deberard, G. I. Spielmans, and D. L. Julka, "Predictors of academic
achievement and retention among college freshmen: a longitudinal study,"
College Student Journal, vol. 38, pp. 66-80, 2004.

M. B. Miles and A. M. Huberrnan, Qualitative data analysis: a sourcebook of
new methods: vol. . CA: Beverly Hills: Sage Publications, 1984.

M. Koro-Ljungberg and E. P. Douglas, "State of qualitative research in
engineering education: meta analysisof J EB articles, 2005-2006," Jormal of
Engineering Education, vol. 97, pp. 163-176, 2008.

N. K. Denzin and Lincoln. Y. S., The discipline and practice of qualitative
research, 3rd ed. Thousand Oaks, CA: Sage Publications, Inc., 2005.

J. W. Creswell, Clark, Vicki L Plano, Designing and conducting mixed methods
research 2007 ed. Thousands Oaks: Sage Publications Inc., 2007.

R. M. Felder, Sheppard, Sheri D., Smith, Karl A., "A new journal for a ﬁeld in
transition," Journal of Engineering Education, vol. 94, pp. 7-10, January 2005
1995.

Q. H. Malik, Koehler, Mathew J ., Mishra, Punya, Buch, Neeraj, Shanblatt,
Michael A., "Understanding student attitudes in a freshman design sequence - a
qualitative study (accepted)," in ASEE Annual Conference Louseville, KY, 2010.

129

 

[78]

[79]

[80]

[81]

[82]

N. 8. Board, "Science and engineering indicators 2006," N. 8. Board, Bd.: NSF,
2006.

K. S. Kurasaki, "Intercoder reliability for validating conclusions drawn ﬁom
open-ended interview data," Field Methods, vol. 12, pp. 179-194, 2000.

C. Glesne, Becoming qualitative researchers: an introduction, 5th ed. Boston:
Pearson/Ally & Bacon, 2006.

R. E. Petty and j. A. Krosnick, Attitude strength: antecedents and consequences.
Mahwah, NJ: Lawrence Erlbaum Associates, 1995.

J. D. Singer and J. B. Willett, Applied longitudinal data analysis: modeling
change and event occurrence. New York: Oxford University Press, 2003.

130

 

   

"'iliiiliililiiilil[iiiiiliiiiﬂijiif