Psychology 3840F-001

Research in Test Construction

If there is a discrepancy between the outline posted below and the outline posted on the OWL course website, the latter shall prevail.

1.0    CALENDAR DESCRIPTION

A review of methods for developing psychological tests, questionnaires, and surveys. Topics will include reliability, validity, norming, standardization, and strategies of test construction. Opportunities for developing measures of human abilities and personality will be provided. Strongly recommended for students planning graduate training in psychology and related disciplines.  3 lecture hours, 0.5 course.
Antirequisites: Psychology 2080A/B.
Antirequisites are courses that overlap sufficiently in content that only one can be taken for credit. So if you take a course that is an antirequisite to a course previously taken, you will
lose credit for the earlier course, regardless of the grade achieved in the most recent course.
Prerequisites:  Psychology 2800E and 2810, plus registration in third or fourth year
Honors Specialization in Psychology or Honors Specialization in Developmental Cognitive Neuroscience.  Third or fourth year Psychology Majors and Psychology
Special Students who earn 70% or higher in Psychology 2820E or 60% or higher in Psychology 2800E and 2810 or 60% higher in both Psychology 2839A/B and 3830F/G
(at Huron) also may enrol in this course.
Unless you have either the requisites for this course or written special permission from your Dean to enroll in it, you may be removed from this course and it will be deleted from
your record.  This decision may not be appealed.  You will receive no adjustment to your fees in the event that you are dropped from a course for failing to have the prerequisites.

2.0    COURSE INFORMATION

Instructor:    S. Paunonen (paunonen@uwo.ca  519-661-2060) Lectures: Thursdays, 2:30-5:30, Room 3120 SSC
Office Hours:    Mondays, 4:30-5:30, Room 6304 SSC Te aching Assistant: Rachel Plouffe (rplouffe@uwo.ca)
Office Hours:    Wednesdays, 1:00-3:00, Room 6301 SSC

If you or someone you know is experiencing distress, there are several resources here at Western to assist you.  Please visit:  http://www.uwo.ca/uwocom/mentalhealth/ for more information on these resources and on mental health.

Please contact the course instructor if you require material in an alternate format or if you require any other arrangements to make this course more accessible to you.  You may also wish to contact Services for Students with Disabilities (SSD) at 519-661-2111 ext 82147 for any specific question regarding an accommodation.

3.0  TEXTBOOK

Paunonen, S. V.  Psychology 3840 Lecture Notes.  UWO Bookstore.

4.0    COURSE OBJECTIVES

The primary objective of this course is to introduce the student to the major methods of psychological test construction.  Those methods will be examined in terms of their underlying psychometric theory related to reliability and validity estimation, standardization, and equating. Students will write and evaluate test items by constructing an in-class personality questionnaire.

5.0     EVALUATION

Students will be required to write one 2-hour midterm exam and one 3-hour final exam.  Exam items will be of the short answer, short essay, and problem-solving variety.  The final exam will be cumulative in its content coverage.  The graded midterm along with a complete exam scoring key can be examined with the TA.  In addition to the two exams, a written project (2500 words) on item analysis will be assigned.  Details of that project will be available on a separate handout.

Although the Psychology Department does not require instructors to adjust their course grades to conform to specific targets, the expectation is that course marks will be distributed around the following averages:


70%    1000-level and 2000-level courses
72%     2190-2990 level courses
75%     3000-level courses
80%     4000-level courses
   
The Psychology Department follows the University of Western Ontario grading guidelines, which are as follows (see http://www.uwo.ca/univsec/handbook/general/grades_undergrad.pdf):

A+    90-100    One could scarcely expect better from a student at this level
A    80-89        Superior work that is clearly above average
B    70-79        Good work, meeting all requirements, and eminently satisfactory
C    60-69        Competent work, meeting requirements
D    50-59        Fair work, minimally acceptable
F    below 50    Fail


6.0  TEST AND EXAMINATION SCHEDULE

WEIGHT    LOCATION
Midterm    Oct. 22    .30    EC 2155
Project    Dec. 3    .30 (-5% p.d. late)    6304 SSC
Final    Dec. 11-22    .40    tba

7.0   CLASS SCHEDULE

The lecture topics and the assigned readings are listed at the end of this outline.  Most readings can be found online.  Some readings and the Lecture Notes are sold through the Bookstore.


8.0     STATEMENT ON ACADEMIC OFFENCES

Students are responsible for understanding the nature and avoiding the occurrence of plagiarism and other scholastic offenses. Plagiarism and cheating are considered very serious offenses because they undermine the integrity of research and education. Actions constituting a scholastic offense are described at the following link:  http://www.uwo.ca/univsec/pdf/academic_policies/appeals/scholastic_discipline_undergrad.pdf

As of Sept. 1, 2009, the Department of Psychology will take the following steps to detect scholastic offenses. All multiple-choice tests and exams will be checked for similarities in the pattern of responses using reliable software, and records will be made of student seating locations in all tests and exams. All written assignments will be submitted to TurnItIn, a service designed to detect and deter plagiarism by comparing written material to over 5 billion pages of content located on the Internet or in TurnItIn’s databases. All papers submitted for such checking will be included as source documents in the reference database for the purpose of detecting plagiarism of papers subsequently submitted to the system. Use of the service is subject to the licensing agreement, currently between Western and Turnitin.com http://www.turnitin.com

Possible penalties for a scholastic offense include failure of the assignment, failure of the course, suspension from the University, and expulsion from the University.

9.0    POLICY ON ACCOMMODATION FOR MEDICAL ILLNESS

The University of Western Ontario’s policy on Accommodation for Medical Illness can be found at:
http://www.westerncalendar.uwo.ca/2015/pg117.html

Students must see the Academic Counsellor and submit all required documentation in order to be approved for certain accommodation:
http://counselling.ssc.uwo.ca/procedures/medical_accommodation.html


10.0        OTHER INFORMATION

MAKE-UP EXAMS:

1.    Only under special circumstances will a student be allowed to write a make-up
test or exam or be given an extension on an assignment.  These include medical or
compassionate reasons substantiated by the proper documentation (see above) approved
by your faculty’s Academic Counselling Office.

2.    If the missed exam is the final examination (set by the Registrar’s Office) the student must contact the Academic Counselling Office to request a permission form for writing a Special Final Examination.  That form must be signed by the instructor, the department chairperson, and the Dean.

3.    A student missing an exam or assignment must notify the instructor on the day of the exam or deadline, or on the following day at the latest.

4.    The student should try to notify the instructor in person or by telephone, rather than by email.  If leaving a phone mail message, be sure to leave your telephone number.

5.    A student who misses an exam or deadline for other than medical or compassion reasons, or who is unable to substantiate a claim, or who fails to notify the instructor by the day following the exam or deadline will be assigned a grade of zero for that evaluation.

6.    Make-up exams or extensions to deadlines will not be scheduled to accommodate students’ travel or work plans.

ELECTRONIC AIDS:

The only electronic aids allowed during the exams are basic basic standalone calculators. Computers, cell phones, PDAs, MP3 players or other digital devices will not be permitted.

Office of the Registrar web site:  http://registrar.uwo.ca

Student Development Services web site: http://www.sdc.uwo.ca

Please see the Psychology Undergraduate web site for information on the following:

    http://psychology.uwo.ca/undergraduate/student_responsibilities/index.html

- Policy on Cheating and Academic Misconduct
- Procedures for Appealing Academic Evaluations
- Policy on Attendance
- Policy Regarding Makeup Exams and Extensions of Deadlines
- Policy for Assignments
- Short Absences

- Extended Absences
- Documentation
- Academic Concerns
- 2015 Calendar References

No electronic devices, including cell phones, will be allowed during exams.

NOTE:
-Each reading’s online file name is underlined, but...
-Kline and Dick & Hagerty must be purchased from the
bookstore due to copyright restrictions
-Note the skipped/skimmed pages where applicable
-The only formulas you need are on the Formula Sheet


1. INTRODUCTION

Kline,  Chapter 1 - The characteristics of good tests in psychology.  (pp. 1-16) Chapter 3 - Making tests reliable II:  Personality inventories.  (pp. 58-75)

Jackson, D. N.  (1970).  A sequential system for personality scale development.
In C. D. Spielberger (Ed.), Current topics in clinical and community psychology
(Vol. 2, pp. 61-96).  New York:  Academic Press.  (pp. 72-96)

Paunonen, S. V., & Ashton, M. C.  (2002).  The nonverbal assessment of personality: The NPQ and the FF-NPQ.  In B. De Raad & M. Perugini (Eds.), Big Five assessment (pp. 171-194).  Göttingen, Germany: Hogrefe & Huber.  (pp. 171-183)


2. CORRELATION, FACTOR ANALYSIS

Wiggins, J. S.  (1973).  Personality and prediction:  Principles of personality assessment.
Reading, MA:  Addison-Wesley.  (pp. 6-30; skim formulas)

Edwards, A. L.  (1970).  The measurement of personality traits by scales and inventories.
New York:  Holt, Rinehart, & Winston.  (pp. 69-87; skim computations in
sections 6.7, 6.8, 6.10, 6.12, 6.14)


3. RELIABILITY

Kline,  Chapter 5 - Computing test reliability.  (pp. 118-129; skip pp. 127-128)

Allen, M. J., & Yen, W. M.  (1979).  Introduction to measurement theory.  Monterey, CA: Brooks/Cole.  (pp. 56-70; Proofs are optional)

Dick, W., & Hagerty, W.  (1971).  Topics in measurement.  New York: McGraw-Hill. (pp. 1-56)

Paunonen, S. V.  (1984).  The reliability of aggregated measurements:  Lessons
to be learned from psychometric theory.  Journal of Research in Personality,
18, 383-385.


<<<<<<    MIDTERM EXAM    >>>>>>
 


4. VALIDITY

Cronbach, L. J., & Meehl, P. E.  (1955).  Construct validity in psychological tests.  Psychological Bulletin, 52, 281-302.  (skip pp. 290-299)

Campbell, D. T., & Fiske, D. W.  (1959).  Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.  (abridged)


5. ITEM ANALYSIS

Holden, R. R., & Jackson, D. N.  (1979).  Item subtlety and face validity in personality assessment. Journal of Consulting and Clinical Psychology, 47, 459-468.

Jackson, D. N.  (1970).  A sequential system for personality scale development.
In C. D. Spielberger (Ed.), Current topics in clinical and community psychology
(Vol. 2, pp. 61-96).  New York:  Academic Press.  (pp. 72-96)

Paunonen, S. V., & Ashton, M. C.  (2002).  The nonverbal assessment of personality: The NPQ and the FF-NPQ.  In B. De Raad & M. Perugini (Eds.), Big Five assessment (pp. 171-194).  Göttingen, Germany: Hogrefe & Huber.  (pp. 171-183)


6. STANDARDIZATION

Seashore, H. G.  (1955).  Methods of expressing test scores.  Test Service
Bulletin, No. 45.  The Psychological Corporation.  (pp. 7-10)

Ghiselli, E. E., Campbell, J. P., & Zedeck, S.  (1981). Measurement theory
for the behavioral sciences.  San Francisco: W. H. Freeman.  (pp. 59-75)


7. COMPARISON OF METHODS
Kline,  Chapter 9 - Other methods of test construction.  (pp. 175-190) Nunnally, J. C.  (1978).  Psychometric theory.  New York: McGraw-Hill.
(pp. 265-274)

Burisch, M.  (1984).  Approaches to personality inventory construction.
American Psychologist, 39, 214-227.


8. INTRODUCTION TO CAT AND IRT (optional)

MacDonald, P.  (2002).  Computer Adaptive Testing for measuring personality factors using Item Response Theory.  Ph.D dissertation, U.W.O.  (pp. 1-34)