Psychology 9540

Research Design and Statistical Modeling

Instructor: Paul F. Tremblay, Ph.D.

email: ptrembla@uwo.ca

 

office hours: flexible; contact me by email to set appointment

Summary of online course structure:
Lectures: Asynchronous material consisting mainly of narrated video material with slides.
Weekly meetings: Weekly Zoom meetings to discuss bi-weekly lab assignments (schedule TBA).
Course work: Bi-weekly (12 in total) written assignments (90%); Submitting a weekly question or thought (10%).
Material: Suggested books and secondary journal articles and book chapters all available electronically through our Western library.

 I. COURSE DESCRIPTION

My personal objective in this course is to help you develop your skills, experience, and confidence to independently design and conduct research from start to finish. In order to reach that objective, I am convinced that we must concentrate on three pillars of research: research design, measurement, and statistical analysis. In his book, Becoming a Behavioral Science Researcher (2020), Professor Rex B. Kline from Concordia University refers to these three “pillars” as the trinity of research.

The course content consists of 24 topics organized into four general units:
• I. Fundamental and descriptive statistics (estimation, sampling distributions, data visualization and management, missing data techniques, inferential statistics, confidence intervals, effect size, replication, power analysis)
• II. General linear model and experimental designs with ANOVA, ANCOVA and MANOVA,
• III. Multiple Regression and Extensions (including mediation, moderation, multilevel modeling, and models for categorical outcomes such as logistic regression)
• IV. Factor Analysis and Structural Equation Modeling with applications to measurement, test construction and construct validation.

The course work consists mostly of lab assignments that will provide hands-on training by having you generate hypotheses, analyze data, interpret and report results, conduct simulations, write mini research proposals, and evaluate published research. Students will have the flexibility to work with their preferred software. My course lectures and demonstrations include presentations in R (and the related jamovi and jasp applications with
easy to use graphic user interface), SPSS, and Mplus (for multilevel and structural equation modeling).

II. METHOD OF EVALUATION

Assignments: (90% of course grade).
• Background preparation for labs. Although I have never used formal tests or exams in this course, the lab assignments capture knowledge and ability to apply the content presented in the lectures. Students are responsible for keeping up with the weekly material.
• 12 bi-weekly assignments. The lecture schedule indicates when the assignments will be distributed, and you will have two weeks to complete an assignment. I will conduct weekly Zoom meetings with the class to explain the assignments and to answer questions.
• Most assignments will include data analysis, interpretation, discussion of results, and writing brief reports in APA style. A few assignments consist of developing mini research proposals and conducting power analyses or evaluating the procedures used in an article.
• Assignment reports will typically consist of a two double-spaced page write-up including a description, interpretation and discussion of your results, answers to specific questions, and an appendix with your analysis output.
• Late assignments will receive a 5% deduction per 24 hours. Assignments that are more than one week late will not be accepted for partial marks.
• Rules about working in groups. I am supportive of students working in groups to conduct the analyses and discuss the assignments. However, you are required to write your own report with no duplication from your colleagues’ work. The assignments will often require you to choose a subset of variables, to make decisions about plausible strategies, or to describe research ideas from your own area of interest. Also, some questions will ask you to design your own hypothetical research designs. As a result, it is unlikely that two students will work with the exact same material.

Weekly submission of a follow-up question or thought (10% of course grade).
We will use the Message feature in the course OWL site for you to submit to me a weekly question, thought, or comment about the previous lecture. I answer all these questions individually and often incorporate them as a follow-up in subsequent lectures.

 III. LECTURE SCHEDULE

 

 

 

Date

 

Topic

 

Readings*

 

Lab

 

1

 

Sep 16

 

Overview

 

Ch.1, K1, K2

 

 

2

 

Sep 23

 

Basic Statistics and Distributions - Visualization

 

Ch. 2,3,4

 

A 1

 

3

 

Sep 30

 

Inferential Statistics – Simulation and Power analysis

 

Ch. 5, 6, K3

 

 

4

 

Oct 7

 

Data Inspection and Missing Data Analysis

 

K4, K9

 

A 2

 

5

 

Oct 14

 

t-tests and Effect Size

 

Ch. 7, K5

 

 

6

 

Oct 21

 

One-way ANOVA

 

Ch. 11, 9, 12, K6

 

A 3

 

7

 

Oct 28

 

Factorial ANOVA

 

Ch. 13, K7

 

 

8

 

Nov 11

 

Repeated Measures and Trends

 

Ch. 15

 

A 4

 

9

 

Nov 18

 

Split Plot ANOVA and Other Longitudinal Methods

 

Ch. 15

 

 

10

 

Nov 25

 

Measures of Association (including Chi-Square)

 

Ch. 8, 10

 

A 5

 

11

 

Dec 2

 

Introduction to Meta-Analysis

 

See list

 

 

12

 

Dec 9

 

Bivariate Linear Regression

 

Ch. 17

 

A 6

 

13

 

Jan 13

 

Multiple Correlation – Understanding “Statistical Control”

 

Ch. 18

 

 

14

 

Jan 20

 

Multiple Regression

 

Ch. 18

 

A7

 

15

 

Jan 27

 

Categorical Predictors in Multiple Regression

 

Ch.

 

 

16

 

Feb 3

 

ANCOVA – Mix of Categorical and Continuous Predictors

 

Ch. 14

 

A8

 

17

 

Feb 10

 

Moderation in Multiple Regression

 

Ch. 20

 

 

18

 

Feb 24

 

Mediation in Multiple Regression

 

Ch. 20

 

A9

 

19

 

Mar 3

 

Logistic Regression and Other Regression Models

 

Ch. 19

 

 

20

 

Mar 10

 

Multilevel Modeling-I – Subjects Within Groups

 

See list

 

A10

 

21

 

Mar 17

 

Multilevel Modeling-II – Observations within Individuals

 

See list

 

 

22

 

Mar 24

 

Factor Analysis

 

Ch.

 

A11

 

23

 

Mar 31

 

Confirmatory Factor Analysis and Structural Equation Mod

 

See list

 

 

24

 

Apr 7

 

Measurement Theory (Classical and Item Response)

 

K8

 

A12

 

*Readings. Ch refers to Hahs-Vaughn & Lomax (2020); K refers to Kline (2020)

 

IV. STATEMENT OF ACADEMIC OFFENCES

Scholastic offences are taken seriously and students are directed to read the appropriate policy, specifically, the definition of what constitutes a Scholastic Offence, at the following Web site: http://www.uwo.ca/univsec/pdf/academic_policies/appeals/scholastic_discipline_grad.pdf

All required papers may be subject to submission for textual similarity review to the commercial plagiarism-detection software under license to the University for the detection of plagiarism. All papers submitted for such checking will be included as source documents in the reference database for the purpose of detecting plagiarism of papers subsequently submitted to the system. Use of the service is subject to the licensing agreement, currently between The University of Western Ontario and Turnitin.com (http://www.turnitin.com).


V. COURSE READINGS

I will use the two textbooks listed below throughout the course. These are available online through our Western library and can be accessed through the OWL course page. You will be able to download sections in pdf. (If you prefer your own hard copy, the books are available at amazon.ca).

Hahs-Vaunghn, D. L. & Lomax, R. G. (2020). An introduction to statistical concepts. Fourth Edition. Routledge. 978-1138650558
Kline, R. B. (2019). Becoming a Behavioral Science Researcher (Second edition). Guilford Press.
978-1462541287
A list of supplementary articles and book chapters (available electronically through library system or in the OWL course website) are listed below by lecture topic. These are additional resources that may serve you beyond this course in your own research. I will discuss most of these in my lecture material.

1. Overview

Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., Rao, S. M., & Clinic, C. (2018). Journal article reporting standards for quantitative research in Psychology: The APA Publications and Communications Board Task Force Report. American Psychologist, 73(1), 3–25. http://dx.doi.org/10.1037/amp0000191

Morling, B., & Calin-Jageman, R. J. (2020). What psychology teachers should know about open science and the new statistics. Teaching of Psychology, 47, 169-179. doi: 10.1177/0098628320901372

Smith, E. R. (2014). Research design. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology. 2nd edition. (p. 27–48). Cambridge University Press.

2. Basic Statistics and Distributions

Cumming G., & Finch, S. (2005). Inference by eye. Confidence intervals and how to read pictures of data.
American Psychologist, 60, 170-180. doi: 10.1037/0003-066X.60.2.170

DeCarlo, L. T. (1997). On the meaning and use of kurtosis. Psychological Methods, 2, 292-307.

3. Inferential Statistics and Power analysis

Amrhein, V., Greenland, S., & McShane, B. (2019). Retire statistical significance (Comment). Nature, 567, 305-307.

Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008). Sample size planning for statistical power and accuracy in parameter estimation. Annual Review of Psychology, 59, 537-563. doi: 10.1146/annurev.psych.59.103006.093735

Kruschke, J. K., & Liddell, T. M. (2017). Bayesian data analysis for newcomers. Psychonomic Bulletin & Review (published online). doi: 10.3758/s13423-017-1272-1

Hesterberg, T. C. (2015) What teachers should know about the bootstrap: resampling in the undergraduate statistics curriculum, The American Statistician, 69, 371-386. http://dx.doi.org/10.1080/00031305.2015.1089789

4. Data Inspection and Missing Data Analysis

Baraldi, A. N., & Enders, C. K. (2010). An introduction to modern missing data analyses. Journal of School Psychology, 48, 5–37. doi: 10.1016/j.jsp.2009.10.001

Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549–576. doi: 10.1146/annurev.psych.58.110405.085530

5. T-tests and Effect Sizes

Kelley, K., & Preacher, K. J., (2012). On effect size. Psychological Methods, 17, 137-152. doi: 10.1037/a0028086

Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. frontiers in Psychology. doi: 10.3389/fpsyg.2013.00863

Lakens, D., Scheel, A. M., & Isager, P. M. (2018). Equivalence testing for psychological research: A Tutorial. Advances in Methods and Practices in Psychological Science, 1, 259-269. doi: 10.1177/2515245918770963

6. One-way ANOVA

Sauder, D. C., & DeMars C. E. (2019). An Updated recommendation for multiple comparisons. Advances in Methods and Practices in Psychological Science, 2, 26-44. doi:10.1177/2515245918808784

Smith, E. R. (2014). Research design. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology. 2nd edition. (p. 27–48). Cambridge University Press.

7. Factorial ANOVA
Spinner, B., & Gabriel, R. M. (1981). Factorial analysis of variance with unequal cell frequencies.
Canadian Psychology, 22, 260-270.

Pierce, C. A., Block, R. A., & Aguinis, H. (2004). Cautionary note on reporting eta-squared values from multifactor ANOVA designs. Educational and Psychological Measurement, 64, 916-924. doi: 10.1177/0013164404264848

8. Repeated Measures (Within subjects design)

Atkinson, G. (2001). Analysis of repeated measurements in physical therapy research. Physical Therapy in Sports, 2, 194-208. doi: 10.1054/ptsp.2001.0071

9. Split Plot ANOVA (a.k.a. Mixed ANOVA designs) and Overview of other Longitudinal Methods

Gibbons, R. D., Hedeker, D., & DuToit, S. (2010). Advances in analysis of longitudinal data. Annual Review of Clinical Psychology, 6, 79-107. doi: 10.1146/annurev.clinpsy.032408.153550

10. Measures of Association

de Winter, J. C. F., Gosling, S. D., & Potter, J. (2016). Comparing the Pearson and Spearman correlation coefficients across distributions and sample sizes: A tutorial using simulations and empirical data. Psychological Methods, 21, 273-290. http://dx.doi.org/10.1037/met0000079

Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149-1160. doi:10.3758/BRM.41.4.1149

11. Introduction to Meta-Analysis

Cheung, M. W. L., & Vijayakumar, R. (2016). A guide to conducting a meta-analysis. Neuropsychology Review, 26, 121-128. doi: 10.1007/s11065-016-9319-z

Johnson, B. T., & Eagly, A. H. (2014). Meta-analysis of research in social and personality psychology. In H.
T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (2nd Ed., pp. 675-707). London: Cambridge University Press.

Schäfer T., & Schwarz, M. A. (2019). The Meaningfulness of effect sizes in psychological research: Differences between sub-disciplines and the impact of potential biases. Frontiers in Psychology, 10, 813. doi: 10.3389/fpsyg.2019.00813

12. Bivariate Linear Regression

Lorenz, F. O. (1987). Teaching about influence in simple regression. Teaching Sociology, 15, 173-177. https://www.jstor.org/stable/1318032

13. Multiple Correlation – Understanding “Statistical Control”

Becker, T. E. (2005). Potential problems in the statistical control of variables in organizational research: A qualitative analysis with recommendations. Organizational Research Methods, 8, 274-289. doi: 10.1177/1094428105278021

14. Multiple Regression

Hoyt, W. T., Leierer, S., & Millington, M. J. (2006). Analysis and interpretation of findings using multiple regression techniques. Rehabilitation Counseling Bulletin, 49, 223-233.

Hoyt, W. T., Imel, Z. E., & Chan, F. (2008). Multiple regression and correlation techniques: Recent controversies and best practices. Rehabilitation Psychology, 53, 321-339. doi: 10.1037/a0013021

Williams, M. N., Gomez Grajales, C. A., & Kurkiewicz, D. (2013). Assumptions of multiple regression. Correcting two misconceptions. Practical Assessment, Research & Evaluation, 18(11). Available online: https://pareonline.net/getvn.asp?v=18&n=11.

15. Categorical Predictors in Multiple Regression

Wendorf, C. A. (2004). Primer on multiple regression coding: Common forms and the additional case of repeated contrasts. Understanding Statistics, 3, 47-57.

16. ANCOVA – Mix of Categorical and Continuous Predictors

Miller, G. A., & Chapman, J. P. (2001). Misunderstanding analysis of covariance. Journal of Abnormal Psychology, 110, 40-48. doi: 10.1037//0021-843X.110.1.40

17. Moderation in Multiple Regression

Hayes, A. F., & Rockwood, N. J., (2016). Regression based statistical mediation and moderation analysis in clinical research: Observations, recommendations and implementation. Behaviour Research and Therapy, 1-19. http://dx.doi.org/10.1016/j.brat.2016.11.001

18. Mediation in Multiple Regression

Hayes, A. F., & Rockwood, N. J., (2016). Regression based statistical mediation and moderation analysis in clinical research: Observations, recommendations and implementation. Behaviour Research and Therapy, 1-19. http://dx.doi.org/10.1016/j.brat.2016.11.001

19. Logistic Regression and Other Regression Models

Coxe, S., West, S. G., Aiken L. S. (2009). The analysis of count data: A gentle introduction to poisson regression and its alternatives. Journal of Personality Assessment, 91, 121-136. doi: 10.1080/00223890802634175

Huang, F. L., & Moon, T. R. (2013). What are the odds of that? A primer on understanding logistic regression. Gifted Child Quarterly, 57, 197-204. doi: 10.1177/0016986213490022


20. Multilevel Modeling-I – Subjects Within Groups

Huang, F. L. (2018). Multilevel modeling myths. School Psychology Quarterly, 33, 492-499. http://dx.doi.org/10.1037/spq0000272

Nezlek, J. B. (2008). An introduction to multilevel modeling for social and personality psychology. Social and Personality Psychology Compass, 2(2), 842-860.

Peugh, J. L. (2010). A practical guide to multilevel modeling. Journal of School Psychology, 48, 85-112. doi:10.1016/j.jsp.2009.09.002

21. Multilevel Modeling-II – Observations within Individuals

Peugh, J. L. (2010). A practical guide to multilevel modeling. Journal of School Psychology, 48, 85-112. doi:10.1016/j.jsp.2009.09.002

Gibbons, R. D., Hedeker, D., & DuToit, S. (2010). Advances in analysis of longitudinal data. Annual Review of Clinical Psychology, 6, 79-107. doi: 10.1146/annurev.clinpsy.032408.153550

22. Factor Analysis

Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272-299. doi: 10.1037//1082-989X.4.3.272

Preacher, K. J., & MacCallum, R. C. (2003). Repairing Tom Swift’s electric factor analysis machine.
Understanding Statistics, 2, 13-43.

Watkins, M. W. (2018). Exploratory factor analysis: A guide to best practice. Journal of Black Psychology, 44, 219-246. doi: 10.1177/0095798418771807

23. Confirmatory Factor Analysis and Structural Equation Modeling

Weston, R. & Gore Jr, P. A. (2006). A brief guide to structural equation modeling. The Counseling Psychologist, 34, 719-751. doi: 10.1177/0011000006286345

Whitley, B. E., & Kite, M. E. (2018). Principles of Research in Behavioral Science. Fourth Edition. NY: Routledge. See chapter 12: Factor analysis, path analysis, and structural equation modeling.

24. Measurement Theory (Classical and Item Response)

DeVellis, R. F. (2006). Classical test theory. Medical Care, 44, S50 S59. http://www.jstor.org/stable/41219505

Toland, M. D. (2014). Practical guide to conducting an item response theory analysis. Journal of Early Adolescence, 34, 120-151. doi: 10.1177/0272431613511332

Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development.
Psychological Assessment, 7, 309-319. doi: 10.1037/1040-3590.7.3.309

Clark, L. A., & Watson, D. (2019, March 21). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment. Advance online publication. http://dx.doi.org/10.1037/pas0000626