Psychology 4190F-001
Special Topics in Sensation and Perception
"Multisensory Integration"
If there is a discrepancy between the outline posted below and the outline posted on the OWL course website, the latter shall prevail.
1.0 CALENDAR DESCRIPTION
This course will cover how our perceptual systems combine sensory information between sensory modalities to form cohesive perceptions of the external world, including vision, audition, and touch, among others. Topics covered will include how multisensory integration is measured, its neurological underpinnings, impacts on cognitive processes, changes across the lifespan, and atypical multisensory integration as observed in clinical populations. 3 lecture hours, 0.5 course.
Antirequisite: Not Applicable.
Antirequisites are courses that overlap sufficiently in content that only one can be taken for credit. So if you take a course that is an antirequisite to a course previously taken, you will lose credit for the earlier course, regardless of the grade achieved in the most recent course.
Prerequisite: Psychology 2220A/B or Psychology 2221A/B and registration in third or fourth year Honors Specialization in Psychology, Honors Specialization in Developmental Cognitive Neuroscience, Honors Specialization in Neuroscience, or Honors Specialization in Animal Behavior. Other Psychology students and Psychology Special Students who receive 70% in the prerequisite course(s) may enroll in this course.
3 lecture hours; Course Weight: 0.5.
Unless you have either the prerequisites for this course or written special permission from your Dean to enrol in it, you may be removed from this course and it will be deleted from your record. This decision may not be appealed. You will receive no adjustment to your fees in the event that you are dropped from a course for failing to have the necessary prerequisites.
2.0 COURSE INFORMATION
Instructor: R. Stevenson
Office: WIRB 5146
Office Hours: By Appointment
Email: rsteve28@uwo.ca
Time and Location of Classes: Thursdays, 14:30-17:30, Room 1160 WIRB
If you or someone you know is experiencing distress, there are several resources here at Western to assist you. Please visit: http://www.uwo.ca/uwocom/mentalhealth/ for more information on these resources and on mental health.
Please contact the course instructor if you require material in an alternate format or if you require any other arrangements to make this course more accessible to you. You may also wish to contact Services for Students with Disabilities (SSD) at 519-661-2111 ext 82147 for any specific question regarding an accommodation.
3.0 TEXTBOOK
All required readings for this course will be academic journal articles posted on the course’s OWL website. You will also need a computer account, which gives you access to the UWO network.
4.0 COURSE OBJECTIVES
Historically, perceptual research has focused on studying individual sensory modalities (vision, audition, proprioception, vestibular inputs) in isolation. However, more recent investigations have begun to consider how these sensory inputs are integrated in the brain. In this course we will review the basic tenets of multisensory integration and review supporting empirical research using physiological, psychophysical/behavioural, and computational modelling techniques. We will also consider how multisensory integration changes throughout development and aging and how it may be affected by sensory loss and cognitive loss.
The review assignment will provide students with the opportunity to deeply explore a question relevant to the multisensory integration, and to receive and incorporate feedback, and to go through the “academic journal writing” process. Through the in-class assignments students will gain experience reading and understanding academic journal articles, critically evaluating and discussing complex concepts, and presenting such concepts to others.
4.1 STUDENT LEARNING OUTCOMES
|
Learning Outcome |
Learning Activities |
How Assessed |
|
Depth and Breadth of Knowledge Understand theoretical frameworks, methods, and concepts of multisensory research and its psychological applications. |
Readings Class discussion Review Paper Seminar presentations |
Quizes Class participation grades Review paper |
|
Application of Knowledge Articulate the above concepts when reviewing or producing their own research |
Review Paper Class/group discussion |
Quizes Class participation grades Review paper |
|
Application of Methodologies Conceptualize and distill findings from a body of primary literature and communicate these in writing. |
Review Paper |
Review Paper |
|
Communication Skills Ability to present and discuss complex ideas surrounding multisensory integration research |
Class discussion Review Paper Seminar presentations |
Quizes Class participation grades Review paper |
5.0 EVALUATION
Review paper: 50%. Students will complete an academic journal style review paper of an area of research involving multisensory integration. Portions of the review will be completed throughout the semester, with a final version due near the end of the semester (2500-3000 words excluding references). The initial submissions of each subsection (review idea, search terms & databases, list of included papers, outline, working draft) will be worth 2% each, for a total of 10%. Students will then be required to provide feedback for two other students’ reviews, worth 5%. The final grant submission will be worth 35% of the grade. (8% of the pre- Nov 7th Grade will be accounted for here in the first 4 initial submissions).
Assignments are due before class on the due date. Late assignments will not be accepted and will be given a grade of 0. Students are responsible for their own assignments--copying assignments constitutes plagiarism, even if it is the student’s own work from another course. This assignment constitutes the written component of 2,500 words required for a half-year essay course.
Because this is an essay course, as per Senate Regulations, you must pass the essay component to pass the course. That is, the average mark for your written assignments must be at least 50%.
In Class Assignments
Quizzes: 10%. Quizzes will take place at the beginning of classes to ensure that students have read the assigned material and will be designed as such. (7% of the pre- Nov 7th Grade will be accounted for here).
Presentations/Facilitation: 20%. Students will sign up for and present two articles assigned for different week’s seminars. Students will be responsible for a brief presentation of the article (~10 min, powerpoint optional), followed by facilitation of the seminar discussion (~20-30 mins).
Seminar Discussion Participation: 20%. Each week, students will submit 1 question per article the day before class in order to stimulate discussion. Students will be expected to actively participate in the seminar, ask questions, give opinions, etc.
Although the Psychology Department does not require instructors to adjust their course grades to conform to specific targets, the expectation is that course marks will be distributed around the following averages:
70% 1000-level and 2000-level courses
72% 2190-2990 level courses
75% 3000-level courses
80% 4000-level courses
The Psychology Department follows Western's grading guidelines, which are as follows (see http://www.uwo.ca/univsec/pdf/academic_policies/general/grades_undergrad.pdf):
A+ 90-100 One could scarcely expect better from a student at this level
A 80-89 Superior work that is clearly above average
B 70-79 Good work, meeting all requirements, and eminently satisfactory
C 60-69 Competent work, meeting requirements
D 50-59 Fair work, minimally acceptable
F below 50 Fail
6.0 TEST AND EXAMINATION SCHEDULE
Not Applicable.
7.0 CLASS SCHEDULE
The schedule for the lecture topics and readings are listed at the end of this outline. Also included is a list of weekly reading requirements.
8.0 STATEMENT ON ACADEMIC OFFENCES
Students are responsible for understanding the nature and avoiding the occurrence of plagiarism and other scholastic offenses. Plagiarism and cheating are considered very serious offenses because they undermine the integrity of research and education. Actions constituting a scholastic offense are described at the following link: http://www.uwo.ca/univsec/pdf/academic_policies/appeals/scholastic_discipline_undergrad.pdf
As of Sept. 1, 2009, the Department of Psychology will take the following steps to detect scholastic offenses. All multiple-choice tests and exams will be checked for similarities in the pattern of responses using reliable software, and records will be made of student seating locations in all tests and exams. All written assignments will be submitted to TurnItIn, a service designed to detect and deter plagiarism by comparing written material to over 5 billion pages of content located on the Internet or in TurnItIn’s databases. All papers submitted for such checking will be included as source documents in the reference database for the purpose of detecting plagiarism of papers subsequently submitted to the system. Use of the service is subject to the licensing agreement, currently between Western and Turnitin.com http://www.turnitin.com
Possible penalties for a scholastic offense include failure of the assignment, failure of the course, suspension from the University, and expulsion from the University.
9.0 POLICY ON ACCOMMODATION FOR MEDICAL ILLNESS
Western’s policy on Accommodation for Medical Illness can be found at:
http://westerncalendar.uwo.ca/PolicyPages.cfm?Command=showCategory&PolicyCategoryID=1&SelectedCalendar=Live&ArchiveID=#Page_12
Students must see the Academic Counsellor and submit all required documentation in order to be approved for certain accommodation:
http://counselling.ssc.uwo.ca/procedures/medical_accommodation.html
10.0 OTHER INFORMATION
Office of the Registrar web site: http://registrar.uwo.ca
Student Development Services web site: http://www.sdc.uwo.ca
Please see the Psychology Undergraduate web site for information on the following:
http://psychology.uwo.ca/undergraduate/student_responsibilities/index.html
- Policy on Cheating and Academic Misconduct
- Procedures for Appealing Academic Evaluations
- Policy on Attendance
- Policy Regarding Makeup Exams and Extensions of Deadlines
- Policy for Assignments
- Short Absences
- Extended Absences
- Documentation
- Academic Concerns
- 2018 Calendar References
No electronic devices, including cell phones and smart watches, will be allowed during exams.
LECTURE TOPICS AND SCHEDULE
Week Date Topics, Readings, and Assignments
- Sept 6 Introduction and overview
- Sept 13 Basic principles of multisensory integration
- Sept 20 Neuralmechanismsandneurophysiologyofmultisensory
integration
Assignment: Review idea due
- Sept 27 Audiovisual integration
Assignment: Search terms and databases due
- Oct 4 Multisensory integration and speech perception
Assignment: List of included papers due
- Oct 11 *** Reading Week - no class ***
- Oct 18 Visual-tactile/somatosensory/proprioceptive integration
- Oct 25 Visual-vestibular integration
Assignment: Outline due
- Nov 1 Development of multisensory integration
- Nov 8 Multisensory integration in older age
- Nov 15 Multisensory integration in special populations and links to cognitive
functioning
- Nov 22 No class
Assignment: Full grant draft due
- Nov 29 Writing workshop
Assignment: Reviews feedback due
- Dec 6 Crossmodal plasticity, reorganization, and rehabilitation
Assignment: Final paper due
List of Assigned Papers
Week 2: Basic principles of multisensory integration
Ernst, M. O., & Bülthoff, H. H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences, 8(4), 162–169.
Alais, Newell & Mamassian (2010). Multisensory processing in review: from physiology to behaviour. Seeing and Perceiving, 23, 3‐38.
Stevenson, R. (2014). Identifying and Quantifying Multisensory Integration: A Tutorial Review. Brain Topography, 27(6), 707–730.
Spence, C. (2011). Crossmodal correspondences: A tutorial review (2011). Atten Percept Psycho, 73 (971995)
Week 3: Neural mechanisms and neurophysiology of multisensory integration
Meredith, M. A., & Stein, B. E. (1986). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology, 56(3), 640–662.
Stein, B.E. & Stanford, T.R. (2008). Multisensory integration: current issues from the perspective of the single neuron. Nature Reviews Neuroscience 9: 255‐266.
Molholm, S. Ritter, W., Murray, M., Javitt, D.C., Schroeder, C.E., Foxe, J.J. (2002). Multisensory auditoryvisual interactions during early sensory processing in humans: a high‐density electrical mapping study. Cognitive Brain Research, 14, 115‐128.
Week 4: Audiovisual integration
Shams, L., Ma, W. J., & Beierholm, U. (2005). Sound‐induced flash illusion as an optimal percept. Neuroreport, 16(17), 1923–1927.
Alais D, Burr D (2004). The ventriloquist effect results from near‐optimal bimodal integration. Curr Biol 14:257‐262.
Fujisaki W, Shimojo S, Kashino M, Nishida S (2004). Recalibration of audiovisual simultaneity. Nat Neurosci 7:773‐778.
McGurk, H. and MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264 (5588), 746‐748
Week 5: Multisensory integration and speech perception
Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. The journal of the acoustical society of america, 26(2), 212-215.
Crosse, M. J., Di Liberto, G. M., & Lalor, E. C. (2016). Eye can hear clearly now: inverse effectiveness in natural audiovisual speech processing relies on long-term crossmodal temporal integration. Journal of Neuroscience, 36(38), 9888-9895.
Dick, A. S., Solodkin, A., & Small, S. L. (2010). Neural development of networks for audiovisual speech comprehension. Brain and language, 114(2), 101-114.
Sekiyama, K., Kanno, I., Miura, S., & Sugita, Y. (2003). Auditory-visual speech perception examined by fMRI and PET. Neuroscience research, 47(3), 277-287.
Week 7: Visual‐ Tactile/Somatosensory/Proprioceptive Integration
Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870), 429–433.
Botvinick, M., and Cohen, J. (1998). Rubber hands “feel” touch that eyes see. Nature, 391, 755‐756. **Let’s do a Rubber Hand demo in class!**
Ehrsson HH, Spence C, Passingham RE (2004). That's my hand! Activity in premotor cortex reflects feeling of ownership of a limb. Science, 305:875‐877.
Blanke, O. (2012). Multisensory brain mechanisms of bodily self‐consciousness. Nat Rev Neurosci, 13, 556571.
Zampini, M., Spence, C. (2004).The role of auditory cues in modulating the perceived crispness and staleness of potato chips. Journal of sensory studies, 19, 347‐363.
Week 8: Visual – Vestibular Integration.
Fetsch, C. R., DeAngelis, G. C., & Angelaki, D. E. (2010). Visual‐vestibular cue integration for heading perception: applications of optimal cue integration theory. European Journal of Neuroscience, 33, 17211729.
Gu, Y., Angelaki, D. E., & DeAngelis, G. C. (2008). Neural correlates of multi‐sensory cue integration in macaque area MSTd. Nature Neuroscience, 11(10), 1201–1210.
Butler, J. S., Smith, S. T., Campos, J. L., & Bülthoff, H. H. (2010). Bayesian integration of visual and vestibular signals for heading. Journal of Vision, 10(11), 23.
Week 9: Development of multisensory perception in infancy/childhood
Hillcock, A.R., Powers, A.R., Wallace, M.T (2011). Binding of sights and sounds: Age‐related changes in multisensory temporal processing, Neuropsychologia, 49(3), 461‐467.
Cowie, Makin and Bremner (2013). Children’s Responses to the Rubber‐Hand
Illusion Reveal Dissociable Pathways in Body Representation, Psychological Science, 24(5) 762‐769
Gori, M., Del Viva, M., Sandini, G., Burr, D.C. (2008). Young Children do not integration visual and haptic form information. Current Biology, 18, 694‐698. Plus the commentary by Ernst (Current Bio, 2008).
Murray, M.M., Lewkowicz, D.J., Amedi, A., Wallace, M.T. (2016). Multisensory Processes: A balancing act across the lifespan. Trends in Neuroscience, 39(8), 567‐579.
Week 10: Multisensory integration and older age
Peiffer, A. M., Mozolic, J. L., Hugenschmidt, C. E., & Laurienti, P. J. (2007). Age‐related multisensory enhancement in a simple audiovisual detection task. Neuroreport, 18(10), 1077–1081. http://doi.org/10.1097/WNR.0b013e3281e72ae7
Mozolic, J. L., Hugenschmidt, C. E., Peiffer, A. M., & Laurienti, P. J. (2012). Multisensory Integration and Aging. In M. M. Murray & M. T. Wallace (Eds.), The Neural Bases of Multisensory Processes. Boca Raton (FL): CRC Press. Retrieved from http://www.ncbi.nlm.nih.gov/books/NBK92841/
Freiherr, J., Lundström, J. N., Habel, U., & Reetz, K. (2013). Multisensory integration mechanisms during aging. Frontiers in Human Neuroscience, 7.
Noel, J‐P., De Niear, M., Van der Burg, E., Wallace, M. (2016). Audiovisual simultaneity judgment and rapid recalibration throughout the lifespan. PlosOne, 1(8): e0161698. doi:10.1371/ journal.pone.0161698
Week 11: Multisensory integration in special populations and links to cognitive functioning
Delbeuck, X, Collette, F., Van der Linden, A (2007). Is Alzheimer’s disease a disconnection syndrome? Evidence from a crossmodal audio‐visual illusory experiment. Neuropsychologia, 45(14), 3315‐3323.
Festa, E.K., Katz, A.P, Ott, B.R, Tremont, G., Heindel, W.C. (2017). Dissociable effects of aging and mild cognitive impairment on bottom‐up audiovisual integration. Journal of Alzheimer’s Disease, 59(1), 155‐167.
Wallace, M. T., & Stevenson, R. A. (2014). The construct of the multisensory temporal binding window and its dysregulation in developmental disabilities. Neuropsychologia, 64, 105-123.
Stevenson, R. A., Segers, M., Ncube, B. L., Black, K. R., Bebko, J. M., Ferber, S., & Barense, M. D. (2018). The cascading influence of multisensory processing on speech perception in autism. Autism
Week 14: Cross‐modal plasticity, reorganization, and rehabilitation
Powers, A. R., Hillock, A. R., & Wallace, M. T. (2009). Perceptual training narrows the temporal window of multisensory binding. Journal of Neuroscience, 29(39), 12265-12274.
Ramachandran, V. S. and Altschuler, E. L. (2009). The use of visual feedback, in particular mirror visual feedback, in restoring brain function, Brain 132, 1693–1710.
Glick, H., Sharma, A. (2017). Cross‐modal plasticity in developmental and age‐related hearing loss: Clinical implications. Hearing Research, 343, 191‐201.
Merabet, L.B., Pascual‐Leone, A. (2010). Neural reorganization following sensory loss: the opportunity of change. Nature Reviews Neuroscience, 11(1), 44‐52.
Template to Help Guide the Deconstruction and Interpretation of Papers
For Empirical Papers
‐ What was the general objective of the study(ies)?
‐ What made it novel at the time?
‐ What approach did the authors take, or what method was used? Why did this approach make sense?
‐ What were the main take home messages?
‐ Why do you think these findings were so important/influential?
‐ What limitations could you identify?
‐ What future studies could this study have laid the groundwork for?
For Review Papers
‐ What were the key concepts being reviewed?
‐ Why are these concepts important for people wanting to learn about multisensory integration (and beyond)
‐ What aspects of the reviewed research could be related to your own research and how? ‐ How did it change your perspective on certain ideas/thinking?
