The overall learning goal of this activity is for the students to conceptualize and visualize how computational models of memory manifest in behavioral data (e.g., ROC) without having to do Matlab coding or understand the mathematics of the models. Specifically, the students are expected

1. To conduct the hands-on practice of model visualization without any coding.
2. To visualize and conceptualize the various computational models for human memory.
3. To elaborate on the strengths and weaknesses of each model.
4. To observe how individual model components affect the overall shape of the ROCs. 
5. To interpret different aspects of the ROC curves.

This class activity is designed to help students understand the computational models for memory without students knowing anything about coding. As such, there is no prerequisite for prior coding experience. However, students must be familiar with the various memory models (see the reference section).

This learning activity can be used for upper-division topic courses (e.g., seminar on memory) or method courses (computational psychology) for undergraduate and graduate courses.

The simulator can also be used as a demonstration of the model behaviors by the instructors.

Background

Computational modeling is one of the three foundations of cognitive neuroscience. To study human behaviors, It is important for us to integrate various methods, including psychophysics (for perception and cognition), neuroscience ( for the brain mechanisms), statistics, and computational thinking. In Cognitive Neuroscience,  "... hallmark of cognitive models is that they are derived from basic principles of cognition. This is what makes cognitive models different from generic statistical models or empirical curve fitting models" (Busemeyer & Diederich, 2010). As such computational modeling in psychology is pivotal for several reasons. First, models are abstractions that explain behavior and neural activities. Second, models provide an interpretation of the data that may be counter-intuitive. There are major exemplars of this almost in every corner of psychology.  For instance, our recent work that attributes a fundamental limit of short-term memory to the limited amount of information that can be briefly maintained in mind (Zhang & Luck, 2008, Nature) instead of memory quality (e.g., Bays and Hussain, 2009, Science; Wilken & Ma, 2004, J. of Vision) may be inconsistent with phenomenal experiences or visual inspection of the data. Third, models are the absolute form of theories. Compared to verbal descriptions of theories in Psychology which are often vague and underdefined, models are explicit and testable.  "Formal (i.e., mathematical or computational) theories have a number of advantages that psychologists often overlook. They force the theorist to be explicit, so that assumptions are publicly accessible and the reliability of derivations can be confirmed ..." (Hintzman, 1991). It is potentially an effective way to solve the biggest challenge in social science, which is the lack of replicability.

×

Provenance: weiwei zhang, University of California-Riverside
Reuse: This item is offered under a Creative Commons Attribution-NonCommercial-ShareAlike license http://creativecommons.org/licenses/by-nc-sa/3.0/ You may reuse this item for non-commercial purposes as long as you provide attribution and offer any derivative works under a similar license.

Aim

This teaching activity demonstrates several major cognitive models of human memory and their manifestation on behaviors using an ROC simulator. It can be used as a class activity and/or demonstration. It can be further used to visualize the theoretical predictions of the various models in response to a given experimental manipulation. 

App installation  
The students are expected to install the ROC simulator by following these steps.

1. The app can be downloaded from Zhang lab GitHub site (https://github.com/CONPAMlab/ROC-Simulator).

2. Unzip the downloaded file.

3. Double-click on the app installer "ROC Simulator.mlappinstall".

4. The ROC Simulator app can then be located in the APPS section of Matlab.

5. For each model, an initial value for a model parameter needs to be chosen to show the curves ( the future update will set a random initial value and show the curves).

Project activity 

The students are expected to conduct learning activities centered around the learning goals and the assessment. Some example practices are provided below.

1. For each model, understand the psychological meaning of each parameter (model component) and the psychological meaning of the values, and the range of the values for a given parameter .

2. For each model,  change the value of a parameter and observe how it affects the overall shape of the ROC curve.

3. For a given model, observe how different parameters jointly and independently affect the overall shape of the ROC curve.

4. Inversely, how can the different aspects (or changes in these aspects) of the ROC curve (curvature and asymmetry) be mapped onto the model components/parameters? For instance, which component of the slot model versus the DPSD model affects the asymmetry of the ROC curves?

5. How does decision criterion affect or fail to affect the overall shape of the ROC curves? How does the decision criterion conceptually contribute to the ROC curve (and also in terms of measurement)?

×

This class activity is designed to help students understand the theoretical and empirical ROCs and their components for several major models for human memory. As such the emphasis should be on the computational principles of these models and how they manifest on the overall shape of the ROC curve.

The simulator first simulates the data for a few hundred trials of the old versus new memory responses on the standard 6-point confidence scale, given the parameters the students choose for a given model. The empirical ROC is then computed and visualized.  Alternatively, theoretical ROCs can be directly computed and visualized without simulating the empirical data first. However, the current approach is preferred, given the added benefits ( e.g., demonstrating how the number of observations affects the overall model fit).

This project is intended to visualize the various memory models for students without any coding experience. It can be used as a class activity. The learning goals can be assessed using the following demonstrations and open-ended questions.

1. Can the students run the simulator and test the various models?
2. Can the students articulate the various computational models for human memory? 
3. Can the students explain the strengths and weaknesses of each model?
4. Can the students demonstrate how model components affect the overall shape of the ROCs? Some examples are provided below
   • Which component of the DPSD, UVSD, and slot model can account for the asymmetry of the overall ROC shape? How do the opposite model components for the DPSD (the high-threshold component) and slot model (the signal detection theory component)  account for the asymmetry of the ROC curves?  
   • What are the relationships of the model components (e.g., linear summation versus convolution) in each model? 
   • How do recollection (R) and familiarity (d') components of the DPSD model independently affect the overall shape of the ROC  curves?
   • How do the two components of the slot model independently affect the overall shape of the ROC  curves?
   • How does the placement of the decisional criterion affect a given data point on the ROC curve,  without changing the overall shape of the ROC curves?
5. Can the students interpret the different aspects of the ROC curves given each model?

1. Zhang, W. & Luck, S. J. Discrete fixed-resolution representations in visual working memory. Nature 453, 233–235 (2008).

2. Bays, P. M. & Husain, M. Dynamic Shifts of Limited Working Memory Resources in Human Vision. Science 321, 851–854 (2008).

3. Wilken, P. & Ma, W. J. A detection theory account of change detection. J Vis 4, 1120–1135 (2004).

4. Busemeyer, J. R., & Diederich, A. (2010). Introduction to cognitive modeling. In Cognitive Modeling. Sage.

5. Hintzman, D. L. (1991). Why are formal models useful in psychology? In W. E. Hockley & S. Lewandowsky (Eds.), Relating theory and data: Essays on human memory in honor of Bennet B. Murdock (pp. 39–56). Lawrence Erlbaum Associates, Inc.