Outcomes from a Workshop on "Educating Skillful Visualizers"

by workshop conveners: Kim Kastens, Tim Shipley, and Martin Storksdieck

(last updated 6 March 2018)

Introduction

Visualizations can be a powerful tool for both thinking and communicating. This website is the product of a workshop designed to gather, organize, and disseminate good ideas for how educators can help learners extract insights from visualizations and express their own ideas via visualizations, and can assess learners' progress towards visualization mastery. By "visualizations," we include both data-driven visualizations (graphs, maps, and other images built from quantitative data) and concept-driven visualizations (diagrams, flow charts, concept maps and other images generated from a concept or theory).

The workshop was designed around the premises:

that there is a suite of approaches and resources that skillful, experienced users and creators of visualizations draw on;

that there are important commonalities in these approaches across disciplines;

that effective use of these approaches can be fostered by instruction and improved through practice; and

that it would be possible to articulate these approaches and resources through pooling the experiences of experts from many fields in which use of visualizations is a key competency.

Our group included expertise in science and engineering, formal and informal science education, learning science, education reform, cognitive science, instructional design, evaluation, computer graphics and science communication. Within the STEM disciplines, we had representation from bioinformatics, biology, computer science, chemistry, engineering, environmental science, geoscience, marine science, physics, space & planetary science. Workshop attendees were an education-oriented subset of the participants who would be attending the interdisciplinary 2017 Gordon Conference on Visualization in Science & Education. This wide range of expertise allowed the workshop group to seek out strategies and approaches that could work across disciplines and topics.

This website captures insights and suggestions that emerged from the workshop, subsequently polished and somewhat expanded by the conveners. The website was reviewed by workshop attendees and two external invited reviewers. The intended audience includes educators who use visualizations in their teaching, plus researchers seeking to better understand how humans learn about the universe through representations. The span of learners under consideration ranges from elementary school through adult life-long learners, with the strongest emphasis on undergraduates. We address both visualization literacy (a set of practices, understandings, and habits of mind that would be of value to all inhabitants of 21st century Earth) and visualization mastery (a set of practices, understandings, and habits of mind that would be important to professionals in fields in which use of visualizations is a key competency).

If you find this website interesting, please consider attending the 2019 Gordon Conference on Visualization in Science & Education, which will has this same theme: Educating Skillful Visualizers.

Learning Goals for skillful visualizers

We agreed upon a shared set of learning goals that span across disciplines, ages, and visualization types. These include:

Learners should be able to interpret existing visualizations, as documented by their ability to articulate a narrative about the referent based on the representation. This ability encompasses standard types of visualizations as well as novel types of visualization they have not seen before.

Learners should be able to express their own ideas by creating appropriate visualizations.

Learners should be able to evaluate the quality of evidence presented in a visualization, and assess whether it supports, refutes or is irrelevant to a claim about the referent.

Learners should have the habit of mind of considering the goals and mindset of the visualization creator: Why was this visualization created? what was the creator trying to say or accomplish? Who was the intended audience?

Overarching takeaways from the workshop

Experienced interpreters of visualizations draw on a powerful arsenal of cognitive resources when confronted with a novel visualization. Some key steps in the process by which our minds derive insights from visualizations seem to be unavailable for metacognition, and therefore difficult to teach. (See additional page about Approaches and Resources Used by Skilled Interpreters of Visualizations.)

There are strategies of recognized value, for both fostering and assessing visualization competency, that could be more widely employed. There are many pitfalls and challenges in teaching for visualization competency. But every pitfall presents a research opportunity, to investigate why is this a pitfall and what instructional approaches can ameliorate it.

There is a gold mine of research questions around this topic. The "recognized strategies" mentioned above tend to be supported at the level of practitioners' wisdom or pedagogical content knowledge, with a rather slim basis in education research or cognitive science research. A well-planned program of research seeking evidence-based guidance for using visualizations in education would benefit both educational practice and the underlying model of human learning and thinking.

Like facility with language, facility with visualizations is something that undergirds education at all levels and across most disciplines. And yet no part of the educational enterprise takes responsibility for ensuring that all learners achieve visualization literacy and that those bound for technical careers achieve visualization mastery.

There is an inherent tension in the division of instructional time and effort between investing in helping learners improve their generalizable skill set for interpreting visualizations, and just using the visualizations at hand as a vehicle to convey disciplinary content about the how the referent system works.

Strategies your colleagues are using to foster visualization competency

In advance of the workshop, the conveners collected, from the literature and their own experience, nine instructional strategies that seemed to have potential to improve learners' capacity to work with visualizations. The pre-work for the workshop included a survey in which workshop participants were asked how often they used each of these strategies in their role as teachers or encountered each strategy in their role as students. Results from the survey were as follows (click figure to enlarge):

Workshop participants collaborated to identify the potential affordances and pitfalls of each strategy, to find or invent examples of how the strategy could be used in multiple disciplines, and to articulate pertinent research questions. The conveners then added or extracted "Emergent Insights" for each Strategy when we had the luxury of looking across the full set of contributions and insights from all groups. In order from most-frequently used to least-frequently used strategy:

Strategy #1: Learners create concept-driven visualizations to explain their ideas.

Examples include sketching a progression of events that happen over time; sketching a hypothesis or prediction; concept mapping; engineering drawing to shows how a device works.
This strategy is especially powerful for multidimensional phenomena that are not easily conveyed with words alone. Instructional design needs to ensure that learners are assembling the visualization from their own ideas and observations, not merely recreating a textbook diagram.

Strategy #2: Instructor teaches distinctive forms or patterns that are that are important in the discipline.

Examples include map patterns of synclines/anticlines in geology; scatter plot signature of strong versus weak correlation in psychology; patterns characteristic of specific diseases in medical diagnostics.
This strategy requires two steps: first, stocking the mental "library" with useful patterns, and second, learning to judge whether a novel stimulus is a "close enough" match to a relevant pattern.

Strategy #3: Instructor teaches formal attributes of visualizations

Examples include coordinate systems, map projections, molecular models, or earthquake first motion diagrams.
This strategy is time efficient, but by itself builds just procedural knowledge; further hands-on work by learners will be necessary to apply this knowledge to real problems.

Strategy #4: Learners use visualizations to persuade or convince others (e.g. peers, stakeholders)

Examples include using GIS-based visualizations to address a challenge in local land-use planning, and creating an info graphic about a health or environmental issue.
This strategy taps into some learners' interest in advocacy and societal issues, and elicits metacognitive reflection on how visualizations work to communicate and convince.

Strategy #5: Learners create data-driven visualizations to answer a question or test a hypothesis.

Examples span both learner-collected and professionally-collected data; the key is that learners must plan what kind of visualization to make, implement the plan, and then use the product to accomplish something.
This strategy works well across all disciplines of STEM and social sciences, and should be built up over the years of the educational trajectory.

Strategy #6: Learners compare, contrast and critique multiple representations about the same concept or phenomenon.

Examples would include multiple representations of the water cycle, carbon cycle, tectonic plate boundaries, molecules, protein configurations, or engineering designs.
This strategy allows learners to appreciate both commonalities and variations in natural phenomena, and to distinguish variation that exists in the referent system from variations introduced by the visualization creator.

Strategy #7: Learners translate between representations of the same concept or phenomenon

The best-known example is translating between multiple molecular representations in chemistry.
This strategy can help to establish the insight that all visualizations are models, and all models are incomplete, and that multiple visualizations taken together can give a more complete grasp of a complex phenomenon than any single representation taken by itself.

Strategy #8: Learners use a model to make a prediction, and then use a data-driven visualization to test their prediction.

Examples: given a computational weather forecasting model, predict a severe weather event; given two conceptual models of how humans recall a set of previously seen items, predict how the time to verify if an item was included in the set will vary across different sized sets.
This is a form of a broader instructional strategy called Predict-Observe-Explain. The added value of using visualizations at the Predict and Observe steps is that a match between model (prediction) and data (observation) can be easier to perceive if there are visually-available parallels.

Strategy #9: Learners invent a way to represent a data type they have not previously encountered.

Examples from the literature include young children inventing mapping strategies and graphing strategies; the key is that the context has established a need to communicate an idea that is best expressed visually.
This may be the hardest of the fostering strategies to implement as it takes substantial time and skillful facilitation; on the other hand, it prepares learners to work creatively in frontier domains where visualization standards do not yet exist.

Assessing learning from visualizations

Assessing learning begins by clarifying the claim that you want to be able to make or test. Are you interested in gauging what individual students have learned or understood for the purpose of classroom instruction? Or do you want to be able to make a larger scale claim about the effectiveness of specific visualizations or an instructional sequence that you have developed? Or are you seeking "generalizable knowledge" about the learning process that has the potential to be applicable beyond the scope of your specific context?

On a different dimension, are you interested in probing what learners have understood about the referent system (the system of which the visualization is a representation) as a consequence of interacting with the visualizations? Or are you more interested in whether learners are deepening their mastery of techniques for interpreting or creating visualizations (their visualization literacy or visualization mastery)?

A myriad of techniques is available to elicit behaviors and products that can provide evidence around claims of these sorts. Techniques discussed at the workshop included think-alouds, worked examples, exit slips, having the learner annotate or mark up visualizations, focus groups, logging data from interactive visualizations, eye-tracking, various kinds of interviews, concept mapping, classroom clickers, written assessments, and having students create a visualization or a portfolio. Many of the activities used to foster skill with visualizations also generate student products that are suitable for assessment.

Acknowledgements

The workshop conveners thank the attendees for their wise insights and enthusiastic collaboration during the workshop. Able logistical, organizational and web support were provided by Monica Bruckner, Beth Lively, and Mia Velazquez. The usefulness and accuracy of the website were greatly improved through invited reviews by Kristen St. John and Michael Stieff.

×

×

Funding has been provided by the National Science Foundation through grant DRL-1743234 to Temple University, and by NASA through grant NNX15AG04G to James Madison University. Any opinions, findings, and conclusions or recommendations expressed on this website are those of the authors and do not necessarily reflect the views of the funders.