Using Logic Diagrams to Organize Knowledge and Pinpoint Ignorance

(adapted from essay and talk prepared for the Cutting Edge Workshop on "Developing Student Understanding of Complex Systems in the Geosciences," Carleton College, April 18-20, 2010.)

Introduction

One of the hardest aspects of learning about complex Earth Systems is organizing fragments of knowledge into some kind of coherent framework. I teach students who want to be science journalists or environmental journalists. These students are looking towards a career in which they will frequently have to jump right into the deep end and come quickly up to speed on complicated new ideas. I am always on the lookout for techniques that will help them construct understandings quickly and accurately, techniques that will continue to work for them when they no longer have me and their other professors to scaffold their learning, techniques that will help them to become effective self-directed learners.

One of the most powerful techniques that I have come up with is a type of concept map I call a logic diagram. A concept map is a diagram showing the relationship among concepts. Concept map relationships can take various forms, but the subspecies of concept map that I am advocating here includes only one kind of relationship: causality relationships. The links in such a concept map can be read as "causes" or "influences" or "gives rise to" or "leads to" or "enables." The name, "logic diagram," is inspired by the so-called [link https://ctb.ku.edu/en/table-of-contents/overview/models-for-community-health-and-development/logic-model-development/main #LogicModel '"logic models"'] used in educational evaluation, which spell out the logic or reasoning by which specified inputs are intended to enable specified activities which in turn are supposed to lead through various intermediate steps to specified outcomes. Another possible name would be "influence flowcharts."

I use logic diagrams in three contexts: to build explanations in class, to support student research projects, and to critique student writing.

Using logic diagrams to build understanding in class:

A cleaned-up version of the first logic diagram I ever constructed for my Case Studies in Earth & Environmental Science Journalism class is shown below. The case was about the Dead Zone in the Gulf of Mexico, and as I read in preparation for developing the case, I felt overwhelmed by how many different things were going on. (Another challenge in teaching journalism students is that I am continually teaching about topics I don't know enough about; if they were sufficiently well established that I already knew the full story, they wouldn't be newsworthy.) As I read and puzzled, I gradually sketched out a crude version of the diagram below, adding onto it as I read more papers and assembled more fragments of the big picture. The logic diagram shows that both biological processes and physical processes contribute to causing a zone of anoxic bottom water and dead organisms offshore from the mouth of the Mississippi River each spring and summer.

In class, I construct logic diagrams on the black board interactively, collecting student suggestions, fielding student questions, and adding new boxes and arrows to the diagram as the various bits of insight emerge from students. Based on the assigned readings, the group of students brainstorming collectively are usually able to come up with the important elements of the system under discussion, but the ideas tend to be fragmentary and disconnected. As students articulate fragments of how the system works, I probe for pieces of understanding of what causes or influences what, and ask students to direct my chalk as I to where to fit each snippet of causal logic into the diagram. When contributions stall, I add in a few boxes and arrows myself, and then return to collecting student ideas. (I don't try to make up logic diagrams from scratch in real time in front of the class; I always have a pencil sketch ahead of time. These things are hard to make.)

When the basic framework of the logic diagram is in place, my students and I can then use that framework to provide attachment points to incorporate new insights, new datasets, into our understanding of the system. Here are two examples from the Dead Zone case study. A key paper seems to be obsessed with silica in sediments. Why, I ask, are these authors going on and on about silica, when the Dead Zone story is supposed to be about carbon, nitrogen, and phosphorous? Eventually, we are able to attach this line of reasoning to an existing portion of our logic diagram (lavender boxes in figure below), revealing that silica in the sediments preserves a record of a key element in the dead zone puzzle--how much phytoplankton productivity is going on in the sunlit surface waters.

The Dead Zone in the Gulf of Mexico seems to be getting worse, and some of our readings blame this on bad policy decisions or lack of good policy decisions. If a paper or an interviewee claims that a policy decision, either past or future, impacts or will impact the extent of the Dead Zone, then we should be able to put our finger on a point in the logic diagram where the policy influences some component of the system (green boxes in figure below). If we can't do so, then either we don't fully understand the system, we don't fully understand the policy, or the policy will not make any difference.

Asking good questions is one of the most important tools of the journalists' craft. Learning by asking questions and seeking answers is the essence of inquiry, either in science or in journalism. Logic diagrams help my students and me to pinpoint our areas of ignorance, and thus to ask fruitful questions. Each arrow in the logic diagram asserts a claim that A (in the upstream box) causes or influences B (in the downstream box). For each arrow in the logic diagram we can ask: What is the evidence that A causes or influences B? Do I understand the evidence? Isthere actually any empirical evidence for this step in the logic chain, or is this a place where more observations would be useful? What is the mechanism by which A is supposed to cause or influence B? Do Iunderstand the mechanism? Does anyone understand the mechanism, or is this the frontier between the known and unknown? Trying to answer these questions reveals gaps in one's own personal knowledge, where more learning is needed, and gaps in humanity's collective knowledge, where more research is needed.

Using logic diagrams to support student research projects:

When my journalism students are formulating and implementing their science masters research projects, I encourage them to construct their own logic diagrams for portions of the Earth System they are trying to understand or elucidate. They typically find this hard, but deeply illuminating.

Below is a logic diagram constructed by Earth & Environmental Science Journalism student Diya Chacko. Her research project used stable isotopes of boron in corals to investigate variations in paleo-pH levels in the tropical Pacific. The boron pH proxy is so new that it doesn't yet show up in textbooks, and the handful of published papers in the specialized research literature using this proxy make some pretty ambitious assumptions about how much the reader already understands about stable isotope systematics. Diya struggled mightily to understand this complex system. Again and again she thought she had it figured out, but her inability to construct a chain of coherent, logical steps running from the pH of seawater to the stable isotopes of boron telegraphed to me and to her that her understanding was still incomplete. She returned again and again to the seminal papers and to her geochemist research advisor with newly refined questions, each time further organizing her understanding and pinpointing her ignorance. After many iterations, she constructed an internally consistent line of reasoning, conveyed in her explainable and defensible logic diagram.

Using logic diagrams to critique student writing:

One final use I make of logic diagrams is to critique student writing about complex systems. For example, in a story pitch, a student wrote:

This paragraph attempts a compact description of the ramifications of a change in the earth system, encompassing both the causes of the change and its consequences for human society. In a story pitch, the writer has only a few words with which to capture the attention of a busy reader, in this case the editor of a newspaper in the Pacific Northwest. The urgency of the implications come through, but the nature of the threat may not make sense to a non-geoscientist reader. If we turn this paragraph into a logic diagram, we come up with:

When spelled out stepwise in a logic diagram, a missed step in this line of reasoning becomes glaringly obvious. It is not logical, on the face of it, to say "...as glaciers melt and river flows decrease...." If a glacier feeds a river, then as the glacier melts, more water--not less water--should flow into the river. Something must be either wrong or missing in the logic spelled out in the student's paragraph. Further scrutiny reveals that a step is missing: the complete line of reasoning requires the extra step that global warming leads to decreased snowfall which leads to shrinkage of the glacier mass, which in turn leads to decreased available stream flow in the summer months when water and electricity are most in demand. When the extra step is added into the logic diagram, we have:

A newspaper editor scanning the pitch letter might not be able to pinpoint the missing step in the chain of reasoning, but might very well come away with the overall sense that "this doesn't make sense to me; I don't want this story."

Concluding comments:

School-based science experiments tend to deal with short and simple causality chains. Students come to us with experience conducting science inquiries in which they have been rewarded for successfully isolating one "manipulated variable" and holding constant one "controlled variable." To understand complex Earth Systems, they need to stretch their view of science to include causality chains that are many steps long, and may include multiple converging or diverging pathways. Logic diagrams can help.

Sketching a logic diagram helps my students and me construct a framework to organize our fragments of information into a coherent systemic overview. Logic diagrams provide an attachment point for newly-arrived bits of knowledge, increasing the chances that those knowledge snippets will become lasting learning.

Journalists and scientists both learn through the process of asking questions and seeking answers. Logic diagrams help us to pinpoint our areas of ignorance, and thus generate well-focused questions, as we ask of each arrow: What is the evidence that A causes or influences B? And what is the mechanism by which A causes or influences B?

---

This essay is in the spirit of other writings on the general topic of "making thinking visible." If this broader topic is of interest to you, check out:

• The Visible Thinking Project at Harvard.
• Concord Consortium, NSF funded project on Making Thinking Visible, PI Janice Gobert, emphasis on students creating concept-driven visualizations.
• WISE, at UC Berkeley, has "Making Thinking Visible" as one of its four design goals.