I Just Want to See the Raw Data!

Originally posted (by me) on LinkedIn, December 9, 2014.

This is pretty close to raw data. How informative is it?

This is pretty close to raw data. How informative is it?

“I just want to see the raw data! No interpretation, no massaging the numbers, just the raw data straight out of the instrument!”

I sympathized with my non-scientist friend. She felt frustrated after reading a series of news items that began with a promising discovery, followed by a series of caveats, followed by more news stories reporting that no one knew for sure what was going on, and several more years of research would be required to clarify the findings from the initial report.

She didn’t know whom to believe. Scientists presented what looked like clear and convincing evidence, only to be shouted down by political activists and religious leaders claiming, “That’s just your opinion!” and citing past scientific studies proven biased, fraudulent, or just plain wrong.

The problem is, raw data points don’t tell you much of anything. Even an experienced scientist needs some kind of interpretation to convert the numbers into knowledge. What question was being asked and how did the person go about trying to find an answer? What did the instrument measure directly, and what assumptions were used to make indirect observations? What good does it do you to know that an unknown sample produces six times the voltage response at 7.3 minutes elution time than it did at 6.9 minutes if that’s all you know?

What were the conditions of the experiment? The same instrument can generate high- and low-resolution data, depending on how it is set up. Each type of data is useful for some purposes, but not others. Attachments, filters, thermostats — each of these can be added, adjusted, or calibrated to increase the sensitivity toward some observations, but they can also obscure other observations. The star you’re looking at might shine most brightly at the wavelengths you’re filtering out to cut interference from the sodium-vapor street lamps nearby.

How closely do the observations mimic processes in the real world? Lab-scale syntheses often fail to predict the results of a full-scale industrial production run. Data taken in the field under real-life conditions can be biased or inaccurate, because the act of observing has altered the behavior of thing you’re observing.

The progress of science itself can prove previous science wrong or expose the limitations of previous theories. Some 19th-century physicists thought that physics would shortly become a closed field of inquiry. All the questions had been answered, they thought. Only a few small loose ends needed to be wrapped up, and then the books could be closed. One of those loose ends turned out to be quantum theory, which underlies the technologies behind the automatic doors at the grocery store and flash drives that let you store hundreds of tunes in a device you can carry in your pocket.

Scientists do share raw data among themselves, especially when they are seeking alternate interpretations or reusing data for another purpose. The studies they publish in the journals contain interpreted data, along with information on how the data were obtained and what assumptions and processing steps were used. Other scientists with experience in the strengths and limitations of various instruments and methods review each others’ studies and identify gaps or alternate interpretations.

Some tests have been repeated so often, with such consistent results, that the interpretative steps can be programmed into the instruments themselves. You see this type of lab analysis on television shows where the forensic lab tech puts a paint chip from a crime scene into an instrument, and immediately sees that it could only have come from a 2009 Fiat. This data is not raw — the interpretation has merely been automated.

A wide spectrum of knowledge spans the territory between “That’s just your opinion!” and “There’s so much evidence here that I would stake my life on this.” The difference lies not in seeking some pure spring of unsullied data, but in knowing what questions were asked, how they were answered, and how the answers fit in with everything else. It requires seeing things happen the same way over and over and trusting things to happen that way again under the same conditions. It also requires a willingness to change your thinking if new information puts established knowledge into a new and broader context.

Can We Talk?

(Reprinted from Flying Lessons: verbal-aviation.blogspot.com)

My year off is officially over. I’m still at home, but I’m actively looking for work. Maybe that will be a job in the conventional sense of the word, or maybe it will be a more creative way to bring in income. I’m not sure. I’ve tried telling myself that I ought to be checking the job ads more assertively, mailing brochures to potential clients, making cold calls and all. But it just doesn’t feel right. I have chastised myself, telling myself to just get over my inertia and start the wheels moving again. Don’t be such an introvert, I say. Which is kind of like telling myself to stop having such blue eyes or stop being so short.

I have found a much better source of motivation, though. It comes through in the morning when I am writing in my journal — or sometimes at 3AM when I am wishing that I could get back to sleep. In the deep stillness of my room, little ideas make their way to the surface. Concrete, definite actions that I could take. Actions with energy and enthusiasm behind them. Actions based on what I do naturally, what I enjoy, what gives me satisfaction.

There’s a stillness born of time away from the daily commute, the meetings and deadlines and periods of boredom interspersed with crises. In that space has come an appreciation of things that I have been doing for years, but have not integrated into the way I make my living. With that realization has come a desire to more fully integrate the marketable skills with the calling of what is most important to me.

I have come to acknowledge more fully my talent for talking about scientific and technical issues in language that is engaging and easy to understand. I might not get the story first, but I get it in context and I do my best to get it right. That’s a real talent. It’s much harder than summarizing one’s research in the specialized language of one’s own field. It goes beyond opening up a stream of data in the hopes that the more information you throw at people, the more likely they are to come around to your point of view. It’s very different from “dumbing it down”. Good science writing requires me to respect my reader’s intelligence and convey an accurate, nuanced picture in language that is both precise and accessible.

Good science writing is an act of empathy. When I’m conducting interviews for an article, I have to do my homework ahead of time to know who it is I’m talking to and have some basic knowledge of their work. I don’t want to waste this person’s time asking basic questions that could be answered easily with a little online research. It’s incumbent on me to know and convey to the person I’m interviewing the purpose of the article I’m writing and to ask for information that makes my article into something worth reading. I’m responsible for communicating to my source just why it is that he or she is exactly the person who can best help me learn and convey the specific information I’m going after.

I have to use my skills in asking questions to encourage my sources to talk about the most interesting, relevant, or important parts of their work. I have to listen carefully to what my sources are telling me. I can’t assume that I know in advance what they are going to say. I can’t shape their answers into what I think they should have said. I can’t be reluctant to ask for clarification — even if I think I understand what they meant to say. This requires a certain humility on my part — a willingness to relinquish any concept of myself as an expert and to let my sources speak for themselves. My expertise comes in organizing and conveying the voices of all my sources as accurately and understandably as possible.

Empathy requires me to know something about the audience I’m writing for. What this audience is most interested in might not be the same thing that most interests my sources. Depending on the type of article I’m writing, I might have to spend some time educating my audience, but I can’t be overly didactic without losing their attention. Everyone is busy these days, and a multitude of information sources compete for my reader’s attention. I have to make it worth my reader’s time to read my writing. I have to show my readers something new or present a different point of view on something familiar. Some part of what I say has to be relevant to the world they inhabit.

I have to be trustworthy. If I come across as selling a particular point of view or advocating for a particular cause, I might capture the attention of those who already agree with me. But I will lose those readers who disagree with me — the very readers who might have an “aha!” moment or engage others in a constructive dialogue after having read my article. On the other hand, false balance is just as misleading as blind advocacy. On some issues (climate change is one notable example), the scientific consensus is so strong that giving equal weight to a small opposing minority is a distortion of the facts.

Trustworthiness also requires clarity. “Baffle them with bullshit” is not an acceptable approach here. Few intelligent readers come away convinced that because an issue is presented in dense technical prose, it must be important and correct. On the other hand, talking down to one’s readers, using lazy metaphors, or affecting a false hipness only makes the writer look incompetent. The goal is to convey a message and convey it well.

Conveying a message also requires an understanding that we are not completely rational beings. Two reasonably sane, intelligent, well-intentioned people can look at the same set of facts and draw very different conclusions. We all operate within our own social, historical, and experiential frameworks, and we interpret what we see accordingly. A good writer must provide enough context and perspective to inform, but not overwhelm, her readers.

Our emotions affect how we react to information, whether or not we are aware of it. Thus, humor, diplomacy, and yes, empathy are far more than ways to “spice up” an article. They are necessary elements in connecting with one’s audience and opening a space for dialogue — or perhaps drawing the lines for battle.

The work I enjoy best draws on all these skills, but some of my previous jobs have required a “just-the-facts” approach. For some purposes, that’s enough. An activity report for a government agency is not the right place to hone one’s skills in humorous narrative nonfiction. But since I have all of these skills, it’s up to me to find an outlet for them. Leave the cut-and-dried work to those who excel at it.

Lately, I’ve been exploring social media (FaceBook, Twitter, and the like) as a means of staying in touch with the people I’ve met through my travels, career, and various stages of my life. This has evolved into a means of conversing with people whom I have never met face-to-face, but with whom I share common interests and affinities. I’m exploring the nuances of brief written communications and asking myself how well it is possible to know another person through electronic interaction alone. Electronic communities are changing the way we understand friendship and the way information (or misinformation) spreads.

Increasingly, interest groups, businesses, and other organizations use these channels to shape what we think and how we talk to each other in ways that go far beyond the pop-up ads and “you might be interested in…” suggestions. Skilled communicators realize that this is where their audience is, and they seek out the people they want to reach in this way. Perhaps the entire message can be conveyed right there on the spot. Perhaps a brief note on Twitter alerts readers to a more detailed account elsewhere. Perhaps flinging an idea into the fray sparks a conversation or elicits a wealth of crowd-sourced information, an exchange of diverse points of view. In any case, it’s a matter of going out and engaging in dialogue with readers where they are rather than passively waiting in the backwaters of the information stream.

If this is so interesting to me in my personal interactions, why not investigate ways to build this into my profession as a science writer? After all, one of the reasons I took 2012 as a year off was to discover ways of integrating the various values and interests I have into a means of supporting myself while contributing something worthwhile to the world.

Update: If you would just listen to the facts…

Janet Raloff, a friend of mine and a senior editor at Science News, has weighed in on the topic of how scientists can communicate effectively with the public on divisive issues such as climate change. In her May 29 online article, Climate skepticism not rooted in science illiteracy, she reports on an interview with Dan Kahan of Yale Law School, one of the authors of the Nature Climate Change article I cited in my previous posting. Raloff explores several approaches to interacting with a polarized public in her May 30 follow-up article, Depolarizing climate science.

What comes through most strongly in these articles is that divisions occur when scientific findings have a direct relevance to things that affect us every day, or findings that could make significant changes in our everyday lives. As far as I know, The Heartland Institute has not posted any billboards comparing people who believe in black holes or the law of gravity with Unabomber Ted Kaczynski. We accept these things because we can do so without making any changes in our daily routines.

Raloff cites political scientist Arthur Lupia of the University of Michigan in Ann Arbor, who states that some of the most science-literate critics will listen to experts only to generate compelling counterarguments. What’s stopping climate scientists from doing the same thing — listening to the most rational, articulate deniers and opponents of climate change legislation in order to scope out what matters most to climate change deniers and formulating compelling counterarguments of their own?

Disruptive innovation is another possible approach — one that sidesteps the battleground altogether by providing people with alternatives that they adopt readily because they see a clear benefit to the new technologies. After all, no one waged a political war against paper phone books, wristwatches, or the Sony Walkman, and yet those things have almost completely fallen out of use by Americans under the age of 30, in favor of smartphones, smartphones, and smartphones. The main problem with this approach is that disruptive innovations are notoriously unpredictable. We cannot guarantee the timely arrival and widespread adoption of any particular form of alternative energy or resource-conserving capabilities.

So for now, we are stuck with learning to talk to each other. Which is something that we needed to do anyway.