In response to a recent post on CreateEquity — Stories vs. Data — I wrote the following comment, which I thought I might like to share. The gist of the original post, if you don’t care to read it, is that data = mass-produced stories. While valuable to funders, and perhaps easier to gather than we think it is, data is still somehow inferior to storytelling: a point with which I disagree. Data, I believe, tells a story; it’s an alternative to the stories we normally tell, but it’s not evil. Here’s what I wrote:
The thing about data is that it doesn’t lie; people lie about data, but honestly-collected data always tells the truth. This is often why people fear it; it reveals what we might prefer to conceal, or at least to not see.
The reason we tell stories is because we want to shape narratives the way we believe they ought to be shaped. We aren’t comfortable being subject to the story told by the data. If a particular show doesn’t sell well, we’d rather focus on how transformational it was for the smaller number of people who saw it, which is where we enter the realm of anecdote and difficult-to-measure data points.
What if we COULD measure how meaningful or valuable a show was to its audiences? What if we could determine a VQ — a Value Quotient — that could serve as a multiplier for the number of tickets sold to a given show? (I’m not saying HOW we could do this; I’m merely suggesting we figure it out.) The result would be a more accurate measure of the reach or effect of our work. For each show, we’d have a formula like this:
# Tickets Sold x VQ = Adjusted Reach
Here’s another angle: what if we started measuring the impact of our work in social media? Counting the number of Twitter references to our shows, the number of Facebook posts about them, the number of blogs written about them? What if we measured the traffic to the pages of our websites on which we promote our shows? There’s a great deal of analytics data that might be useful.
The bottom line, for me, is this: we need to be less afraid of data. We need to embrace it, for all its difficulty, which is (I think) what you’re suggesting. So data isn’t the same thing as a story: so what? It is what it is, and it’s valuable on its own terms. There are valid reasons funders ask for it; let’s not forget that.
I’m *ahem* kind of a big fan of data, but I’d disagree that it doesn’t lie. You give me the same 1,000 pieces of data, and I could create 5 different stories out of them, and at least 2 would have conflicting messages. I (and any other data geek) can manipulate the data itself, or the display of the data, even which questions we choose to answer with that data is a manipulation of sorts.
Indeed to work effectively with data, I think one *must* be a storyteller. The problem comes when we divorce data from story. Stories are effective because we’ve been telling stories for a really long time. The era of big data/infographics is a little more recent. It takes time for the data story teller and the audience to get adept at understanding the construct. Much like ‘web reporting versus web analysis’: http://www.kaushik.net/avinash/2011/04/difference-web-reporting-web-analysis.html.
I had a strong feeling that someone was going to make the “data lies, you can tell different stories about it” argument. I’m honored that of all people, it was you!
However, I have to disagree. Data, assuming it’s been collected accurately and its integrity has been maintained, is neutral. In the “five different stories” scenario you’ve described, the bad actor — the one telling the lie — is the data geek, to my mind.
I agree that a data geek must be a storyteller — that’s actually sort of what I was saying. The thing is… there are lots of different kinds of stories. Lies are stories, but so are documentaries. The highest order of data geek/storyteller, to my mind, is the documentarian: the one who culls of the narratives that can be culled from a data set and weaves them into a single meta-narrative; the one who identifies the conflicts you’ve rightfully suggested are frequently found in data, tries to resolve them, and (if that fails) simply lays them bare for others to investigate.
It’s difficult, but it’s manifestly brilliant when done well.
it’s more of an academic response at this point–but documentaries aren’t objective in the way that data isn’t objective. someone has to decide what that narrative through-line is, and leave out a whole bunch of other stuff that is “true” but doesn’t help the story. Yes-a pile of numbers/shots is objective, all else being equal. But as soon as someone takes the data/shots and starts putting things in order, leaving things out, deciding on a conclusion, it’s no longer objective.
Which is why I think it’s important we talk more about *how* to read & write with data.
I do think this is an important point; yes, documentaries aren’t fully objective, but they are moreso than out-and-out fibs, spin, obfuscation, etc. Aspiration toward objectivity is still, to my mind, admirable.
I honestly don’t believe the data-driven storytelling is complete unless it acknowleges outlying data points and conflicts; nothing should be left out at all, though of course there are questions of emphasis.
As for deciding on a conclusion? Sometimes, no conclusion is the right conclusion, is it not? And sometimes a conclusion is just right. But a conclusion is not the same thing as dogma; a data-derived conclusion is only “true” in the same sense that a scientific law is “true” — provisionally and only if confirmed.
But yes, it’s important to talk about how to read and write with data.
You know I’m into data. That’s my job, my education, and my inclination. And an aspiration to objectivity is admirable. However, a human cannot even objectively gather data, because gathering data requires choosing which data to gather. Intent and aspiration are very weak countermeasures to this. The best of intentions can’t debias me or erase my culture blindspots. And those blindspots lead us to gather the data that we can see, and miss the data that we can’t. However, hearing a compelling story from somebody with different biases can at least help me understand that I have those blindspots.
And on the other side, our illusions of objectivity can lead us to fetishize data. We give data more credence just because it is data. Did our education program reach 50,000 kids or 100,000 kids? Well, we might objectively be able to answer that question with great specificity, but it doesn’t really matter unless our education program actually made some impact. We’ll use those numbers on our next grant application, anyway.
You are, of course, quite right.
The reason I wrote this post, however, was to counter what I think was a subtle antipathy toward data in the original post. I don’t believe that the difficulties with data collection and analysis are enough to suggest we should abandon data entirely or make it our enemy.
I believe data is vital, and I believe that with the right boundaries and countermeasures and checks-and-balances and feedback mechanisms in place — more, yes, than intent and aspirations toward objectivity — it can help us understand the effect our work is having. More than just the number of kids a program reached, but how powerfully it affected them.
I’m not, to be clear, fetishizing data. I’m just saying that a fear of data — because we consider ourselves “artists” or “storytellers” and are thus somehow “above” it — does us no good.
Or at least… that’s what I meant to say.
Fair enough, but I’ve actually never thought Ian to be anti-data. Rather on the contrary, he’s a tough editor (I write for Createquity) who’s always expecting evidence behind stories. Remember that other time when we were talking about wanting good editors in the blogosphere? Ian is one.
Of that I have no doubt. Honestly, it was the title of the post that most irked me; the substance of it wasn’t far off… or not farther off, perhaps, than my own post, which also seems to have missed part of the story 🙂
Gwydion, thanks for the thought-provoking post. It looks like I’m quite late to the discussion – I was traveling back from DC and this is the first time I’ve had internet access all day, alas. I just want to say, for the record, that you and I are 100% on the same page about the value and flexibility of data, as well as the potential for integrating the two concepts. That’s why I spent the two paragraphs following the “data = mass-produced stories” equation defending our use of data, and pointed out in the last paragraph that stories and data are not incompatible. The “vs.” in the title was meant to be sort of ironic, a nod to the way in which they are typically placed in opposition to each other by others. I’m sorry if that didn’t come across as clearly as I intended.
You know what? The more I return to your original post, the more I realize we are in accord here. This is yet another example of the controversy evaporating into nothingness style of post… and I apologize for my errant analysis!
No worries. 🙂
You are a gentleman!
Hi Gwydion,
I actually work in the philanthropic sector and this is a trend I’m seeing: moving toward storytelling. A lot of foundations are moving toward using stories to illustrate the impact they’re making in communities, instead of just stating how much money they gave to an organization. The storyteling trend suggests to me that funders are trying to communicate their value to society at large, to show how they are making a difference in people’s lives. And I think they want to do that with the warm and fuzzy instead of just stats on grants awarded. I think stories can convey the unquantifiable. Yes stories are “crafted” for a purpose. You’re absolutely right. Maybe it’s not a zero sum game, maybe not stories vs. data. But just recognizing that using one or the other has different results/impact on the audience/listener.
I definitely believe that stories and data serve different purposes. They aren’t each other’s enemies, that’s for sure! And that’s exactly why I wrote: to counter the “vs.” in the original “Data vs. Story” post by CreatEquity. It just seemed off to me.