More data won't solve our problems
Rethinking our obsession with knowledge in the social sector
Hey folks. This is the 12th edition of The Great Near. This post is for anyone in the social sector who seeks knowledge to understand the root of big problems. It’s about a billion-dollar industry of “knowledge curation,” from white papers and research reports (how we diagnose problems) to impact measurement and evaluation (how we determine if our solutions are working), and how we’ve gotten it oh-so-wrong. A heads up, this is a bit more ~*niche*~ than our usual programming, so if that’s not your jam, I’ll get back to more explainer-y stuff next time.
—
*enters confessional booth*
I’ve rewritten this piece three times. Why? Because I struggle with ambiguity. I want to know how we get from A to B and every possible route and contingency plan. I like when I can put things in neat little color-coordinated boxes.
But the question I’m curious about doesn’t have the crisp, satisfying answer you and I both want. It strikes at something core to what makes us human: our search for knowledge and fundamental truths. It summons a belief that rational, data-driven inquiry is the key to unlocking progress (though I question this in 2021…).
To solve social inequalities, how do we find and resource the best solutions?
It’s a question that drives well-intentioned philanthropists, policymakers, impact investors, advocacy groups, consultants, and others invested in finding a shortcut to impact. To turn back the clocks on our warming oceans. To close educational inequality. To put every dollar to its most efficient use.
I think the question obscures a few assumptions. It assumes that there is a “we” that should be “finding” solutions. That there is a “best” way to get from A to B. That one solution or organization working in a silo might solve a systemic problem.
But only a select few have the power to ask this question. What if communities identified the knowledge gaps, not funders? What if we valued systems change work as much as direct services (and accepted that the impact might not be as clear or immediate)? What if we stopped pursuing data to find magic bullets?
If you don’t read any further, there’s your tl;dr.
I do believe that we should seek knowledge. The discovery that the median net worth of African-Americans in Boston is eight dollars, for example, provoked an urgency around the racial wealth gap that moved many to action.
But something is broken about our top-down approach. Here are 10 suggestions for flipping the power dynamics and allowing communities to own their own knowledge.
Ask if you really need the data— I’m all for science. The pathway out of this pandemic only exists because of evidence-based medicine, population health data, and magic vaccines. Still, this data-based orientation has been applied too broadly to the social sector to identify what works and what is “ineffective.” This tends to fail because people aren’t as predictable as a virus responding to a vaccine.
The authors of this SSIR essay argue that every dollar spent on poorly conceived research to find more efficient solutions for the future is a dollar wasted on helping people today. Under the guise of “doing the most good with every dollar” is distrust of Black and brown communities. It’s a way to kick the can down the road that translates to: “prove that your solution works and we’ll consider funding you.” This quest for proof diverts dollars that could be spent on doing the work towards reports or toolkits that end up sitting on a shelf. It also promotes upwards (toxic) accountability to funders that shifts a nonprofit’s attention away from community interests, and towards stale, staid metrics that help funders tick their boxes.
Think about the world-changing businesses from the last twenty years that received millions of dollars to pilot insane ideas. How much market research was done to design the iPhone? None. As Steve Jobs said, “A lot of times, people don't know what they want until you show it to them." Yet community organizations aren’t granted that same trust to experiment. It’s time we stop asking for strategic plans and start funding BIPOC leaders to just do the work.Do your homework — Say you’re a funder who wants to understand what Black-owned small businesses need. Is it crazy to do a basic Google search before investing $1M in a listening tour around America? Here’s what that turned up for me: AEO has a fab report on Black business ownership, Reimagine Main Street put out a survey of business owners of color affected by COVID, Brookings has a great report on investing in Black businesses, and experts like Tim Lampkin and Jessica Norwood have spoken at length, for years, about what Black entrepreneurs need in their communities. This took me all of 15 minutes to find, granted I have some familiarity with this space. While we should be cautious about generalizing insights, even within demographic groups, seeking publicly available information will help to avoid basic redundancy.
Pay people (more than you think) for their insights — We rarely question the market value of consultants and entry-level bankers—even when they have no related experience—yet funders pinch pennies when it comes to compensating community leaders for their insights. When they do, the time and value that they provide far exceeds their compensation, as this report argues. People who lead economic development initiatives, nonprofits, and other community groups are constantly asked to be local experts and connectors. (“Can I pick your brain for a sec?”) These leaders put their relationships on the line for funders in earnest hopes that a project will yield additional investment for their communities. In my conversations, many shared how the money usually dries up after projects are complete—even those bankrolled by million-dollar institutions. If you are seeking community insight, or perhaps asking for a 25-page impact report as a condition of your funding, please provide appropriate compensation.
Measure what matters — Rodney Foxworth, our CEO at Common Future, shared this about measuring success: “If our work is about resource allocation and shifting power then the question is simple: did we do that? Did we move capital to BIPOC community organizations? Do they feel like they have more power and agency because of it? If the answer is yes, then job well done. We shouldn’t need much more than this to demonstrate impact.”
60 Decibels, an impact measurement company, takes an interesting approach that is centered on listening and dialogue, as a way to help communities do their jobs better. This means leaning into the qualitative or the anecdotal, and making data readily available to guide real-time decisions. Sasha Dichter, Co-Founder of 60 Decibels, also reminded me that it’s not always so easy to measure what matters. We can strive to do it, but as Banks Benitez of Uncharted points out in this essay, it’s challenging to measure cause and effect when there are so many variables that affect the systems we operate in.
Be explicit about your intentions — I’ve mentioned the “ecosystem analysis” a few times: a type of project that maps out the specific needs of a community or sector. You can imagine that local stakeholders who participate in these projects might reasonably expect funders to actually invest in the needs that the project uncovers (crazy right?). Yet the report is often the final product. For a “good job” sticker, funders and researchers could be explicit and honest about their intentions. For the gold star, future investments or implementation grants could be baked into all community assessments. Note the power dynamics that exist when one party poses the question, dictates the research terms, interprets the data, and funds the work.
Shoulder the burden — My colleague often reminds me that sharing or ceding power doesn’t mean doing less. It means using your unique strengths to unburden community partners from doing more labor. Is your foundation funded by one of the leading tech platforms? Does your firm work with systems or IT specialists? If you’re going to fund organizations to produce data or insights on your terms, try offering resources to make their work easier. Recently, our team partnered with three community organizations to launch a character-based lending fund. In edition #10, my colleague Eric Horvath talked about the power dynamics that come with being the primary funder. Instead of raising capital and calling it a job well done, we built partnerships for legal resources and backend loan infrastructure. We also designed an evaluation tool in partnership with each of the community lenders and built dashboards to provide real-time data on each loan. While the pilot is in its early phases, we hope that these resources will be time-savers for our partners.
Play the long game — At Common Future, my colleagues often share that the impact of our programs often goes beyond what is immediately observed. We believe the same principle applies to our grantees, so we don’t make funding contingent on short-term wins. Vu Le says it best: “Data usually just reveal short periods in history, as longitudinal studies are time-consuming and expensive. The risk of that is that sometimes we fail to see whole systems and ecosystems and how different elements affect one another. Solutions based on these data, then, may tend to focus on the short-term gains vs. systems change.”
Prioritize influence — If a white paper is written and no one is around to read it, does it have any impact? I love a good report, I really do! But it’s 2021: the age of Tiktok and Vox Explainers and New York Times infographics that will blow your mind. I’m not suggesting we package all content in 30-second soundbites, but so many great insights get lost in hundred-page reports that no one has the patience to read. We’ve been investing in knowledge management 💁♀️ and communication at Common Future, because we believe that shifting narratives through digital media is key to building a movement. If you find yourself saying “we need data,” think about who it serves and what packaging (maybe it’s Tiktok!) is needed to reach your audience.
Put philanthropy in the public domain — What does this have to do with anything? Well, we’ve outsourced the responsibility of changing the world to the billionaires and descendants of billionaires. Thus, the responsibility of picking the winners—and validating those choices by asking grantees to spit out more impact data—falls to them. So many white papers and research projects are born from these quests for knowledge. Instead, philanthropy could invest way more in the capacity of community leaders who have been most impacted by injustice. Then, they might own their own research, curate data that is equitable and actionable, and write their own narratives. Assuming every foundation isn’t just going to give away all their money, building real relationships with communities is a decent first step. For more, I highly recommend Rodney Foxworth’s loving critique of philanthropy.
Build real relationships — We look to data to reveal the signals through the noise, but there are other ways to find the signals. With a job title that includes “knowledge and insights,” it’s funny to spend a whole piece arguing that more information won’t speed up the pace of change. As I’ve said, there are plenty of knowledge gaps that merit research, but relationships can also be a good proxy. Because our work is grounded in relationships at Common Future, we find ourselves in an ongoing two-way conversation about what communities need. In times of emergency, like the start of this pandemic, it was easy for us to reach out to this network and direct grants where they were most needed. We didn’t research the “state of nonprofits during COVID” or design arduous application processes—we just did it. Relationships are a keystone that many funders either skip or outsource to organizations like ours. But trust cannot be outsourced.
Trust means letting communities define their own success. My colleagues and I will respectively push back on our funders when they ask us to report back on things that don’t matter to our partners. Instead, my team asks our partners to define their own benchmarks and provide anecdotal, short reports that share impact in their own words. As my colleague, Jess Feingold writes: “Instead of asking for their plan, ask for their process. Instead of evaluating their numbers, analyze their approach. Trust their wisdom, even and especially when what they know doesn’t fit your frame.”
~*~*
Perhaps I could have summed up this whole piece with a James Baldwin quote: “not everything that is faced can be changed, but nothing can be changed until it is faced.” It’s time to own the knowledge we have and face today’s problems. It would be a far more efficient use of resources and a small act of kindness toward community leaders who just need money to do the damn work.
Many thanks to Vu Le and his writing on weaponized data, rethinking accountability, and our fixation on innovation, Ingrid Jacobson, Rodney Foxworth (Common Future), and Sasha Dichter (60 Decibels).
What’s Up This Week? 👀 ✨
My team has written all about experimentation and failure on Medium this week!
This news out of Oakland: 😍 A collaborative of community groups are in the final running to shape the city’s next comprehensive plan. If they win, this will be a first.
Native Women Lead (profiled in #10) was awarded $10M in the Equality Can’t Wait Challenge 🎉🎉🎉.
My friend Trevor Smith writes a daily(ish) news digest on reparations.
This webinar recording from Social Venture Circle about why, when, and how we can reform the venture capital industry.
until next time,
Caitlin