“We All Count is a rage-founded organization, but we are now a joy-driven organization”

Listen to Podcast

Like podcasts? Find our full archive here or anywhere you listen to podcasts: search Community IT Innovators Nonprofit Technology Topics on Apple, Spotify, Google, Stitcher, Pandora, and more. Or ask your smart speaker.

AI Bias and Data Equity with Heather Krause

Carolyn Woodard explores why statistical analysis and evaluations are never neutral with Heather Krause, mathematical statistician and founder of We All Count.

Heather built her career doing statistical consulting, evaluation, and causal analysis in the global south, work that eventually convinced her that the widespread belief in value neutral quantitative analysis is a myth. In this conversation, she and Carolyn dig into why every statistical task, even something as simple as calculating an average, is really a series of choices, and why those choices are never neutral.

Using a deceptively simple classroom size example, Heather shows how the same data can produce two different, equally correct answers depending on whose experience you are measuring for, teachers or students, and why that matters far beyond the classroom. She and Carolyn also talk about a dairy cooperative project in Bangladesh, the discomfort people feel when the myth of neutrality gets challenged, and why transparency, not dumbed down science, is the real fix. 

The conversation wraps with a candid look at how AI fits into all of this: it is not value neutral either, and understanding its hidden choices may be one of the most useful data literacy skills nonprofits can build right now.

Heather and Carolyn discuss:

Resources Mentioned:

Presenters

Heather Krause, a mathematical statistician with more than 20 years of experience working across government, philanthropy, education, tech, health, and the social sector founded We All Count in 2012. Heather is one of fewer than 200 PStat-accredited statisticians globally and a Knight Foundation award-winner for innovative data use in communities. While in the field, Heather developed ways to work with data that were technically rigorous, ethically grounded and socially responsible. That experience became the We All Count Data Equity Framework, now used by data practitioners, governments, foundations, universities, nonprofits, and companies around the world.

Carolyn Woodard

Carolyn Woodard is currently head of Marketing and Outreach at Community IT Innovators. She has served many roles at Community IT, from client to project manager to marketing. With over twenty years of experience in the nonprofit world, including as a nonprofit technology project manager and Director of IT at both large and small organizations, Carolyn knows the frustrations and delights of working with technology professionals, accidental techies, executives, and staff to deliver your organization’s mission and keep your IT infrastructure operating. She has a master’s degree in Nonprofit Management from Johns Hopkins University and received her undergraduate degree in English Literature from Williams College.

She was glad to have this conversation with Heather Krause about AI bias and data equity in philanthropy and nonprofits.


Ready to get strategic about your IT?

Community IT has been serving nonprofits exclusively for twenty-five years. We offer Managed IT support services for nonprofits that want to outsource all or part of their IT support and hosted services. For a fixed monthly fee, we provide unlimited remote and on-site help desk support, proactive network management, and ongoing IT planning from a dedicated team of experts in nonprofit-focused IT. Our clients benefit from our IT Business Managers team who will work with you to plan your IT investments and technology roadmap if you don’t have an in-house IT Director.

Being 100% employee-owned is important to us and our clients. It is an important aspect of our culture as a business serving nonprofits exclusively for 25 years.

We constantly research and evaluate new technology to ensure that you get cutting-edge solutions that are tailored to your organization, using standard industry tech tools that don’t lock you into a single vendor or consultant. That’s even more important now in the age of AI. And we don’t treat any aspect of nonprofit IT as if it is too complicated for you to understand.

We think your IT vendor should be able to explain everything without jargon or lingo.

If you can’t understand your IT management strategy to your own satisfaction, keep asking your questions until you find an outsourced IT provider who will partner with you for well-managed IT.

More on our Managed Services here. More resources on Cybersecurity here.

If you’re ready to gain peace of mind about your IT support, let’s talk.

Transcript

Carolyn Woodard: Welcome everyone to the Community IT Innovators Technology Topics podcast. I’m Carolyn Woodard, your host, and today I’m really excited to be speaking with a new guest on the podcast, Heather Krause, who is the founder of We All Count. So, Heather, would you like to introduce yourself?

Heather Krause: Yes. Thank you so much, Carolyn, for inviting me to join the podcast. I am a big fan, and it was very humbling to be asked to have a conversation with you.

My name is Heather, and I’m a mathematical statistician. I started my career as a single mom of three little kids, and I basically built a statistical consulting company from the jump because I couldn’t afford child care.

I did mostly work as a statistical consultant doing evaluations and impact analysis and causal analysis in countries in the global south. My very first job was in Bangladesh, and I was able to bring the kids along and have an adventurous life. And we went from there.

I think it was because I was a statistician, but I wasn’t working in the same cities or cultures that most of my statistical training came from, nor the same cities and cultures where most of the funding and methodological ideas were coming from, that I had an unusual point of view, I guess. And it was a point of view that led me to become upset by what I felt like was people imposing values and power dynamics and points of view onto communities in the name of statistical objectivity.

And I can understand that that was happening with people who wanted to promote the status quo, shall we say. But I was working for companies and with people who really had a lot of rhetoric around wanting to create empowering… they were doing statistics because they wanted to improve the situations or bring down the structures of marginalization and oppression and poverty. So that’s what the rhetoric was.

But because I was a statistician, kind of straddling those two roles, I could see that the way the data was being used was not aligned with that rhetoric. It took me a while, a decade at least, to realize that this wasn’t actually most of the time because people were bad or stupid or lazy, which is what I thought at first.

It turns out that there was just this big gaping hole in our education, in our practice, and in our thoughtfulness, that centered around this myth that quantitative analysis is value neutral. And that if it’s a number and it’s done with this magical methodological recipe, then there’s nothing we can do but shrug our shoulders and say the numbers don’t lie.

And so that kind of rage inspired the rest of my work. And that’s how I got here today. I like to say that We All Count is a rage-founded organization, but we’re now a joy-driven organization.

Why Math and Statistics Are Never Value-Neutral

Carolyn Woodard: I love that. I love that way of putting it. So thank you so much for coming here today and talking with me more about… I saw you give this presentation, and I have to laugh because I’ve been a language person my whole life, and I’m not a math person. So I don’t know how I ended up in your session, but it blew my mind, because I had never really encountered this way of looking at math.

You said, in that session, that math seems neutral, and like you just said, a lot of people talk about it like it’s neutral. The numbers are just the numbers, the numbers don’t lie. But it is never ever neutral. So could you walk… I know it was an hour-long presentation and we only have half an hour or so, but could you walk us through that presentation, a couple of the examples that you gave, to help both the math people and the language people out there who may be listening understand this concept of math not being neutral?

Heather Krause: Yeah, thank you so much for coming to that session. That was such a fun conference. I really enjoyed it.

I just want to clarify one thing: to a certain extent, I think that math probably is neutral. Math and statistics, and the application of statistics, are different things.

Two plus two is four, and that is going to be true from any social location and in any part of the world. But the same is not true when we get into statistical applications.

So, for example, let’s stay with something as simple as two plus two, and let’s pretend that we want to understand the average classroom size in a school. That’s very simple math, and it’s the kind of math we do all the time. Let’s say that there are three classrooms in that school, and one classroom has three students in it, one classroom has six students in it, and one classroom has nine students in it. It is very easy to take the average of those three classrooms and say that the average classroom size in the school is six.

And that is correct, that is not a bad or a wrong answer, but that is not a value-neutral answer. That is not the answer that is going to be true whatever your social location is in this scenario, in this real world. The average classroom size is six if you are assuming that we want to know what it feels like to be a teacher in this school.

If you want to know what it feels like to be a student in this school, you have to do the math using the denominator of students rather than the denominator of teachers. I’m not going to walk through the math here because this is an audio podcast, but I’m happy to give you a link to a YouTube video that does that. If you take the average classroom size in this school from the lived experience of students, the answer is seven. Those numbers are very close together, but only because we are using a really small school.

So when we have a task like calculate an average, that task in math and in statistics is comprised of a bunch of choices. And those choices are unavoidable. You cannot take an average without selecting a denominator, and that choice is inherently value laden.

When you get that number, the average classroom size, is it representing what it feels like to be a teacher or what it feels like to be a student? This really matters from a mathematical point of view because it’s going to change the number. And it really matters from a human point of view because only one of those groups of people is going to have their lived experience elevated as the evidence.

And sometimes people, like you said, see this example and think, oh my gosh, that’s so exciting, let’s talk about this more. And some people want to chase me down the road in anger and fury, because destabilizing this myth of quantitative value neutrality kind of crumbles the foundations of their world in a way that they’re not comfortable with. But it is true, it’s very demonstrably so.

And this is not saying that science is a hoax, or that everybody should dumb down their research to make it palatable and politically correct. What it actually is saying is the opposite: we need to toughen up our science, not dumb down our science. We need to adhere more closely to best practices and pay more attention, because this stuff is not value neutral, and it’s really easy to harm communities that you’re trying to help, with strategic deployment, with developing new policies and programs, with medicine, with, you know, you name it, big tech, of course.

So that bottom line is what I mean when I say the good news and the bad news is that quantitative evidence is not value neutral.

From Anger to Joy: Making Better Choices in Data Work

Carolyn Woodard: Yeah, it’s so interesting to me that people really do get angry.

Heather Krause: Oh, very angry.

Carolyn Woodard: And I imagine people who work in measurement and evaluation…

Heather Krause: Oh, yes. All the time. People who work in every sector. But that doesn’t bother me, because I really do understand it, since I went through a very similar education process. I myself got into the profession of mathematical statistics because I wanted to reduce uncertainty. I wanted to get some real answers to what was going on.

That’s not actually the role of statistics. Its role is not to get rid of uncertainty, it’s to help us describe uncertainty, help us understand where the uncertainty is coming from and which direction it’s pushing. It took me about 20 years of being a statistician to even start to come to peace with that. So I am not afraid or worried when people are angry.

Carolyn Woodard: I feel like this does come up in evaluation, that there is an awareness that when you make choices in the evaluation, you get different outcomes. And you had a great example about the women who were doing the milking, and what you were trying to measure versus whether that was good or bad for the people who were the milkers.

Heather Krause: Yeah, great example. That was in Bangladesh, one of the first projects that I worked on that helped me start to wrap my head around this. I believe this project is still actively underway, because it turned out to be so successful.

Essentially, long story short, this was a project designed to improve the lives of women in rural Bangladesh who own dairy cattle. Part of the pilot, part of the learning program, was teaching the women who owned cows lots of new ways to take care of them and new ways to measure the quality of the milk. All very cool stuff.

This may be hard to talk about on an audio podcast, but essentially what really mattered when we were building the causal statistical models, or doing the impact analysis, was paying attention to whose concept of the way the world works was getting embedded into those mathematical models, because those models have to be built, and they have to be built by humans.

There’s no mathematical model that can tell you this is the way the world works.

What a model can tell you is, if this is the way that the world works, then your outcome is this, or your effect size is this, or if your assumptions about the way the world works are correct, this policy is having such and such an effect. No mathematical model in the world can say this is how the world works.

And that also makes people very uncomfortable and very upset, and it makes other people very excited and overjoyed, that we have an opportunity to check if something works from several different points of view, from several different value systems. That’s what we did with the dairy project. And that’s true whether we’re trying to understand if this program helps women get more milk from their cows, or if the women are replaced by AI robots, and we want to decide if this program is getting more milk from the cows with the AI robots.

It doesn’t have to be about women who live in Bangladesh. It’s about any mathematical model and what we assume to be the way the world works, and the directionality of success. That’s what I try to emphasize.

When I was first starting out with We All Count, I was very angry, and I would mostly spend my time trying to prove how what people were doing was wrong, or was harming the people that they were claiming to help, and that they had all these choices and weren’t paying attention to them.

And then I realized that was an unpleasant way to be in the world, for starters, and it also was totally ineffective.

Most of the people that I work with actually want to make meaning with and for the people they claim to make meaning with and for. There was just this giant gap in the way that a lot of data practitioners, myself very much included, were taught how to work through a data project. So yeah, it’s a lot.

It’s a lot more joyful now. And I try to emphasize that. When people say, oh my gosh, I have to start making all of these choices, I say, no, actually, you’re already making all of those choices. If you’re taking the average classroom size, you’re already making a choice, you’re already choosing a denominator. You don’t have to start making more choices, but you do have to start paying attention to your choices. And that’s actually an opportunity to do the best data project you possibly can, the most useful data project.

How many data practitioners get to the end and just watch people not use their answers, or not really implement their evidence? Part of that is because it’s not usually actually useful. Paying attention to your choices will get you more useful, more meaningful, more rigorous evidence. So I try to emphasize the upside of it.

Trust, Transparency, and What AI Means for Data Equity

Carolyn Woodard: I think there are also situations where people don’t trust the evidence, I’m going to put that in air quotes, because if what your statistical model comes out with, saying is the average class size, doesn’t relate to their experience or their perspective, then it seems invalid, even though you did the math, it’s not reflecting their perspective.

So can you talk a little bit about going forward? You talked a bit about how you can take different perspectives and compare them, so you can make different assumptions when you’re running your math to figure out what you’re trying to find out. If you’re listening to this and you’re kind of stuck in the well, I thought there was just one way to do this, can you give us advice?

Heather Krause: Yes, I can try. And I’m so happy, Carolyn, that you’re asking me this question, because this is another place where people get mad, but from a different perspective, which is: even if what you’re saying is true, you are now feeding people, like, you know, climate deniers everything that they would ever want to hear. Like you’re saying that science isn’t trustworthy, how could you, stop.

I thought about that for a long, long time. I take that very seriously. I don’t just glibly carry on. I spend a long time talking to people about the impact on their lives. And I think that we are in a very tricky period of time right now, where absolutely evidence is being used nefariously, evidence is being used politically, evidence is being manipulated. So I think that not always trusting evidence, regardless of your country or your social location or your politics, is a pretty good idea at the beginning.

I spent a long time working with journalists. I really love the profession of journalism. There was this big data journalism wave about a decade ago, and I was a part of that. One of the things that I used to try so hard to get journalists to do was to make the steps they were taking to make their calculations or get their estimates or build their maps really transparent in a non-technical way. I don’t mean transparent in an academic paper way, where we’re listing the confidence intervals and the Bernoulli adjustments and things like that, but in a non-technical way, like a way that you could explain to your five-year-old why you’re making her a bowl of spaghetti.

Because once you can do that, you really do understand your evidence. And again, the well-resourced, very intelligent, well-meaning journalists that I was working with were terrified to do that, because they said if we do that, people will not trust the science. And they were not wrong, but at the same time, not being transparent is not the solution.

I’m not a policymaker, I am not an ethicist, and I am not necessarily going to be the person who has the perfect plan to get us out of this conundrum, off the double or triple or quadruple-edged sword that we are on right now around evidence. But I do know for sure that transparency has to be part of it.

And that is what we are promoting. We are not saying that to be equitable, you have to use the denominator of teachers, or to be equitable, you have to use the denominator of students. What we’re saying is, to do good science, you have to tell us which one you used, so we can decide whether that answers our question or not, or whether the evidence you’re generating is trustworthy and aligned with the work we’re trying to do. And that is a tough, tough sled right now in a lot of areas.

Carolyn Woodard: You mentioned AI previously, just the AI robots that are going to milk the cows. And I don’t want to end on a downer, but I was very struck in your presentation that you talked about this tendency we have, especially when it’s something we’re not an expert in, to accept something that’s presented in a very confident way, maybe not without all of the assumptions spelled out in a way that we can understand, but it’s just presented as, well, here is the thing.

And it struck me in that presentation, and just in life, that that is something AI does a lot, right? That’s one of the things it’s well known for. It can have a wrong answer, but it will tell you very confidently.

Heather Krause: Correct.

AI Bias and Data Equity

Carolyn Woodard: So can you leave us with your thoughts on the ways we fact-check AI, and how we should think about those confident answers, in the way we think about science when science is very confident about something?

Heather Krause: Yeah, I think it’s a great question, and I don’t think it’s a downer at all.

I think that we are still in the very, very early stages of AI. And I think that we as a society could decide to make the best of it and really transform some things into wonderful situations, or we could decide to make the worst of it and really suffer the consequences.

And again, I am not an ethicist or an AI expert, but I do know that, at least right now, when you’re working on a data project, AI is not doing anything different than you or I or our data teams are doing. You ask AI, you give AI the school and say, here are these three classrooms, what’s the average classroom size in this school? And the AI is going to very confidently tell you whatever answer it’s going to tell you.

Most AIs that we have tested are going to take the teacher’s position and tell you that the average is six. And it is not going to be transparent about the choices that it made.

So part of the good news, if we do decide to find a way to use AI that is not going to destroy the earth or our human selves, would be to use AI to make the choices more transparent.

Lots of things about AI do free up time. And one of the things we can do with that time is produce more rigorous, more accurate, more equitable, more transparent science. I hear every single day, people say, I would love to take the time to really consider each of these steps in my data project and document them, but my boss would never give me all this time. And I say, well, if you’re going to use AI, you’re going to have more time.

And you do not want AI to be making those choices for you. Because if there’s one thing we know for sure, AI is not value neutral.

No AI model in the world is value neutral, or is going to be in the future. Predictive algorithms cannot be value neutral, it’s a mathematical impossibility. That doesn’t mean we shouldn’t use them, but it does mean we shouldn’t think that they’re value neutral.

I think it’s possible that AI could be really good news for data equity, in that it could help us think, in a series of structured choices, about each task we’re doing, and make sure that we are making those choices in alignment with the communities, the values, and the purpose of our projects. So I think that question actually ends us on a hopeful note.

Carolyn Woodard: Me too. I see so many opportunities with AI, and I think, like everything, especially as you get to know more about its limitations and what it’s good at, then you can take advantage of that, use it to do the things it’s really good at, like finding patterns, and maybe take some of the busy work off your plate so you have more time to design better data structures, or what have you. So yeah, I think that is a positive note. So thank you.

Heather Krause: Yeah, of course.

Carolyn Woodard: Well, thank you, Heather, so much for your time. And I am going to share in the show notes, you said you would share a link to the math problems, and then of course we’ll share your company, weallcount.com. You can find out more information about Heather and what her company does around this data collection.

Heather Krause: Thank you for having me.

As advocates for using technology to work smarter, we’re practicing what we recommend. This transcript was drafted with the assistance of AI, and is not a verbatim transcript. The content was edited for clarity, and was reviewed, edited, and finalized by a human editor to ensure accuracy and relevance.

Photo by Ian Taylor on Unsplash