This summer I worked with Professor Benjamin Ho from the Economics department studying NGO partisanship. This is a rather funny question to tackle as Non-Governmental Organizations are legally required to be non-partisan. They cannot support or oppose  political parties, candidates, and the like or they face the risk of losing tax-exempt status. However, NGOs can take positions on political issues. Because of this, when looking through a list of popular NGOs, one usually has some intuition on whether it is right or left leaning, despite the non-partisan status. A good is example is the organization Planned Parenthood, which has associated itself with liberal values.

While it is not surprising that NGOs have partisan bias, there is a lack of research that specifies how much, or to what degree. We wanted to give every organization some sort of score, which would then allow each to fall somewhere on a partisan scale. We decided to utilize CharityNavigator, a website that evaluates charities’ performance by analyzing their financial records, accountability, and transparency. This provided our list of organizations to study. There are a few categories that CharityNavigator collects for each organization including but not limited to: program expenses, tax information, mission statements, etc. With a wealth of information on each charity, we then had to figure out what was relevant to us on our quest to tease out partisanship. So began my first task: researching the different methods people have used to determine partisanship.

Naturally, the are several ways to do this, but the method that became apparent to use was some form of text analysis. This was surprising to me at first, and slightly daunting. If my task was to create some sort of numerical score, the last thing I wanted to work with was words. Luckily it’s been done before. At this point it’s common knowledge that Fox news is more conservative than the New York Times, but the research that proves this was done by researchers Matthew Gentzkow and Jesse Shapiro. Here is their basic idea: liberals and conservatives use different vocabulary, therefore language choice is an indicator of partisanship. They created a liberal and conservative corpus (basically a compilation of words) by collecting congressional records (a daily document that contains all the speeches made in congress) and sorting each speech based on whether the speaker was a Democrat of Republican. Then, they compared the language from various media outlets to each corpus and developed similarity scores.

I set out to do the same thing, which meant I had to first learn some technical skills. As I mentioned before, CharityNavigator conveniently has each organization’s mission statement (we would use these to compare to the liberal and conservative congressional documents), but there are over 8,000 charities. Collecting these mission statements by hand would be impossible. Likewise, the daily congressional record PDF document is usually over 200 pages. I had to learn how to webscrape, which is an automated process of collecting data off of web pages, and to do this, I had to learn the coding language Python.

Luckily, I have coding experience, so learning these skills wasn’t too difficult and actually ended up being one of the most satisfying parts of this project. The idea of having to somehow collect 8,000+ mission statements was panic-inducing, but once I had worked out the code, pressed the ‘run’ button, and watched as all the information I wanted showed up in my excel document, I finally had confidence that I could successfully complete this project.

We decided to use Bayes’ Theorem as our text analysis method, which again was inspired by work done by Gentzkow and Shapiro.

We use this equation to answer this question: Given a certain word or phrase, what is the probability that the speaker is Republican?

We did this for all relevant words and phrases in each mission statement. Let’s take a look at the ACLU Foundation of Maryland’s mission statement: “The Maryland ACLU works to ensure that all people in the state of Maryland are free to think and speak as they choose and can lead their lives free from discrimination and unwarranted government intrusion. We are guided in our work by the United States Bill of Rights and the Maryland Declaration of Rights. The Maryland ACLU acts without partisanship to achieve these goals. We work in the state and federal courts, the Maryland General Assembly and local governments, and in communities across the state to realize this goal.”

It explicitly states that it is a nonpartisan organization, but it’s compilation of words and phrases have a .18 score, which indicates that democrats are more likely to use the language that this organization used. In turn, we see this as an indicator that this organization is more likely to be liberal-leaning. We attempt to verify this by checking the voting data from the county the organization is located in, the assumption being that liberal-leaning charities are more likely to congregate in liberal-leaning counties. While this is not accurate for every single charity, it helps us gauge the accuracy of our Bayes’ theorem method.

Overall, our findings have mostly lined up with our natural intuitions, which is a good sign. The nature of this project doesn’t quite lend itself to discovering groundbreaking results – if our measure said Planned Parenthood was actually conservative, it would cause me to take a second look at my code rather than get excited about publishing my surprising findings. This has been the first time that I’ve looked for comforting results, rather than exciting ones.

Our next stage is a survey that will examine the link between people’s partisanship and their relationship and perception of these very same organizations that we’ve already scored. Whether the results are comforting or exciting, I’m greatly looking forward to continuing with this project and am incredibly grateful for the experience I’ve had this summer.