PolitiFact’s Biased Selection of Statements to Fact-Check

 

By MATT SHAPIRO

In the first piece in our series on PolitiFact bias, we looked at how PolitiFact generally rates Republicans as less honest than Democrats and how their word count betrays the need to constantly explain why Republican statements that seem true are actually false.

While this approach gives us some insight into how PolitiFact skews against Republicans, another important component is which facts they choose to check and not check.

Obviously, it is extremely hard to pin down exactly which facts PolitiFact declines to check. We could argue all day about individual article after individual article, but how do you show bias in choosing which statements they choose to evaluate? How do you look at the facts that weren’t checked?

Our first stab at this question came from asking which lies each candidate was famous for and checking to see how PolitiFact evaluated them. These are necessarily going to be somewhat subjective, but the results are still instructive.

With Donald Trump, we discovered the lies that were best known were usually checked and frequently re-checked later on. All but one was marked as “False” or “Pants on Fire.”

Donald Trump’s Top 5 Lies

1) Claimed he did not support the Iraq War

6 Checks

Mostly False False, False, False, False, False

PolitiFact checked this.

And checked it and checked it and, just in case they missed something, they checked it again.

Then they re-checked it.

They checked this exact statement six separate times, finding it false each time. While it is definitely false, keep in mind that PolitiFact frequently points back to the sum total of all their checks when determining which politician is more or less truthful, so this lie is counted as six separate falsehoods.

2) Claims over President Obama’s birth certificate

7 Checks

False, False, False, False, False, Pants on Fire!, Full Flop

Politifact checked Trump on issues related to President Obama’s birth certificate a total of seven times. Three times on whether Hillary Clinton had anything to do with starting the birther rumor, two times on statements that were skeptical of Obama’s birth status, once on whether he brought the issue to a close and, finally, a “Full Flop” rating when Trump admitted that Obama was born in Hawaii.

3) Claimed he did not have a relationship with Vladimir Putin

1 Check

Full Flop

They checked this and rated it as a “Full Flop” (Trump said it and then claimed he never said it).

4) Claims of large-scale voter fraud

5 Checks

Pants on Fire!, Pants on Fire!, Pants on Fire!, Pants on Fire!, Pants on Fire!

Politifact rated Trump’s claim that there was large scale voter fraud on and before Election Day as “Pants on Fire.” We tried to limit this to this very specific claim. Trump made several other statements related to the idea of voter fraud that were also checked and found to be false.

5) Claimed Hillary would let 650 million immigrants into the US

1 Check

Pants on Fire!

They checked this claim (which wandered between 600 million and 650 million) and found it to be “Pants on Fire.”

Hillary Clinton’s Top 5 Lies

The Hillary Clinton Politifact story was far stranger than we anticipated. Not only does PolitiFact not check Clinton as frequently, but they also often fact-check her by proxy.

If that “fact-check by proxy” concept confuses you, it confused us too. Strange as it may seem, PolitiFact would often look at someone talking about a Clinton statement and then fact-check the person talking about Clinton’s statement instead of fact-checking Clinton on the statement she made.

Of these five statements, PolitiFact checked Hillary four times and did two “fact checks by proxy,” compared to the 21 fact checks of Trump’s own five statements.

1) Claimed she didn’t send classified emails

2 Checks and 1 “Check by Proxy”

Checked David Cicilline – Half True, False, Mostly True

Clinton stated repeatedly that she did not send out any classified emails. PolitiFact checked this a few times as the story unfolded and gave it a variety of ratings.

Their first rating was when they fact-checked not Clinton, but David Cicilline saying that the former secretary of state claimed she didn’t send any classified emails. They rated Cicilline’s claim “Half True.” Nine months later, they would rate Clinton’s statement that she didn’t receive or send classified info as “False.”

Two months later, they took her claim that the emails didn’t have classified headers and rated it “Mostly True.”

2) Claimed she landed in Bosnia under sniper fire

1 Check

Pants On Fire!

PolitiFact checked Clinton on this claim during the 2008 presidential primary and rated it “Pants on Fire.”

3) Claimed she was in New York City on 9/11

Not Checked

We did not find any PolitiFact fact check on this statement.

4) Claimed the FBI was fine with her email use

1 check

Pants on Fire!

PolitiFact checked Clinton on this claim in August and rated it “Pants on Fire.”

5) Claimed she was named after Sir Edmund Hillary

1 “Check by Proxy”

Mitch McConnell – rated “Half True”

PolitiFact did check this statement, but only by proxy. Mitch McConnell noted that Clinton has a history of lying and included her statement that she was named after Sir Edmund Hillary. PolitiFact verified that he was correct, but proposed that it is possible a young Hillary’s parents lied to her about it. Because they could not verify that this was an intentional lie, they rated Mitch McConnell’s statement as “Half True.”

The simplest explanation (and an argument I fully expect to see people make when they see these charts but don’t read this article) is that when Hillary gets caught in a lie, she stops lying while Donald Trump continues lying.

This is not something we felt could be backed up by evidence. Barack Obama continues to this day to claim that women make 77 cents on the dollar as men. PolitiFact rated this as Mostly False in 2012, but changed their rating to Mostly True when Obama made the same statement in the State of the Union but with fewer details. Their excuse was that Obama’s claim was misleading but technically true if you look at data with the least possible context. (This is a standard PolitiFact prefers for Democrats but abhors applying to Republicans.)

Obama (and most Democrats) continue to this day to make this incredibly misleading claim.

Yet Mitt Romney was checked five times (each one false) for making the arguably true statement that Obama apologized for the US when visiting foreign countries after being elected in 2008. PolitiFact found that Romney was right that Obama expressed regret over recent US policy, but proceeded to go on a 3,000 word diatribe about what exactly constitutes an “apology.”

The pattern is there: PolitiFact checks Republicans repeatedly for the same facts, especially if they feel they can rate them “false.” Keep in mind that PolitiFact routinely touts their aggregate truth ratings, so running up the “false” scoreboard on Republicans matters when they release their “who is more honest” data stories.

Even so, this review of candidate lies suffers from the fact that we have to, by necessity, pick and choose the facts to review and look individually to see if PolitiFact has checked them.

It would be instructive to see how often PolitiFact chooses to fact-check individual politicians. Let’s take the fascinating case of Marco Rubio.

The Senate Candidate Who Was More Important Than Harry Reid’s Entire Career

Marco Rubio was the junior senator from Florida for nearly six years before coming in third place in the 2016 Republican primary. And yet PolitiFact was checking him nearly every month since a year before he was elected to the Senate.

Marco Rubio was fact-checked as many times before he took office than long-time senator and Senate Majority leader Harry Reid has been checked in his entire career. Before he even started his presidential campaign, Rubio was fact-checked 10 times more Democrat Sen. Elizabeth Warren has been during her tenure.

Below is a segmentation of fact checks for Hillary Clinton, Marco Rubio, and Harry Reid, showing where in their careers they have seen PolitiFact check their statements.

Before he even announced his run for president, PolitiFact had checked Marco Rubio more times (87 fact-checks) than they had checked Hillary Clinton during her entire 2008 presidential run (83 fact-checks).

It seems over-eager to check a four-year junior senator more than a presidential candidate, more than the vice president, more than the Senate majority leader or the speaker of the House.

Nor was Rubio alone in this. Before he ran for president, Cruz was fact-checked 46 times, more than any Democratic senator we checked. Paul Ryan was fact-checked nearly twice as often as Nancy Pelosi, even if you take out every single fact-check from the time period in which he was the vice presidential nominee. Michele Bachmann (remember her? Three-term congresswoman who placed sixth in the Iowa Caucus?) was fact-checked a stunning 51 times, nearly as many times as Governor, Senator, and Vice Presidential candidate Tim Kaine.

This would of course be a difficult metric to keep track of on a case-by-case basis, which is how fact-checking is done. But it is clear from the numbers that PolitiFact will, especially in off-election years, seek out a speech from a prominent Republican delivered at a conference or assembly and select a fact from the speech that they feel is inaccurate.

There are predictable patterns to how PolitiFact chooses their facts. Debates and State of the Union speeches are safe bets, which runs up the numbers for presidential candidates. But outside of presidential years, it often feels as if PolitiFact authors get their news from the Daily Show and pick the statements they see there as ones that deserve particular attention.

Matt Shapiro is a software engineer, data vis designer, genetics data hobbyist, and technical educator based in Seattle. He tweets under @politicalmath, where he is occasionally right about some things.