Humans are used to seeing data for what it is, in many cases: an opinion. In fact, in some cases, people skew data to the point where it becomes useless, a mistruth. A computer can’t tell the difference between truthful and untruthful data — all it sees is data. One issue that makes it difficult, if not impossible, to create an AI that actually thinks like a human is that humans can work with mistruths, and computers can’t. The best you can hope to achieve is to see the errant data as outliers and then filter it out, but that technique doesn’t necessarily solve the problem because a human would still use the data and attempt to determine a truth based on the mistruths that are there.

WARNING

A common thought about creating less contaminated datasets is that, instead of allowing humans to enter the data, collecting the data via sensors or other means should be possible. Unfortunately, sensors and other mechanical input methodologies reflect the goals of their human inventors and the limits of what the particular technology is able to detect. Consequently, even machine- or sensor-derived data is also subject to generating mistruths that are quite difficult for an AI to detect and overcome.

The following sections use a car accident as the main example to illustrate five types of mistruths that can appear in data. The concepts that the accident is trying to portray may not always appear in data, and they may appear in different ways than discussed. The fact remains that you normally need to deal with these sorts of issues when viewing data.

Commission

Mistruths of commission are those that reflect an outright attempt to substitute truthful information for untruthful information. For example, when filling out an accident report, someone could state that the sun momentarily blinded them, making it impossible to see someone they hit. In reality, perhaps the person was distracted by something else or wasn’t actually thinking about driving (possibly considering a nice dinner). If no one can disprove this theory, the person might get by with a lesser charge. However, the point is that the data would also be contaminated. The effect is that now an insurance company would base premiums on errant data.

REMEMBER

Although it would seem that mistruths of commission are completely avoidable, often they aren’t. Humans tell “little white lies” to save others from embarrassment or to deal with an issue with the least amount of personal effort. Sometimes a mistruth of commission is based on errant input or hearsay. In fact, the sources of errors of commission are so many that it is truly difficult to come up with a scenario where someone could avoid them entirely. Regardless, mistruths of commission are one type of mistruth that someone can avoid more often than not.

Omission

Mistruths of omission are those in which a person tells the truth in every stated fact but leaves out an important fact that would change the perception of an incident as a whole. Thinking again about the accident report, say that your car strikes a deer, causing significant damage to your car. You truthfully say that the road was wet; it was near twilight, so the light wasn’t as good as it could be; you were a little late in pressing on the brake; and the deer simply darted out from a thicket at the side of the road. The conclusion would be that the incident is simply an accident.

However, you left out an important fact: You were texting at the time. If law enforcement knew about the texting, it would change the reason for the accident to inattentive driving. You might be fined, and the insurance adjuster would use a different reason when entering the incident into the database. As with the mistruth of commission, the resulting errant data would change how the insurance company adjusts premiums.

REMEMBER

Avoiding mistruths of omission is nearly impossible. Yes, people can purposely leave facts out of a report, but it’s just as likely that they’ll simply fail to include all the facts. After all, most people are quite rattled after an accident, so they can easily lose focus and report only those truths that leave the most significant impression. Even if a person later remembers additional details and reports them, the database is unlikely to ever contain a full set of truths.

Perspective

Mistruths of perspective occur when multiple parties view an incident from multiple vantage points. For example, in considering an accident involving a struck pedestrian, the person driving the car, the person getting hit by the car, and a bystander who witnessed the event would all have different perspectives. An officer taking reports from each person would understandably glean different facts from each one, even assuming that each person tells the truth as each knows it. In fact, experience shows that this is almost always the case, and the info that the officer submits as a report is the middle ground of what each of those involved states, augmented by personal experience. In other words, the report will be close to the truth, but not close enough for an AI.

When dealing with perspective, consider vantage point. The driver of the car can see the dashboard and knows the car’s condition at the time of the accident. This is information that the other two parties lack. Likewise, the person getting hit by the car has the best vantage point for seeing the driver’s facial expression (intent). The bystander might be in the best position to see whether the driver made an attempt to stop, and assess issues such as whether the driver tried to swerve. Each party will have to make a report based on seen data without the benefit of hidden data.

WARNING

Perspective is perhaps the most dangerous of the mistruths because anyone who tries to derive the truth in this scenario ends up, at best, with an average of the various stories, which will never be fully correct. A human viewing the information can rely on intuition and instinct to potentially obtain a better approximation of the truth, but an AI will always use just the average, which means that the AI is always at a significant disadvantage. Unfortunately, avoiding mistruths of perspective is impossible because no matter how many witnesses you have to the event, the best you can hope to achieve is an approximation of the truth, not the actual truth.

Think about this other scenario that involves perception: You’re a deaf person in 1927. Each week, you go to the theater to view a silent film, and for an hour or more, you feel like everyone else. You can experience the movie in the same way everyone else does; there are no differences. In October of that year, you see a sign saying that the theater is upgrading to support a sound system so that it can display talkies — films with a soundtrack.

The sign says that talkies are the best thing ever, and almost everyone seems to agree, except for you, the deaf person, who is now made to feel like a second-class citizen — different from everyone else and even pretty much excluded from the theater. In the deaf person’s eyes (from the perspective of their lived experience), the change is not “the best thing ever,” but a step backward. This illustrates how perspective shapes data, and why AI struggles to interpret reality without understanding these layers.

Flashcard Deck: Five Mistruths in Data