Data Quality Series Part 2: Monitoring

Imagine you own a restaurant, and you’ve maybe had some issues when it comes to cleanliness. In fact, you’ve been fined by the city for a few too many violations.

If you could somehow find out the next time the health inspector was going to be stopping by for a visit, you could make sure the place was spotless. All the tables would be wiped, the kitchen mopped clean, the food would be stored at the right temperature, and the staff would be wearing all the required gloves and masks and hair nets.

Guess what – you passed your inspection! Now you can go back to cutting corners. Never mind a few roaches in the kitchen, as long as you know the next time the health department is going to come calling.

Of course, we’d like to believe that most restaurateurs are far more honest, and dedicated to cleanliness, than the above example. Unfortunately we can’t always say the same about the sometimes sketchy market for digital marketing data.

It stands to reason that in 2020 data-driven marketers want all the high quality data they can get. They certainly don’t want to waste time with unreliable or ineffective data. More importantly, they don’t want to get themselves in trouble with consumers or regulators. The worst thing a brand can be accused of these days is not being respectful of people’s privacy.

So naturally, every CMO aims to collect data only from trusted sources – ideally sources that have been vetted, or even better ‘accredited’ by an authoritative body.

But what does accredited mean in this context? Does it mean that a data broker has its data practices fully audited once or twice a year? Or does it mean that someone is always checking in?

From our experience, the majority of data vendors and brokers aspire to do things in an ethical manner. Ideally that means being transparent regarding the methodology they employ to collect data and how they deploy said data while still protecting consumers. However, it’s very difficult from the outside to tell which companies are doing the right thing, and which aren’t.

Which is what regular audits or accreditations are supposed to address. Overall, the fact that organizations like the MRC, IAB, BPA and others are all developing various data accreditation programs, and/or conducting audits designed to give marketers some sense of the validity of consumer data is a good thing. The collective effort of these bodies serves to elevate the importance of data scrutiny, and hopefully gets us closer to a place where the majority of providers do things the right way.

The problem is, these audits are more the equivalent of a twice a year restaurant inspection. And in this case, the brokers always know when the inspector is coming. This makes it easy to “fake it till you make it” for the audit, saying and doing the right things in order to pass the audit, then returning back to less desirable practices once the audit is complete.

Not only does that dynamic potentially let data brokers skate through a few audits and do what they want most of the year – but the very idea that audits exist creates the perception that the data market is cleaner than it is.

That’s why it’s important to think about what the occasional data audits do and don’t accomplish. Most such accreditations provide some sort of verification of segment characteristics and source information. Which is certainly helpful. Yet these certifications don’t indicate whether the auditor was provided with the kind of access needed to truly assess the company’s data processes and standards. They don’t necessarily delve deeply into each data provider’s level of compliance to new and emerging privacy laws. Nor do they always assess data providers’ ability or commitment to protect consumer privacy. And in many cases, vendors are able to claim they follow certain practices without anyone really checking.

On top of all of this, there is no ongoing validation that the provider’s processes and procedures continue to meet the auditor’s standards post official audit review. That’s awfully risky given the massive challenges facing the marketing industry.

Indeed, these infrequent, high-level examinations should not be confused with a full data audit examining quality, collection methodology and transparency that incorporates ongoing verification of compliance.

In our view, a comprehensive audit combined with ongoing monitoring is the only way brands can ensure that the initial quality standards that a provider was validated against continue to be upheld and that any issues that arise are resolved as quickly as possible. To put it another way, the industry needs the threat of a restaurant inspector popping in at any moment in order to truly ensure that data quality and compliance becomes a way of life versus a means to pass the auditor’s test.

That is essentially why a third party like a Neutronian exists. We get inside companies and conduct a full audit. And we keep coming back. And back. Our process entails an in-depth, comprehensive audit that serves as the onboarding to our SaaS platform. Once the initial audit is complete, we work to set up ongoing monitoring of the provider’s data feeds and policies, allowing us to trigger alerts anytime there is a significant deviation from the baseline.

While we conduct re-audits throughout the year to document fundamental methodology and process changes, it is our ongoing monitoring that allows us to have a constant read on the quality and compliance of the data provider. We can alert them any time a potential issue is detected and work with them to ensure the issue is resolved up to industry standards.

We’re the inspector that shows up without calling ahead. We keep our clients on their toes.

If you’re a brand, that’s exactly the kind of restaurant you want to patronize.