Do you even know me? Yes Google I’m talking to you.
You know when you’ve let your partner down, when they ask you: “Do you even know me?”
After not withholding any information from Google for years, it turns out Google has no idea who I am. And it’s not just me. I’ve set up a small research project to check how close Google’s Ad Personalisation Settings match one’s actual situation and interest. The gap led me to reconsider a number of assumptions I made about data and targeted advertising.
Google doesn’t really know me
We assume that Google, or Facebook for that matter, knows everything about us. Netflix documentaries, news outlets, governments, just anyone will warn us that these platforms rake in all of our data traces, connect and enrich them, and build a complete profile of us. The New York Times, could even track down a number of Capitol stormers based on the data they trickled to advertising platforms.
I also assumed that digital clones of myself are living somewhere in marketing clouds. Until I checked my Google Ad Settings — these are the preferences that Google keeps about us to personalise Ads. I found out that Google thinks I have no children, and that I have an interest in BMW and pets, more specifically cats and dogs. Well, I have 3 children, I drive a Fiat 500x — hardly a driver’s car — and I’m scared of dogs.
Google doesn’t really know most of us
To check out if other people experience a similar gap, I set up a small study with 52 respondents. I realise that the sample size and composition does not represent the entire population, but this is sufficient to see general trends. With a homegrown Chrome extension the respondents’ Ad Settings page was turned into a questionnaire. For each Ad Setting, the respondents would answer to which degree their Ad Settings matched their actual situation or interest. Answers could be: no (0% match), not really (33% match), somewhat (66% match) and yes (100% match).
The results are stunning. Only 52% of the Ad Settings that are used to target us, match reality somewhat or completely.
Google seems not very good at inferring personal details
Demographic accuracy is not impressive
Google is relatively good a guessing ages, with a match score of 92%, but take into account that some age ranges span 26 years or more.
Google is average at inferring one’s marital status (‘Married’, ‘In a relationship’, ‘Single’), and plain bad at parental status (‘Parents’, ‘Not a parent”) and diplomas (‘High school diploma’, ‘Bachelor’s degree’, ‘Advanced degree’, ‘Higher university degree’).
Nothing says “I know you” more than knowing one’s musical taste.
The average match score across all musical genres is a low 38%. Google thinks that 52% of the respondents have an interest in Metal, while most respondents state they have no interest in this genre. Folk music, country music and classical music don’t do much better. However Google thinks more than half of the respondents have an interest in these genres. Google states that it estimates our interest in music, based on activities such as Search and YouTube.
A disappointing match between our Ad Settings and our real interests
Overall half of one’s Ad Settings do not reflect their interests
Respondents were presented all the Ad preferences that Google keeps about them. They were asked to which degree each of these Ad preferences match their actual situation or interest. In many cases respondents understood why they got this Ad Setting, but this doesn’t necessarily mean that they are interested, as the comment of one respondent indicates:
”We bought a piece of furniture from a retailer. We were absolutely unhappy about it and we looked up their website a few times to get in touch. Now this retailer is showing up in my Google Ad Settings.”
Some respondents commented that the nature of their job (creative agency, consulting) required them to look up all sorts of products, services and industries. They thought this might be the reason that they got so many irrelevant Ad Settings. Here I singled out respondents who do not work in agencies and consulting jobs. The average match score is similar however.
Another issue is one Google account used by multiple users. This happens when you share a computer with family members. In that case Ad Settings could be even more off. I singled out 2 users sharing a computer. The results again were similar.
Google seems better at detecting shopping interests than personal details, but some Ad Settings completely miss the mark
The 52 respondents share 7217 Ad Settings amongst them of which 1140 are unique. Let’s take a look at the top 5 best scoring Ad Settings — the ones attributed to the most users. (ordered by occurrence * score)
The best scoring Ad Setting is “news” with a match score of 79%. All the best matches are very general: books, restaurants, news, movies, music, etc.
Now let’s take a look at the worst scoring top 5 Ad Settings. These are the Ad Settings assigned to almost all of the respondents, but with the lowest match score. (ordered by occurrence * (1- score))
Google thought that 48 out of 52 respondents would have an interest in “Greeting cards”. Most of them don’t, resulting in a match score of 16%.
It is noticeable that these poor matches are much more specific than the top matches: greeting cards, computer games, TV game shows, credit cards, celebrities,… It makes sense that you’d have a higher chance to miss the mark for a specific category than for a general category. But this also means that you’ll have more waste when trying to target specific audience interests.
Google does not to merge Ad Settings of the same person
The general perception is that Google pieces together all the digital traces that we leave behind to build a complete profile of us. To my great surprise, I didn’t see this reflected in my Google Ad Settings.
I have 2 Google accounts, one for professional use and one for private use. They are both used on the same devices and mostly they’re both open in my browser. It should be abundantly clear to Google that these 2 accounts are used by the same person.
The personal Ad Settings of my two accounts don’t match.
The Ad Settings for both accounts hardly overlap. In the diagram below you see the overlap between both accounts. Another respondent (right diagram) also uses 2 accounts, but they are not used on the same device. Also in this case, less than 1/3rd of the Ad Settings overlap.
I’ve entertained the idea that Google has a much richer and more accurate profile of us under the hood, but that it shows us just the surface of it. In that case our Ad Settings page would give us a false sense of control.
I checked a number of Google Display Network Ads for both accounts and they are in line with the Ad Settings of both.
We gave up our privacy and this is all we get in return?
Despite the fact that most of us share their search history, browsing history and location with the big ad-tech players, only half of Google’s Ad Settings reflect our personal situation or interests.
And contrary to what could be expected from a company that thrives on personal data, there is really no evidence to think that Google is using a much richer profile of us somewhere under the hood. We would get much more relevant Ads if that were the case. And average click through rates for Google Display Network would be higher than 0,5%.
I’m not saying that Google does not have more data on us. We’re seeing the result of a fundamental flaw in digital advertising: the assumption that click behaviour reflects our attitudes, motivations and purchase intentions
We’re seeing the result of a fundamental flaw in digital advertising: the assumption that click behaviour reflects interests and purchase intentions.
I guess we’re also seeing the limitations of machine learning here. Machine learning is good when dealing with specific contained domains. Digital advertising is a very broad domain, with millions of dimensions needed to match thousands of advertisers and thousands of products with the thousands of signals emitted by internet users.
I’m in the target demographic for BMW. Some of my colleagues drive BMWs. Many of my peers love dogs. And I looked at the funniest video of a dog lying in a bed the other day. So average all of that and Machine Learning will determine that the propensity of me being a dog-loving BMW-driver is high. Only, I’m not.
Don’t get me wrong digital advertising works. It has helped to grow numerous brands and companies. But digital advertising does not offer the envisioned relevance in exchange for all of our data. With ever stricter privacy regulations and restrictions on cookies, we will see even less accurate personalisation in the near future. A sole focus on targeting won’t cut the chase.
WPP CEO Mark Read, addressed this topic in an interview last week. He said that the industry is worrying too much about targeting and too little about creative. He added that relevance is not necessarily the same as personalisation:
“Relevance can come in many ways — personal data just being one part of that. There are many ways to target messages that we’ve learned from analogue media over many years.” — Mark Read
I’m looking forward to more creative digital advertising.
