New Report Finds That X May Be Inflating its Ad Performance Results

There’s an anomaly in X’s web traffic data that’s been hurting my head in recent months.

As per data from SEMRush, which measures referral traffic from Google, X saw a very sudden, and massive increase in web referrals in April last year.

As you can see in this chart, web referrals to www.twitter.com went from around a billion per day on average up to April 5th,2023, to more than 2 billion on April 7th. And since then it’s increased even further, and is now up to an average of around 3 billion referrals per day.

Which is a huge increase, and there’s nothing that I can find to suggest why Twitter, or X, would have suddenly seen such a big jump in web referrals at this time.

Maybe, I assumed, it could be due to X limiting access to its API (which went into effect at the end of March), which could have then prompted a big uptick in researchers accessing the platform in different ways. Maybe it’s related to the rate limits that X implemented to stop AI companies from scraping its platform, though those only came into effect in July.

Worth noting, too, that www.x.com has also seen a similar traffic spike very recently, going from 390k referrals on February 14th, 2024, to 1.35m a day later.

The Super Bowl was on the 12th, so that’s not it, and it’s also a day after Valentine’s Day. And the change is huge here, a 3.4x multiple.

So why is this happening? What’s changing within X’s system, or Google’s, to drive such big shifts in its referral numbers?

Maybe, in the case of x.com at least, the spike is a result of the company’s ongoing work to switch over to the new domain.

Then today, I saw this report from Mashable, which, quoting data from ad measurement provider CHEQ, found that of all of the referral traffic driven via X ads during the weekend of the Super Bowl, almost 76% of it came from sources that CHEQ identified as likely fake.

As explained by Mashable:

“CHEQ monitors bots and fake users across the internet in order to minimize online ad fraud for its clients. The company accomplishes this by tracking how visitors from different sources, such as X, interact with a client’s page after they click one of their links. The company can also tell when a bot is passing itself off as a real user, such as when a fraudulent user is faking what type of operating system they are using to view a website.”

Now, while there is definitive method here, the findings are also anecdotal to some degree, as CHEQ only has access to this type of insight for its client list. But for comparison, CHEQ also reported that the exact same measurement report for last year’s Super Bowl found that only 2.8% of the traffic from X was likely fake, out of 159,000 visits.

For clarity, this year, CHEQ’s analysis looked at 144,000 visits to its clients’ sites that came from X during Super Bowl weekend,

So could X be inflating its numbers with fakes and spam sources, in order to boost its own reach figures, as a means to potentially lure more ad spend?

It seems unlikely, but then again, X owner Elon Musk did make fakes on the platform a key focus, as part of his efforts to get out of buying the platform, which was ultimately rejected by the courts.

Back in July 2022, as Musk sought to renege on his $44 billion offer for the app, Musk argued that the platform wasn’t actually worth that price due to the high amount of bot profiles, which Twitter had continually included in its active user figures.

Twitter had long held that the number of fake profiles in the app didn’t exceed 5% of its total mDAU count, based on its own sampling, but Musk claimed that it was actually much, much more, with his own team’s analysis finding that around 33% of Twitter’s active profiles were likely fake. Musk eventually settled on it being a more modest 20%, while noting that it was likely much higher.

So Musk himself claimed that at least 20% of X’s active usage was via bots and spam. Yet since taking over the platform, Musk has made no mention of this, instead quoting “record high” usage figures, while also claiming to have defeated bots in the app.

And seemingly, both of these stats can’t be true.

For example, if bots accounted for 20% of Twitter’s mDAU count at the time of the purchase, as Elon claimed, that would mean that X would have had to eradicate around 50.6 million bot accounts to clean it up and start afresh, without bots included in its active user figures. X’s user count has since climbed from 238 million daily actives, to more than 250 million today, a count that Musk initially reported in November 2022, just a month after he took over at the app. Which would mean that, accounting for bot removals, X would have had to have added around 62 million more users since Musk took over.

The platform has never added more than 30 million more actives over a 12-month period. So either X has seen astronomical growth as a result of Elon taking over, or Elon’s bot estimates were wrong, and were likely juiced as part of a bid to wriggle out of the X deal.

Or X still has a heap of bots, around 20% of its user base, and Musk and Co. have simply opted to do nothing about it.

But then again, X does have ad verification partners who are able to provide third-party analysis on this front, and would likely be able to confirm whether X ads are being seen by real people or not. And if X is actually bulking up its numbers with fake traffic, those verification partners would detect this.

Right?

Well, it depends on what, spefically, each of its verification partners is measuring.

• Integral Ad Science (IAS) offers brand safety and suitability measurement on X, though it has also offered viewability and invalid traffic confirmation in the past. It’s not clear if IAS still provides traffic validity assurance on X (we’ve inquired with IAS as to its thoughts on this new data).
• DoubleVerify (DV) also offers brand safety measurement for X, as well as fraud and viewability measurement across both display and video campaigns (we’ve also asked DV for its thoughts on this new report).
• As of June last year, X was also meeting with a range of additional measurement providers, including Zefr and Unitary, to provide more control over ad placement, but there was no mention of audience authentication.

So, for the most part, X’s ad verification partners are focused on ad placement, not on audience verification, and they only do this for commissioning partners, not all X traffic. So it is possible that they wouldn’t be able to provide definitive data to dispel this concern either.

Does that mean that X is definitely juicing its numbers, in an effort to make its performance seem better than it is?

No, it’s doesn’t. Again, all of these data points could have some alternative explanation, and because CHEQ only has access to a selection of X traffic data, it’s not clear if this is happening across the board.

The real insight will come from your own ad performance data, and what you’re seeing. If you’re noting that you’re X insights include a lot of clicks and engagement, but you’re not seeing the same reflected in your own analytics and performance data, then it could be that something’s not right, and that X is inflating those figures.

Essentially, as with all things digital marketing, individual results will vary, and if you’re still seeing strong ad performance on X, it’s probably worth sticking with it.

But it could be worth keeping an eye on your X performance metrics moving forward, in case of anomalies.