In the Wake of Big Data: The Hype Behind Gartner’s Hype Cycle

When Gartner released their “Hype Cycle for Digital Marketing and Advertising 2019,” I couldn’t help but think to myself, This is the world left in the wake of big data.

Allow me to explain.

The Hype behind the Hype

Over a decade after “Big Data” became the next big thing, the imperative to collect every piece of information— whether structured or unstructured— remains ingrained in the fabric of enterprise businesses. While the hype has died down on big data, companies are still dealing with its effects— arguably, the hottest technologies of today are in response to the complications introduced by the big data mentality of collect everything.

Though the hype for big data has been put out to pasture for years now, the central tenets of big data remain with us: volume of data, variety of sources, and velocity. As InfoWorld wrote in a 2014 reflection on the decline of big data, “When what is now commonly called ‘big data’ becomes no more special than other data, when the technologies used to store and process this data become commonplace, then will the hype around big data start [sic] decline, and fall from grace.”

Here we are, five years on from that assessment, and the promise of big data has become an indelible part of the enterprise information landscape. Regardless of industry, data plays a critical role in every business’ strategic goals and, when not taken seriously enough, represents a source of immense risk (just ask a company like British Airways, which was fined $230 million for violating the GDPR). But the challenges of realizing that promise of big data remains.

When put into that context, Gartner’s “Hype Cycle for Digital Marketing and Advertising 2019,” in which they’ve identified four technologies that will “transform how marketers run their technology ecosystems,” tells a fascinating tale of where the promise of Big Data was unable to be fulfilled during the zenith of its own atmospheric rise.

What Do You Do With a Problem Like Big Data?

When Hadoop came to the market, it promised a way to cheaply store and retrieve tons of unstructured data, organizing it for simple retrieval to run analytics. Further evolutions like Cassandra refined the rough edges to increase the velocity to achieve insights from the data. 

Now that companies were trained to think big about data, they had tons and tons of data sitting in on-site or cloud-based servers. Then they ran into a slew of challenges, all stemming from the original sin— er, promise— of big data: the expectation of meaningful insights from all that information.

As companies discovered, there was a lot of work involved and the insights often came too late in a world beholden to meet real-time needs. For this reason, even in 2019, analytics, AI and big data projects have an astonishing 85% failure rate.

The four technologies Gartner called out as transformative in their most recent Hype Cycle all circle around one common theme: making data useful. The technologies include:

1. Customer Data Platforms
2. AI for Marketing
3. Blockchain for Advertising
4. Real-time Marketing

In their own ways, each one of these take a segment of the original promise of big data and turn it into a fully-realized part of the martech stack. The hype may be the same— turning all of the data we collect into valuable, meaningful information. However, in the intervening decades between the heyday of big data and today’s world, new guiding principles have emerged beyond simply velocity, volume, and veracity.

The ability to collect and control your customer data. The ability to act intelligently based on everything you know about a customer. The ability to (re)instill trust between the customer and the company.

Big data set the table for these principles, and now we’re reaching the moment for them to be served up to marketers, IT professionals, and customer experience teams as achievable goals.

While each of these technologies takes up the task left behind by big data, Customer Data Platforms represent the most “transformational” of the hyped-up tech.

Heir Apparent?

As Gartner notes in their Hype Cycle report, “Marketer’s expectations for CDPs remain at an all-time high, but the reported use of the technology differs from the advertised capabilities. Half of enterprise marketers surveyed by Gartner who have deployed a CDP say its their CRM system…” While Gartner argues there is “redundancy” with “other, more mature marketing technologies,” limiting CDPs to a Marketing and Advertising discussion skates over the technology as heir apparent to the promise of big data. Of course marketers are going to confuse the CDP with a CRM when they don’t think of the CDP as the company-wide foundation for everything to do with the customer.

There’s little technological lineage connecting CDP vendors and big data’s top players, but the promises of the latter live on in the former. There are CDPs out there like Tealium AudienceStream™ that focus on collecting and standardizing data to provide the velocity, volume, and variety, along with the new principles of governance and trust in real-time.

Gartner’s hype cycle clearly points to companies’ need to better control and act on their data in a future where data is constrained by consumer expectations’ and regulations dictating strict privacy standards. However, Gartner’s positioning of CDPs on the “Digital Marketing and Advertising” Hype Cycle continues the broader trend of limiting the strategic potential of this technology across the enterprise. In doing so, it only perpetuates the continual segmentation of data sources and intelligence that make a true single view of the customer impossible.

Even though big data dreamed too big for its time, the thought of having a single source for all of your data lives on today in the CDP. With a data-first CDP like Tealium AudienceStream you can handle enterprise levels of data volume and variety of sources through our 1,200+ integrations while enabling delivery channels with those real-time insights. 

But we’ve got to start admitting that, behind the hype of new technologies, the underlying challenge remains the same: managing data is a company-wide initiative too often limited through lens of the tools and departments. Big data tried to change that. Years later, companies are now starting to take serious the need for an independent data layer built to inspire big, structural change.

If we take only one lesson from the hype of big data, it should be that we continue to think strategically about data on the macro level, at the level of the customer experience, and not simply through the channel and departmental mindsets that can dominate our day-to-day workload.