Remember the cable, telephone and internet combination offers that used to end up in our mailboxes? These offers were highly optimized for conversion, and the type of offer and the monthly price could vary significantly between two adjacent houses or even between apartments in the same building.
I know this because I used to be a data engineer building extract-transform-load (ETL) data pipelines for this type of offer optimization. Part of my work involved extracting encrypted data feeds, deleting rows or columns of missing data, and mapping the fields to our internal data models. Our statistics team then used the clean, updated data to model the best offering for each household.
That was almost a decade ago. If you take that process and run it today on steroids for 100x larger data sets, you get to the scale medium and large organizations are dealing with today.
Every step of the data analysis process is ripe for disruption.
For example, a single video conference call can generate logs that require hundreds of storage tables. Cloud has fundamentally changed the way business is done because of the unlimited storage and scalable compute resources you can get at an affordable price.
Simply put, this is the difference between ancient and modern stacks:
Image Credits: Ashish Kakran, Thomvest Ventures
Why do data leaders care about the modern data stack today?
Self-service analytics
Citizen developers want real-time access to critical business dashboards. They want to automatically update dashboards built on top of their operational and customer data.
For example, the product team can use real-time data on product usage and customer renewal for decision making. Cloud really makes data accessible to everyone, but there is a need for self-service analytics compared to legacy, static, on-demand reports and dashboards.
Contents