For the past 20 years, conventional data warehouse projects — promising one version of the truth — have been costly, complex and stuck behind the firewall. But various advances and market forces have pushed the warehouse across a DMZ and it now faces outwards instead of inwards.
Thinking back to the 1990s, the data warehouse was in vogue. Large projects abounded, promising one version of the truth from which analysts could draw conclusions that would inform decisions made by the company’s analysts and executives.
These systems were relatively large at the time, storing what seemed like vast troves of data. The data came from large, in-house systems such as ERP, HR, CRM et al. Systems were designed and built to process batches of in-house data that was then extracted at great expense by ETL (extraction, transform and load) programs.
Unfortunately, ETL systems labored sluggishly to keep the version of the truth updated eventually, if not currently. Visualization tools provided insight. The term “data mining” was coined. And all this lived behind the firewall at great expense. The industry struggled largely because these projects were expensive, complex, took years and often failed. If they didn’t fail, the benefits were hard to quantify and, often, not fully realized.
Now, 20 years later, a new and essential vision has emerged that’s viable and delivers value in real-time. While value accrues to the enterprise from this new model, it’s the customers outside the company who realize the immediate benefits — which is good because customer demands and expectations have dramatically changed since the 1990s. Today, customers engage online, in real-time, at any time, from anywhere. If an enterprise can’t keep up, customers leave them in the dust and move on to a competitor. Legacy platforms no longer work in this world. There is just not enough time.
The data warehouse has been flipped — what was once inward-focused is now outward-focused. Technology advances have made it possible, and customer expectations have made it an urgent necessity. Customers, more than ever, are in the driver’s seat.
In order to keep pace in today’s hyper-competitive market, enterprises need modern data warehouses that capture data in real-time from outside the building. The data must be stored in real-time repositories where it is combed, combined with data from existing systems and accessed for conversion into knowledge. Once the data is converted to knowledge, it’s then pushed outwards and kept on the edge where it is applied to real-time machine learning-based decisioning engines that algorithmically decide what is the next best action to take in an instant.
Wow, that’s turning the warehouse inside out — business decisions made in an instant by repeatable algorithms which are enhanced and made relevant in real-time. Imagine that! The focus is out, not in. Decisions are immediate and occur millions of times per day.
Recommendation engines, identity management, fraud prevention, digital wallets, all act in real-time by outwardly facing the world. They ingest astounding amounts of data from the outside and pull data as needed from legacy systems in place today. The beneficiaries served are online shoppers, news readers, bill payers, game players, smartphone users and other customers. And companies deploying these new modern data warehouses enjoy a competitive edge, capture and keep more customers and more share of wallet.
The focus now is at the point of engagement. A point where deep learning and prescriptive analytics are combined with real-time, in the moment behavior, to feed algorithmic decisioning engines that prevent fraud, validate identities, deliver recommendations, make online shopping a pleasure, delight user constituencies and improve both the top and bottom line. In short, these decisioning systems “decide” what is the next best action.
The data warehouse has been turned inside-out. The ROI is real, the technology exists, and the future is bright.
Feature image via Pixabay.
