Today's enterprise data warehouses (EDW) — high-end analytical computing
platforms — continue to provide powerful strategic analysis. However, the newest
wave in database-driven insight is operational data warehousing. This involves
new capabilities that enable operational decision making in real time — as well
as strategic data mining. The use of these real-time centralized data
repositories has escalated and expanded. The term most commonly used for this
new approach is Pervasive Business Intelligence (PBI).
This is not a future vision but a present reality. By implementing PBI, the
best businesses are taking knowledge management to new levels. The day of
passive knowledge management is quickly ending. Pervasive BI rapidly evolves
knowledge management into competency optimization for employees — who can truly
interact with customers and the business with a panoramic view, compressed into
actionable tactical intelligence for tangible economic value.
Pervasive knowledge management
In simple terms, pervasive BI delivers data warehouse insights to everyone
in the enterprise, not just an elite group of technologists, analysts or
knowledge workers. This is truly a revolution in the use of knowledge. You might
even call it pervasive knowledge management. The goal is to make thousands of
daily tactical decisions better by using facts instead of intuition alone.
Imagine every employee making 40-50 small decisions a day. Next, imagine adding
the use of Pervasive BI, so that 20 of those decisions are supported by facts
instead of guesswork. Generally, the line-of-business managers understand
pervasive BI instantly and wonder why they haven't had it all along. Some
knowledge management (KM) practitioners don't think of technology as a best
practice arena, however, we would propose that this is indeed an emerging best
practice. Look at it this way: you distill facts, insights, and information then
systematically deliver it to the thousands who need to make smarter decisions.
Maybe think of this 'knowledge distribution' — as the purpose of the larger
server platform. At times, summary data is all that is required.
However, the use of more detailed, actionable information in real time can
certainly provide much more valuable knowledge. Agreed?
Unfortunately, computer technologies were once limited, forming a barrier
between the front line user and the EDW data until around 2001. It existed
because front line users need 'fresh data' in addition to the historical
information normally found in the EDW. While historical data is loaded into the
EDW nightly or weekly, front line employees need today's data as well. For
example, a call center representative (CCR) needs to know a lot about the
consumer calling in: residence, prior calls, profitability score, and the next
best offer to suggest.
All this can be calculated nightly, ready to go each morning. But the CCR
also needs to know what calls the consumer made this morning and what the
consumer did on the company website today. Fresh and historical data need to be
displayed while the consumer is still on the phone. Getting that data from the
production system, into the data warehouse, and back to the front lines is
sometimes called 'real time BI;. Until 2001-2002, the software and computers
could not do this.
As it often occurs, some enterprises weren't daunted by the seeming
limitations of technology. The business-IT visionaries wanted the benefits of
pervasive BI immediately, and they became creative. So in 2001-2002, numerous
enterprises found ways to speed up getting data into and out of the EDW. These
visionaries were able to connect their point of sale machines, airline gate
agents, web site, call center, and dozens of other front line users straight
into the EDW. As IT organizations charged ahead, the software vendors saw
opportunities to improve their products in the same direction. In a symbiotic
relationship, IT experts and IT vendors collaborated to make the technologies
work in real time.
Elements in pervasive BI
How did they do it? What were the barriers that were overcome? First, three
technology subsystems had to evolve. Second, IT organizations needed new designs
and processes. The three technology subsystems — Data Integration Services,
Decision Repositories, and Decision Services — are part of an information supply
chain. Data originates in raw form, gets transformed and cleansed. Next, it's
repackaged, stored and repackaged, then finally analyzed and distributed.
Data Integration Services is a point of convergence, a hub, taking all kinds
of data from many sources and production applications, and preparing it for the
EDW. To meet the 'service level” goal of 'under one hour' or 'under five
minutes,' delivery of the raw data needs to be “guaranteed delivery” to the data
integration hub so nothing is lost and everything is on time. Data integration
services also must clean up the data quickly or else the inaccurate data will
lead to bad decisions. It must remove duplicates like name and address, and swap
out 'codes' for more understandable values. All this must occur as multiple
streams of data flow into the data integration server continuously or in spurts.
The Decision Repositories (data warehouses and marts) need to load the data
and serve it up as fast as possible. This is difficult because data loading can
consume the entire EDW server's capacity, leaving no access for the front-line
and back office employees. Performance for the front line and back office users
is terrible during loading. But around 2002, Teradata began offering two
components to solve this: a real time data loading utility and mixed workload
management tools. Mixed workload management means prioritizing work inside the
data warehouse according to business rules. This enables an enterprise to give
the call center workers top priority, reports medium priority, and data loading
low priority. The result is data loading tasks perform well while reports run
fast, and the call center runs fastest.
Decision Services grabs the EDW data, reformats it, analyzes it, and delivers
it to the front line user. Since the front line user doesn't have time to read
reports, the requested facts have to be short and specific to the business task.
This means delivering analytic information to the entire knowledge network: BI
dashboards, employee portals, mobile devices, and modern 'composite
applications. 'Since nearly 50% of all IT development projects are now composite
applications, the EDW and Decision Services roles must be fit into those
developer tools and applications.
The result is the ability to deliver analytic insights in any business
process: Pervasive BI.
The IT organization, led by the CIO, also has some critical success
requirements. First, there must be a 'service level agreement' between the IT
group and the front line users for performance and availability. Negotiating the
agreement wrings out all the cost issues, true business needs, and project
objectives — resulting in information access priorities. The second success
factor is mission critical availability. The entire information supply chain
must be 'always on, always available.' Since many business processes run any
time of the day, the pervasive BI subsystems must as well. But most companies
have not made the end-to-end information supply chain mission critical. This is
the area of the biggest risk. The good news is most IT teams know how to do it
with their mainframes and UNIX servers. They simply need to apply those
principles to the pervasive BI infrastructure.
With the emergence of Pervasive BI, organizations can at last activate the
terabytes of data sitting passively in massive repositories. They can convert
knowledge into economic value at cyber-speed — in seconds, not hours. They can
create limitless adaptability in their businesses and create competitive
advantage that is truly as sustainable as the capabilities of their centralized
database. Interested in best practice examples? There are examples at many
leading companies across industries. If the readers and editors want further
information on Pervasive BI — and its contributions to the field of knowledge
management, we can report on those instances in the hope that the field of KM
can at last welcome data warehousing technology into its sacred provinces with
open arms. Again, this is about data being distilled into knowledge and
systematically distributed to thousands of knowledge workers.
Pervasive BI is trending upwards, having moved from the early adopters into
the mainstream. The visionaries have blazed a path but there is still time to be
a leader in your market using pervasive BI. You need a vision, a clear strategy,
and a capabilities roadmap, in order to more effectively and economically
differentiate your business. Just don't wait until this trend is, well,
pervasive.
Ashok Ekbote,Country Manager, Teradata India