Next-Gen Business Intelligence

Over the last few years, BI has proved to be a competitive differentiator for

enterprises that relied on BI tools for decision making. BI helps an enterprise

to improve its business performance by providing it the ability to achieve their

mission objectives by making smarter decisions at every functional level of the

business. Though the landscape of BI matured over years, it has come of age from

simple reporting and dash-boarding capabilities to incorporate analytics tools

as well. Analytics tools make a difference for organizations, by giving them

insights into the facts presented by reporting tools and then supporting them in

taking actions based on those facts. With the meteoric increase in data in

organizations, enterprises can now make those data accessible and useful by

tapping into the power of BI technologies. Today, BI systems are emerging in

every aspect of enterprise, ranging from large scale enterprise BI environments

that are deployed to support thousands of people, to smaller departmental BI

systems that cater to the needs specific to that department. The maturity of BI

adoption can be seen in new industry verticals like telecoms, retail,

health-care and BPO sectors.

For an enterprise to successfully implement BI, it becomes imperative to have

proper information management architecture in place. BI tools have to fetch for

information coming from various data stores across departments in an enterprise.

A BI tool sits atop all the data stores as a common layer. However, there always

remains a gap, because BI solution and the information management system, which

incorporates data-warehousing come from different vendors. This gap could be in

terms of compatibility of a BI solution with certain type of underlying

data-store or because of the fact that organizations avoid handling different

vendors and prefer one who can provide a single solution stack. This has

resulted in many joint ventures between BI solution providers and enterprise

solution providers, like MicroStrategy with Sybase, or acquisitions of BI

providers by enterprise solution providers, like BusinessObjects by SAP to

mention a few. This way, an enterprise can have a single solution that would

manage its information and also provide business intelligence solution. Apart

from having a singular stack solution for their BI needs, enterprises also want

BI solution to cover their whole enterprise and provide them with timely

information in an easy-to-interpret way with functional analytics that can help

enterprise make better business decisions. Systems that can provide such

information from cross departments and in faster time limits, are the part of

next gen BI tools. Here are few of the trends that are taking BI to a newer

landscape.

BI on Cloud



Cloud computing is one of the hottest trends in the industry today. Many

solution providers who used to serve on Software-as-a-Service (SaaS) platform

are now exploring Cloud as a platform to offer their services. In

business-economic terms, SaaS is predominantly driven by the wish to deploy IT

resources quickly and flexibly, at a time when organizational structures and

product portfolios are increasingly modified to comply with changing market

conditions. Cloud due to its benefits of auto-scalability and elasticity becomes

the next logical step for many to move into.

Though companies like SAP offer BI solutions (BusinessObject) on SaaS

platform as On-Demand offering, the likes of Google and Amazon have cloud

computing hardware infrastructure already. It's interesting to see how BI

companies would foray into this new domain and use such Cloud platform to

deliver their products. Already there were announcements from companies like

JasperSoft and Talend that they would expand their services on Cloud

infrastructure. Pehtaho has recently released its Cloud based BI product based

on Amazon EC2 platform. However, how enterprises adopt this remains to be seen.

One drawback in a Cloud is that you lose control over the data which is

extremely sensitive and crucial to the decision making for an organization. You

can't risk landing your confidential data in the Cloud. What if the data reaches

your competitor's hands? That depends on how the Cloud environment matures up as

a enabler technology platform for providing On-Demand services.

BI coming out of Silos



For an enterprise, analysis and reporting of data have become an integral part
of its decision making processes and actions. However, each organization has a

number of departments and each department has a BI solution which is specific to

its requirements. For instance, finance department might have its own BI system

residing on the financial information data-store. This makes viable sense for

having department wise BI, which in essence are the packaged BI tools that comes

along with the application. But when it comes to a BI solution that pans through

each of the processes of an enterprise, then it becomes necessary for the BI

solution to access various data-stores of different enterprise applications like

ERP, or SCM and provide cross functional reporting and analysis ability. This

cross departmental reporting and information analysis capability can be achieved

through new tech like multi-source.

Operational BI



BI has always been viewed as a tool to be used by decision makers and

business managers. Given the complexities involved, IT professionals seem to be

the prime users of BI tools. However, this is changing now. BI users now cover a

broad gamut from C-level executives to IT professionals to front-line business

users within an enterprise. Now everyone is allowed to access organizational

information that can help him in his work and generate his own reports. BI when

integrated with operational processes can allow organizations to react to

changing business conditions. For example, while a sales executive makes a sales

pitch to his client, the integrated BI capability can alert him of past deals

with the client and his buying pattern.

The information is delivered in a dashboard like interface which is easier to

interpret. Apart from the pre-generated reports and queries, the user has the

capability to fire ad-hoc queries, and with analytics tools he can do analysis

of what-if scenarios and predictions. The BI user interfaces now provide not

just basic reports, but advanced functionalities of various types also. BI

dashboards and portals are becoming richer and flexible due to implementation of

Rich Internet Application (RIA) tools that incorporate technologies like Flash

and Ajax. This helps deliver performance improvements for an employee. And, due

to dynamic dashboard interfaces, BI usage has now become available to almost

every user in the enterprise, thus making BI more operational in use.

BI breaks free of data warehouse



BI tools fetch information from transactional sources like data stores, data

warehouses or data marts i.e. structured information sources. So for the BI

architecture, data warehouse becomes imperative. This is where from precise

reports can be generated through SQL queries. And in case of unstructured

information, such as, documents, emails, memos, etc, the SQL queries cannot

work, as this is a different architecture.

The new generation of BI solutions combine functionality of both structured

as well as unstructured content to provide information access over wider content

types. This is made possible through data integration layer, where the

unstructured content is transformed and loaded into a format that fits better

into a traditional analytics environment. This is how access to information,

through a BI tool, can be extended to linked information residing in documents,

emails, memos, spreadsheets etc. Typical scenarios where BI tools can access

structured as well as structured contents could be a BPO company that comprises

several applications. The BI system could extract information from various

applications and even from within emails.

New tech in BI



BI tools are becoming more faster in processing complex queries and going

beyond a functional process to access information from across departments.

In-Memory and Multi Source are the two technologies that are behind this

capability.

Multi-Source: A BI system should contain integrated functionalities that

include a database, an ETL technology (extract, transform and load) to move data

into the database, and a BI platform to create reports. A BI solution for a

particular department should allow the user to collect and store data from Excel

or databases and do report designing and conduct analyses against the data. But

for a cross departmental BI to work, the requirement would be accessing many

databases at once. It will require technical resource to aggregate all the data

of the enterprise, BI would need from a myriad of databases into the BI common

data-store, which is the metadata layer. Metadata is the layer where the BI

tools interact with the underlying data layers, i.e. the information from

various data stores is translated into business terms by BI tool that forms the

metadata. It will be a tedious task to move data from operational systems of

various departments into enterprise's data mart. Multi-source rescues you from

the task of migrating various databases. Multi-source allows you to quickly

incorporate various operational databases into the enterprise BI model without

going through the effort of moving the data into the data mart or data

warehouse. Once the new databases are identified in the BI metadata model, their

physical storage location becomes completely transparent to both BI users and

report developers.

SpagoBI incorporates various analytics engines that can be managed by the administrator by selecting Resources > Engine Management from the menu bar. The properties of the engines can be configured by clicking on Details icon.	GeoEngine analytics allow location or map based analytics. Here, by clicking on each state location in the map we can have the corresponding sales information breakup for that state in the other panes.

In Memory: The query performance of a BI system is fundamentally governed and

limited by the query performance of the underlying databases. The BI solution

providers have been developing techniques for SQL optimizations to improve query

performance for all major database technologies. One of these is In-memory query

caching. In-memory offers a new architectural approach for increasing query

performance on a very broad basis. In-memory BI operates like a database, but

has the performance characteristics of a memory-based caching system as well.

For instance, MicroStrategy's In-memory technique takes advantage of the huge

addressable memory space and is now available on 64-bit computers to provide

high performance middle-tier databases that can respond directly to data

requests from reports, dashboards, and OLAP analyses. Since the new middle-tier

databases are stored in computer memory, they avoid disk access delays of

traditional databases. A typical Online Analytical Processing (OLAP) cube

exhibits a performance profile in which execution of query time ranges from

minutes to hours depending on the volume of data and the complexity of the

query. With in-memory caching, the OLAP is supplemented with dynamic caching

technology that saves report instances in memory so that the next user of the

same report can be served the result directly from memory, avoiding database

delays. This creates a performance profile with two response-time distributions

— one for the database queries and one for the cache-based responses and hence

speeding up the BI performance time.

Analytics in BI Tools



Analytics have become an integral part of Business Intelligence suites.

There are many open source BI solution providers, but not all of them have

analytics capabilities. Pehtaho, JasperSoft, SpagoBI are some of the few open

source BI solutions that have analytics as part of the package. SpagoBI is an

open-source BI tool that offers wide variety of analytical tools and in an

intuitive user interface. Let's take a look at SpagoBI's analytics tools.

OLAP engines create complex cubes that gives complete data veiwpoints. In this OLAP inventory report, we can define additional parameters and by drill down feature you can access the breakup of the valus pertaining to a parameter.	KPI monitoring is a key feature of any BI tool. SpagoBI provides the same wherein you can monitor performance of various indicators and can also see the value trend for any particular KPI by clicking on the TimeLine icon.

About SpagoBI



The open source BI solution not only offers just simple reporting

functionality but also complex analytics capability through interactive

dashboards. It satisfies the whole range of typical BI requirements for an

organization in terms of data management and analysis along with administration

and security features. SpagoBI covers the whole range of analytical needs of an

organization with solutions for OLAP analysis, geo-referenced analysis,

data-mining techniques, free inquiry through Query by Example (QbE) module. It

also has intuitive dashboards to monitor the business performance indicators (KPI).

It also supports publication of data, which can be used as a collaborative

process between involved peoples for that report.

SpagoBi is an integration platform of various sets of tools and thus offers

many engines for the same analytical area. This gives the user freedom to choose

any analytical engines for his analytics requirements. All the integrated

engines share the same model that regulate the presentation of the analytical

document, and hence the interface remains same for all engines. It also has

integrated support for important processes of extraction, transformation and

loading (ETL) of data into the repository.

Getting started



You can find SpagoBI along with few other open source BI tools (check table)

in the DVD . SpagoBI comes as a ready to install zipped package along with

pre-configured Apache Tomcat 6.0 application server. All that you require is

that the machine should have a JDK installed and the JAVA_HOME environment

variable is defined. You just need to extract the contents of the zipped file.

From the bin folder of the base installation, you can start the Tomcat server

through startup.bat. Before starting off the server, you also have to define the

roles for users and the admin of the SpagoBI by defining them in tomcat-users.xml

file. After starting the Tomcat server, navigate on your browser to http://localho

st:8080/Spa goBI. Login as administrator and configure the analytic engines that

you want to use for generating reports and also define the databases that will

be used as data source by the analytics engines. While as a user, you can access

the KPI models and reports that are configured by the administrator. Based on

those reports and models, the user can modify, drill down and even do data

mining for accessing further information from them.

SpagoBI comes with a demo database, and we used the same for some report

generation and analytics as follows: