Marc Andrews Observations On Demand

Why BI Deployments Won't Be Slowed Down by the Recession

With everyone finally admitting that we are in a recession, now the speculation has turned to where spending will be cut. Just as important though, is where it should NOT stop.

Analyst Mike Schiff, from Current Analysis, provides some great insight into why BI deployments will continue to be critical for organizations hoping to come out of this recession ahead.  Specifically, in his report on Business Intelligence: How Will the Recession Affect Its Deployment? he claims that, "While in times of economic growth companies may seek to upgrade their operational systems in order to accommodate expected increases in customer demand, during an economic slowdown these efforts may be delayed in order to fund analytic, BI-based projects for the purpose of identifying new revenue opportunities, retaining existing customers, and finding areas where expenses can be reduced."

He also cites government usage of BI technologies on areas like national defense and homeland security as areas that are not likely to see cutbacks...although I'm not as sure about this with coming change in administration.

However, the fact is, business intelligence tools are critical for organizations to make better, faster business decisions, with a greater level of confidence. Whether it is about which customers to focus on, identifying new revenue opportunities that require minimal additional investment, where to reduce costs with the least impact on revenue, or how to better identify and reduce risk.

You can bet that executives want to be more confident in their business decisions than ever before. And that will lead them to want more information, and accurate information, upon which to base those decisions. There is also increased scrutiny on regularly monitoring business performance so that executives can stay ahead of the curve. This will lead to increased demand for the Performance Management aspect of business intelligence.

In this environment, companies will need to generate new intelligence to survive, let alone create competitive advantages. And Business Intelligence and Performance Management tools are one of the best ways to accomplish this.

Analyst Predictions for Information Needs in 2009

The end of this past year left more for speculation than probably any other year in most of our memories. We're experiencing the most challenging financial crisis we've seen since the Great Depression, plummeting retail sales that will likely change the landscape of the retail industry as much as the banking world has already been changed, continued exponential increases in health care, distrust in our banks and financial markets which has only been compounded by Madoff and what is being described as the largest Ponzi scheme ever, and increased turmoil in the Middle East with terrorism reaching Mumbai and renewed fighting over Gaza.

Of course, this has also led to increased uncertainty. The Dow seems to go up and down by hundreds of points on a daily basis with no clear rhyme or reason. Gas prices continue to fluctuate back and forth. And consumer spending...and investing...is seeming unpredictable. Maybe that's why so many analysts are predicting that companies will need to increase their investments in information in 2009.

IDC, in their report on Worldwide Information Access, Analysis, and Management Software 2009 Top 10 Predictions, talks about the importance of solutions that can "unify access to multiple types and sources of information" and "put better information access and analysis capabilities in the hands of more users." IDC recognizes that IT budgets will be impacted, but suggests that "the economic turmoil will also increase the need for insight into operations, finance, and sales processes." Some of the business drivers they cite include "compliance requirements, the need to compete effectively while keeping down costs, the enterprise information glut, and the recognition that unifying access and management of all types of information improves business processes, increases knowledge worker productivity, and decreases the risk of not understanding the real status of an enterprise because of an imperfect view of its internal information."

Of the top 10, the three I thought were most relevant were:

1. Need for tools to increase the performance and competitiveness of organizations
2. Corporate events (e.g. M&A, executive turnover, staff reductions) will trigger need for investment in capabilities to automate "knowledge" processes with reduced staff
3. Increased need to manage ALL types of information spurred by stricter regulatory compliance and needs for better, more intelligent business monitoring and decision making

Gartner Can't Say Enough About the Need for Enterprise Information Management and Business Intelligence
Gartner has also been jumping on this bandwagon. In fact, in the beginning of December, Gartner release five separate reports, all about this topic.  They started off by introducing a new EIM Maturity Model.  In this report, Gartner claims that organizations cannot implement enterprise information management (EIM) as a single project, but must implement it as a coordinated program that evolves over time. They introduce an EIM maturity model to help organizations identify what stage of maturity they have reached and what actions to take to reach the next level. IBM had actually put together a maturity model almost a year ago, based on experience from several hundred customers that are trying to establish their own information agenda. There is even an information agenda checklist that companies can fill out and submit to get help assessing their own information agenda needs.

Gartner also issued a report on Enterprise Information Management Revisited: Foundations, Progress and Futures, in which they suggest focusing on information governance as a first project for those that are new to EIM, and stress the importance of "defining and socializing" the relationship of enterprise information management to the business and the business role and the benefits that they can expect, as there is still a great deal of misunderstanding as to what "enterprise information management" really means and how it can provide business value.

In Predicts 2009: Enterprise Information Management Will Prove Itself, Gartner makes predictions on the value of EIM and claims that enterprises that adopt EIM can outperform rivals in areas such as operational efficiency, product development and customer service. And in How to Really Do the Five EIM-Related Things Everyone Else Just Thinks They're Doing, Gartner highlights how enterprise information management can yield measurable business benefits, such as increased efficiency of critical cross-company processes and greater information transparency. In fact, IBM's Information Agenda initiative is all about providing guidance on how to realize these benefits and get results you can measure.

Finally, in Predicts 2009: Business Intelligence and Performance Management Will Deliver Greater Business Value, Gartner claims that the current economic crisis shows the importance of trust and transparency in the information that organizations use to run their business. They suggest that companies need to integrate the analytical insights derived from this information into the decision-making processes throughout the organization.

So, it seems that the overwhelming message here is that 1) we're in for some challenging times ahead, and 2) companies need better access to better information to make better business decisions so they can improve their business performance and create a competitive advantage in the market.

Banking's Increased Need for BI in Uncertain Times

There have been many arguments about the increased need for business intelligence in challenging economic environments.  As expenses get increasingly scrutinized and businesses face budget cuts, the decisions they make become all the more important.  And organizations need to make sure they have the right information and tools to help them make better decisions, faster.

And now banks are finding themselves at the center of this magnifying glass, faced with increased pressure to provide greater visibility into risk, and understand their true potential performance, adjusted for that risk.  Unfortunately, most banks do not trust their current view on this, which is why they turned from lending too freely, to not lending at all.

Business intelligence tools, along with access to accurate, trusted information, can help banks get back on the road to profitable growth.  Mike Schiff wrote a great piece on Financial Markets' New Demands and BI and identifies some specific areas where BI usage is likely to come into play.

Top 100 Focused on Delivering Intelligence to More Employees

InformationWeek just published their 2008 InformationWeek 500 report, revealing IT strategies from the 500 best and brightest companies in North America.  There are some interesting findings in this report, including an analysis of how the "Top 100" compare the rest of the companies in the study.

One of the most interesting is that the Top 100 are less focused on lowering costs and increasing efficiency than the rest, but are instead investing more in initiatives that have the potential to increase revenue and build business by finding new ways to engage their customers through technology and delivering more timely business intelligence to employees.  This is pretty much in line with my comments about the importance of leveraging information in today's unique economic environment.  Another interesting tidbit is that when asked what steps these organizations are taking to optimize the efficiency of their business processes, 89% said they are improving data integration between systems or departments (80% of the rest of the companies were doing the same).

It seems like these investments are driving productivity gains too.  Deploying business intelligence tools and new types of collaboration software were identified as the two most effective steps managers have made in the past 12 months to raise company productivity.  In fact, 34% of the 500 companies surveyed said that they are disseminating better business intelligence to more employees faster - a significant jump up from 18% last year.

There is also an intriguing view into the disparity of BI usage across different industries.  90% of retail companies claim wide deployment of BI tools...100% of general merchandising retailers!  Other high users of BI tools were companies in hospitality/travel, logistics/transportation, distribution and consumer goods.  What surprised me was that banking/financial services organizations were towards the bottom of the list!  Could this be a reason for the financial crisis we're facing now?

Overall, it seems that the leading organizations have realized that the key to competing in today's market is to make your employees smarter by giving them more intelligence.

Leveraging Information in Today's Challenging Economic Environment

So, given all of the challenges we are seeing companies face in today's economic environment, is now the right time to be making technology investments?  In a recent webcast, John Haggerty, from AMR Research, spoke about the challenges companies are facing in today's unique economic environment and what they are doing to address those.  He also specifically talks about "why investing in business intelligence and performance management makes sense...especially now."

The fact is, especially in challenging times, it becomes increasingly important for organizations to make better business decisions.  When times are good and companies are going through high growth, there is a lot more room for error.  However, when some companies are struggling just to survive, it is critical that you make the right business decisions across the enterprise - from strategic decisions around where to take the business, to everyday decisions, such as which loans to approve, how to pay out claims, and how to best respond to customer inquiries.  You cannot afford to make the wrong bets, and you cannot afford to alienate customers.

And the key to making better business decisions is to have the right information.  Have all the relevant information.  Have information that is accurate and can be trusted.  Have information available when and where it's needed.  This is the reason why it is important for companies to make the investments required to establish information as a strategic asset.  And the way to accomplish this is to begin creating an Information Agenda for your organization and focus on how you can better use information to address your key business challenges.

Focusing on Business Optimization

Business optimization seems to be one of the latest buzz phrases out there.  But what does this really mean?  And how is this different from what companies have been doing in the past?

Well, for the past two decades, companies have primarily focused their IT investments on automating their business processes, with the objectives of driving faster processing and reduced costs.  This has been driven by an "application" agenda to implement ERP and Financial applications, supply chain management solutions, and call center and CRM applications.  However, these types of investments are no longer creating a sustainable competitive advantage for organizations.

As a result, over the past few years, we've seen new IT initiatives increasingly focused on optimizing business to drive a more sustainable competitive advantage.  Think about it like those old BSF advertisements - we don't make a lot of the things you use, we make a lot of the things you use BETTER!  Business optimization is about making your business processes, customer interactions and everyday operating decisions BETTER. This means moving from just leveraging ERP and financial applications to providing increased financial risk insight for better business decisions...moving from just managing your supply chain to enabling more dynamic demand planning...and moving from just managing your call center and customer relationships to providing increased insights to improve customer service and drive greater profitability from your customers.

These new initiatives are all dependent on information, and having an information agenda in place to ensure that your organization is leveraging all the information at its disposal to create a competitive advantage in the market.  So ultimately, information becomes the key to optimizing your business.

Why all the BI acquisitions? What is required for BI?

What does it take to implement a business intelligence solution?  BI tools alone do not do the trick.  While there are a few scenarios where it makes sense to just point a BI tool at a transactional database, most situations require that the information be extracted into a separate data store for reporting and analysis to prevent impacting the operational applications.  Thus, most BI solutions require implementation of not only the BI tools, but a data mart or warehouse to store and manage all of the information for reporting and analysis.

In most scenarios, reporting and analysis requires a different view of the information than what has been delivered in the transactional applications, requiring different meta data schemas and data models.  And unfortunately, no one has reached the nirvana state where all of their information is coming from one place. As a result, most BI solutions also require more advanced information integration capabilities, including the ability to aggregate data from multiple sources, cleanse the data for improved quality and reliability, and transform the data in preparation for loading into data marts or warehouses.

And to really ensure you can trust the reports and analysis being generated, and know that those 3 sales are all really for the same customer, or the same product, it is becoming increasingly important to manage master data across your operational and analytic systems.

So, BI is not just about the BI tools (e.g. query, reporting, analysis, dashboards, scorecards, etc.).  It also requires data warehousing capabilities, information integration and master data management.  A complete platform to establish accurate, trusted information AND the tools to report on and analyze that information.  And this is why we are seeing a lot of acquisitions and consolidation in the BI space.  Both the vendors and the customers are realizing that they need more than just one piece of the puzzle to deliver business intelligence.

And if you want to really complete the puzzle, then you will need the underlying hardware to run all of these components (ideally, hardware that is optimized for the different components), expertise to help you establish the appropriate strategy and implement the solution successfully, and if possible, accelerators for your particular industry and the business domains of importance to you.

This is what is required to implement a BI solution.

The Convergence of BI and BPM

The convergence of BI and BPM is actually not new.  Forrester wrote about it 2 years ago and talked about how BPM products needed to integrate with BI "to support decision-intensive business processes like order management, credit risk management, and sales opportunity management," but that BPM vendors were not currently thinking about BI in this context.  Well, the market seems to be finally catching on.  Forrester is working on a new report on this topic, and there was just a great article posted on TDWI on The Myths of BPM, where they go as far as stating that "you can’t implement BPM or scorecards without a solid business intelligence and data warehousing infrastructure."

I recently participated in an ebizq podcast on BI in BPM and one of my colleagues brought up a great question about how IBM is delivering products to enable this convergence, and just as importantly, whether or not we had any customers that are really doing this.

This is actually a big focus area for us, and a major part of our new Dynamic Warehousing initiative.  Our belief is that the next generation of BI is about delivering information and business insights directly into operational applications as part of real-time business processes and activities.  This will enable a broader set of people across the organization to start leveraging BI, enable organizations to use BI for improving everyday business decisions, and drive up the value of their existing investments in related information infrastructure.

We are excited about this trend because we believe that our warehouse offering, DB2 Warehouse, is better suited to enable dynamic warehousing capabilities and address the demands of this next generation of business intelligence.  In addition, it drives requirements for a broader set of information management capabilities, which IBM is best positioned to deliver through our broader information management stack, including Information Server for information integration, WebSphere Customer Center for master data management and our industry data models to apply industry specific business perspectives.  Of course, with the need to deliver information and analytics directly into operational processes, and use that information as part of the decision making process, this will also drive greater demand for process management capabilities, which IBM can provide with WebSphere Process Server for transactional processing and IBM FileNet P8 BPM for content related processes, along with related BPM tools, such BAM and rules.

And in response to the question about real customer examples, I'm happy to say that we have several already.  One is Blue Cross Blue Shield of Tennessee, a large health care insurance company in the US.  They have moved from just reporting on customer issues to delivering insights on prior customer problems directly into their call center application so that a customer service representative can better understand and resolve the customer issues.  The Real Time Crime Center at the New York City Police Department (NYPD) is another great example, and there is a great video on this success story.  They are delivering relevant information to police detectives while they are being dispatched to the scene of a crime so that they already have a list of leads, potential suspects and their possible whereabouts before they even get there.  A complete case study is also available.  Kookmin Bank, in Korea, is an interesting example of a company that justified their initial investment based on the need to address Basel II requirements, but decided to analyze customer risk as part of its credit application process, as opposed to afterwards.  As a result, they were able to reduce credit processing errors by over 30% and improve the overall credit quality of their loan portfolio to better compete in their lending business.

So as you can see, the convergence of BI and BPM has finally arrived.  IBM is helping enable this convergence through Information On Demand initiatives such as Dynamic Warehousing.  And this is already proving to deliver real business value and allowing companies to achieve greater returns on their investments in information management and business process management technologies.

The Convergence of BI and [Insert Acronym Here]

You know a technology is hot when everything is talking about "converging" with it!  I think we all recognize that business intelligence is a hot topic right now.  According to a Merrill Lynch CIO survey, business intelligence and data warehousing was the top spending priority for IT.  And recently we've seen articles and blogs talking about the convergence of BI and Search, the convergence of BI and CRM, the convergence of BI and Performance Management, the convergence of BI and Web 2.0, the convergence of BI and SOA, the convergence of BI and enterprise applications, the convergence of BI and operational business applications, the convergence of BI and BPM, and so on and so on.

We get it!  Everything is trying to converge with BI!  I will try to comment on the reality of some of these intersections in future posts, starting with the convergence of BI and BPM.

The State of Business Intelligence

David Stodder just did a great write up on The State of Business Intelligence in a feature story in Network Computing that gives a fairly complete vision of where the industry is heading, and more importantly, what organizations really need from the technology to gain business value.  The bottom line identified in this article is dead on - "Conventional data warehouses, disconnected BI and analysis tools need an overhaul. Actionable intelligence and speedier, automated decision-making are the objectives of next-generation BI."

These were the exact drivers behind IBM's dynamic warehousing strategy announced just two weeks ago.  As the Network Computing article (cleverly titled "Good BI Cruel World?") points out, "next-generation BI promises speedier, automated decision-making" and points out that most existing BI platforms are "mired in historical analysis across siloed back ends."  The key is that analytic capabilities must be injected directly into all of your different activities and business processes, and must be able to leverage ALL enterprise information across the organization, including all types of information (e.g. structured, unstructured and semi-structured).

I was talking to Mark Beyer, a Gartner analyst, about this at a recent Gartner BI Summit, and he gave us an interesting suggestion that we stop talking about our solution as a "warehouse" and start referring to it as an "Analytic Information Repository."  Given IBM's propensity to leverage acronyms, I'm not sure attempts to market AIR would go over so well, but this is in essence what we are trying to turn warehouses into - a platform that can be deployed to enable analytics on all relevant enterprise information.  Companies need technology that will enable all users across the enterprise to more easily access the information and generate the insights needed to make better decisions, faster.

I still haven't figured out what to really call this, although from initial feedback, such as Vincent McBurney's extremely entertaining post titled IBM puts the shizzel into the Data Warehouse, people seem to like Dynamic Warehousing and the new IBM Balanced Warehouse.  However, it's important to note that unlike one of the comments on that post, you can NOT achieve these types of benefits with standard databases and reporting tools.  Without dedicated warehousing capabilities, such as shared nothing parallelism and workload management, you will not be able to achieve this vision.  And even some of the new "appliance" solutions to hit the market, while providing affordable, optimized query speeds, will not be able to support the broader analytics requirements.  As highlighted by David Stodder, "appliances are customized for a specific purpose; they are not meant for mixed workloads, such as OLTP plus complex data querying, or even analytical workloads that require specific tuning to meet unusual objectives."

The next generation of BI will be what finally delivers the long promised benefits of BI to the masses.  And Dynamic Warehousing will be the AIR that enables companies to finally start realizing significant business value from their investements.