Marc Andrews Observations On Demand

Information On Demand Goes To EMEA

This week, the first EMEA based Information On Demand (IOD) conference was held in The Hague (Den Haag) in the Netherlands.  There were around 2,000 people from across all of Europe, and even some countries in the Middle East and Africa, who made the trip to hear more about how companies are driving more value from their information.

One of the main messages heard at the event was the need for companies to start thinking about their Information Agenda - how they can better leverage information across their organization to drive more business value, the infrastructure they need in place to support this, and a roadmap for how to get there.  Ambuj Goyal, General Manager of the Information Management Software division at IBM, encouraged people to take information to the next level, and to use it as a strategic asset.  He indicated that organizations who establish trusted information will be able to unlock the business value of information for competitive advantage...and those who do not run the risk of being left behind.

René Carayol, international business guru and media personality, invited customers to the stage to share their experiences in implementing their own Information Agendas.  Jacqui Leggetter, from the Department for Work & Pensions in the UK, spearheaded an effort to integrate enterprise content management (ECM) with customer relationship management (CRM).  As a result, the Department reduced a five day paper-intensive process to one, enhancing the customer experience.

Markus Bayha, from Fiducia, the largest IT service provider to the financial services community in Germany, garnered C-level sponsorship to implement a solution that now effectively serves 800 banks.  With its new-found capabilities, Fiducia is able to help customers increase revenue through the sales of new banking products and services, and can now monitor, control and analyze information to drive its future sales and marketing strategy.

In addition, Nick Donofrio, IBM's Vice President of Innovation and Technology, spoke to the attendees, reminding them that harnessing information is an essential element of innovation that matters, which delivers client value, creates new business models, and spurs growth. He encouraged people to combine technology with business insight, and to lead change versus enduring or tolerating it - only with change comes innovation.

Overall, it was a great event that demonstrated the enthusiasm and interest of companies all over EMEA in driving increased business value from their information.  One of the biggest questions this leaves everyone to take away is, what's YOUR Information Agenda?  Do you have one?

Focusing on Business Optimization

Business optimization seems to be one of the latest buzz phrases out there.  But what does this really mean?  And how is this different from what companies have been doing in the past?

Well, for the past two decades, companies have primarily focused their IT investments on automating their business processes, with the objectives of driving faster processing and reduced costs.  This has been driven by an "application" agenda to implement ERP and Financial applications, supply chain management solutions, and call center and CRM applications.  However, these types of investments are no longer creating a sustainable competitive advantage for organizations.

As a result, over the past few years, we've seen new IT initiatives increasingly focused on optimizing business to drive a more sustainable competitive advantage.  Think about it like those old BSF advertisements - we don't make a lot of the things you use, we make a lot of the things you use BETTER!  Business optimization is about making your business processes, customer interactions and everyday operating decisions BETTER. This means moving from just leveraging ERP and financial applications to providing increased financial risk insight for better business decisions...moving from just managing your supply chain to enabling more dynamic demand planning...and moving from just managing your call center and customer relationships to providing increased insights to improve customer service and drive greater profitability from your customers.

These new initiatives are all dependent on information, and having an information agenda in place to ensure that your organization is leveraging all the information at its disposal to create a competitive advantage in the market.  So ultimately, information becomes the key to optimizing your business.

Why You Should Understand What's Going On With Your Data Warehouse

Data warehouse offerings are increasingly adding new tools to help manage and re-configure your warehouse with the goal of increasing performance.  In fact, IBM recently announced new "extreme" workload management capabilities as part of its recent DB2 Warehouse release to go beyond priority based workload management and enable companies to truly address the requirements for operational BI applications that must reliably deliver real-time responses.

However, to best take advantage of these tools, it is important to understand exactly what's going on with your data warehouse.  Many organizations are constantly re-configuring their data warehouse environments in a trial by error approach, blindly re-partitioning their data, setting up new nodes and defining various workload classes.  The best way to use these tools and make configuration changes is to first understand what data is being used, how it is being used, and by what users and applications.

IBM just introduced a new suite of tools to help organizations better understand how their warehouse is being used, and leverage that knowledge to improve performance, making it easier to grow their data warehouse environments.  This will not only provide better performance and greater scalability, but will help enable companies to use their enterprise data warehouses for operational BI applications.

A recent TDWI article provides some details on this.

Why all the BI acquisitions? What is required for BI?

What does it take to implement a business intelligence solution?  BI tools alone do not do the trick.  While there are a few scenarios where it makes sense to just point a BI tool at a transactional database, most situations require that the information be extracted into a separate data store for reporting and analysis to prevent impacting the operational applications.  Thus, most BI solutions require implementation of not only the BI tools, but a data mart or warehouse to store and manage all of the information for reporting and analysis.

In most scenarios, reporting and analysis requires a different view of the information than what has been delivered in the transactional applications, requiring different meta data schemas and data models.  And unfortunately, no one has reached the nirvana state where all of their information is coming from one place. As a result, most BI solutions also require more advanced information integration capabilities, including the ability to aggregate data from multiple sources, cleanse the data for improved quality and reliability, and transform the data in preparation for loading into data marts or warehouses.

And to really ensure you can trust the reports and analysis being generated, and know that those 3 sales are all really for the same customer, or the same product, it is becoming increasingly important to manage master data across your operational and analytic systems.

So, BI is not just about the BI tools (e.g. query, reporting, analysis, dashboards, scorecards, etc.).  It also requires data warehousing capabilities, information integration and master data management.  A complete platform to establish accurate, trusted information AND the tools to report on and analyze that information.  And this is why we are seeing a lot of acquisitions and consolidation in the BI space.  Both the vendors and the customers are realizing that they need more than just one piece of the puzzle to deliver business intelligence.

And if you want to really complete the puzzle, then you will need the underlying hardware to run all of these components (ideally, hardware that is optimized for the different components), expertise to help you establish the appropriate strategy and implement the solution successfully, and if possible, accelerators for your particular industry and the business domains of importance to you.

This is what is required to implement a BI solution.

The Convergence of BI and BPM

The convergence of BI and BPM is actually not new.  Forrester wrote about it 2 years ago and talked about how BPM products needed to integrate with BI "to support decision-intensive business processes like order management, credit risk management, and sales opportunity management," but that BPM vendors were not currently thinking about BI in this context.  Well, the market seems to be finally catching on.  Forrester is working on a new report on this topic, and there was just a great article posted on TDWI on The Myths of BPM, where they go as far as stating that "you can’t implement BPM or scorecards without a solid business intelligence and data warehousing infrastructure."

I recently participated in an ebizq podcast on BI in BPM and one of my colleagues brought up a great question about how IBM is delivering products to enable this convergence, and just as importantly, whether or not we had any customers that are really doing this.

This is actually a big focus area for us, and a major part of our new Dynamic Warehousing initiative.  Our belief is that the next generation of BI is about delivering information and business insights directly into operational applications as part of real-time business processes and activities.  This will enable a broader set of people across the organization to start leveraging BI, enable organizations to use BI for improving everyday business decisions, and drive up the value of their existing investments in related information infrastructure.

We are excited about this trend because we believe that our warehouse offering, DB2 Warehouse, is better suited to enable dynamic warehousing capabilities and address the demands of this next generation of business intelligence.  In addition, it drives requirements for a broader set of information management capabilities, which IBM is best positioned to deliver through our broader information management stack, including Information Server for information integration, WebSphere Customer Center for master data management and our industry data models to apply industry specific business perspectives.  Of course, with the need to deliver information and analytics directly into operational processes, and use that information as part of the decision making process, this will also drive greater demand for process management capabilities, which IBM can provide with WebSphere Process Server for transactional processing and IBM FileNet P8 BPM for content related processes, along with related BPM tools, such BAM and rules.

And in response to the question about real customer examples, I'm happy to say that we have several already.  One is Blue Cross Blue Shield of Tennessee, a large health care insurance company in the US.  They have moved from just reporting on customer issues to delivering insights on prior customer problems directly into their call center application so that a customer service representative can better understand and resolve the customer issues.  The Real Time Crime Center at the New York City Police Department (NYPD) is another great example, and there is a great video on this success story.  They are delivering relevant information to police detectives while they are being dispatched to the scene of a crime so that they already have a list of leads, potential suspects and their possible whereabouts before they even get there.  A complete case study is also available.  Kookmin Bank, in Korea, is an interesting example of a company that justified their initial investment based on the need to address Basel II requirements, but decided to analyze customer risk as part of its credit application process, as opposed to afterwards.  As a result, they were able to reduce credit processing errors by over 30% and improve the overall credit quality of their loan portfolio to better compete in their lending business.

So as you can see, the convergence of BI and BPM has finally arrived.  IBM is helping enable this convergence through Information On Demand initiatives such as Dynamic Warehousing.  And this is already proving to deliver real business value and allowing companies to achieve greater returns on their investments in information management and business process management technologies.

The Convergence of BI and [Insert Acronym Here]

You know a technology is hot when everything is talking about "converging" with it!  I think we all recognize that business intelligence is a hot topic right now.  According to a Merrill Lynch CIO survey, business intelligence and data warehousing was the top spending priority for IT.  And recently we've seen articles and blogs talking about the convergence of BI and Search, the convergence of BI and CRM, the convergence of BI and Performance Management, the convergence of BI and Web 2.0, the convergence of BI and SOA, the convergence of BI and enterprise applications, the convergence of BI and operational business applications, the convergence of BI and BPM, and so on and so on.

We get it!  Everything is trying to converge with BI!  I will try to comment on the reality of some of these intersections in future posts, starting with the convergence of BI and BPM.

What Exactly is the IBM Balanced Warehouse?

IBM recently announced the introduction of the IBM Balanced Warehouse, and there have been a few questions about this and how it is related to the BCU (Balanced Configuration Unit) and our recent Dynamic Warehousing announcement, so I thought I'd try to explain what it is...and what it isn't.  Let's start off with a quick recap on Dynamic Warehousing.

Dynamic Warehousing is about providing extended capabilities and infrastructure to enable third generation BI, which can be used to improve real time business processes and create competitive advantage. DB2 Warehouse is at the heart of IBM's solution for Dynamic Warehousing with its unique "best of both worlds" architecture that provides the benefits of an OLTP based, transactional data server, along with dedicated warehousing capabilities that enable it to better handle larger data volumes and more complex queries and analytics. This, along with the embedded analytics and support for extracting knowledge from unstructured information, make it the ideal platform for dynamic warehousing.

The IBM Balanced Warehouse is something completely orthogonal, and a new set of offerings that can be leveraged to simplify and optimize deployments of Dynamic Warehousing through an appliance like approach. It is the evolution of our BCU methodology to provide greater simplicity and supportability, additional entry points for more flexibility, and optimized performance by combining IBM hardware, storage and software as all encompassing, out-of-the-box building blocks that can be used to address an organization's enterprise wide or departmental warehousing requirements.

The IBM Balanced Warehouse comes pre-configured, pre-tested and pre-certified. It can be considered an appliance with benefits and there are several advantages to this solution. In addition to the drastic reduction in deployment time, we have found that a majority of customer problems or performance concerns with their warehouse are a result of configuration issues - most of which we have found to be resolved when customers deploy an IBM Balanced Warehouse. In addition to the resulting reduction in support issues, IBM Balanced Warehouse customers get a single number for complete solution support - hardware, storage and software. However, unlike most appliances, IBM provides multiple offerings that can address the entire spectrum of data warehousing needs - from large enterprise data warehouses, to high performance, departmental data marts, to packaged data warehouse solutions for small and medium sized businesses.

The IBM Balanced Warehouse actually came out of our initial efforts to provide customers with a methodology for deploying individual warehouse server nodes - balanced configuration units (BCUs). There have been several customers that have taken our BCU methodology and customized it to match their existing hardware and/or storage assets. This methodology helps them reduce the overall deployment time, but was just an initial step.  Customers purchasing a true IBM Balanced Warehouse experience even greater reduction of support issues, and benefit from having a pre-configured, pre-tested solution.

There are currently a lot of companies that have leveraged the BCU methodology, but are not necessarily "IBM Balanced Warehouse" customers - mainly because the BCU methodology has been around longer than the IBM Balanced Warehouse. However, we expect this trend to change.

Of course, we recognize that some customers will continue to have biases towards certain infrastructure components, and we will continue in true IBM fashion to maintain openness and support alternative hardware and storage devices. However, customers should clearly understand the additional value and benefits of a "true" IBM Balanced Warehouse.

Data Warehouse Appliance Vendors - Are you willing to risk your business?

Data warehouse appliances seem to be a growing trend, and a few small vendors, such as Netezza and DataAllegro, took in a bunch of venture capital and began shaking up the market with low cost, alternative solutions.  They have been competing on "good enough" functionality - I'll admit that they enable fast query and reporting, but, as they have admitted, they don't have all the bells and whistles of warehousing solutions that have been around in the market - but they have really differentiated on their low cost and simplicity.

Now, as Netezza announces their intentions to go public, we get our first glimpse into their actual financials.  And as told in the recent TDWI article, In Search of...DW Appliance Profitablity, and in Computerwire's Data Warehouse Appliances: a Profitable Venture?, the story is quite telling.  It is now clear that they have been overly aggressive in their attempts to "buy" customers, but have not truly figured out how to deliver a successful profitable warehousing solution.  As most of you know - and hopefully those of you that are buying  business intelligence platforms already know this - a company cannot survive if they continue to lose money and do not have any profitable products.

Netezza lost $14 million in fiscal 2007, $20 million in 2006, $7 million in 2005, almost $14 millin in 2004, and $17.5 million in 2003.  That's over $70 million lost in 5 years...and they only have around 70 customers! Thus, one of two things is going to happen.  Either these companies are going to have to start charging more money, and lose one of their key benefits - lower cost - OR, they will go out of business.  Either way, when you look at their financials, you should be extremely worried about investing in their products, unless you are just looking for a very short term solution that you are willing to throw away in a few years.

So the question is, are you willing to risk your business on this type of solution?

The State of Business Intelligence

David Stodder just did a great write up on The State of Business Intelligence in a feature story in Network Computing that gives a fairly complete vision of where the industry is heading, and more importantly, what organizations really need from the technology to gain business value.  The bottom line identified in this article is dead on - "Conventional data warehouses, disconnected BI and analysis tools need an overhaul. Actionable intelligence and speedier, automated decision-making are the objectives of next-generation BI."

These were the exact drivers behind IBM's dynamic warehousing strategy announced just two weeks ago.  As the Network Computing article (cleverly titled "Good BI Cruel World?") points out, "next-generation BI promises speedier, automated decision-making" and points out that most existing BI platforms are "mired in historical analysis across siloed back ends."  The key is that analytic capabilities must be injected directly into all of your different activities and business processes, and must be able to leverage ALL enterprise information across the organization, including all types of information (e.g. structured, unstructured and semi-structured).

I was talking to Mark Beyer, a Gartner analyst, about this at a recent Gartner BI Summit, and he gave us an interesting suggestion that we stop talking about our solution as a "warehouse" and start referring to it as an "Analytic Information Repository."  Given IBM's propensity to leverage acronyms, I'm not sure attempts to market AIR would go over so well, but this is in essence what we are trying to turn warehouses into - a platform that can be deployed to enable analytics on all relevant enterprise information.  Companies need technology that will enable all users across the enterprise to more easily access the information and generate the insights needed to make better decisions, faster.

I still haven't figured out what to really call this, although from initial feedback, such as Vincent McBurney's extremely entertaining post titled IBM puts the shizzel into the Data Warehouse, people seem to like Dynamic Warehousing and the new IBM Balanced Warehouse.  However, it's important to note that unlike one of the comments on that post, you can NOT achieve these types of benefits with standard databases and reporting tools.  Without dedicated warehousing capabilities, such as shared nothing parallelism and workload management, you will not be able to achieve this vision.  And even some of the new "appliance" solutions to hit the market, while providing affordable, optimized query speeds, will not be able to support the broader analytics requirements.  As highlighted by David Stodder, "appliances are customized for a specific purpose; they are not meant for mixed workloads, such as OLTP plus complex data querying, or even analytical workloads that require specific tuning to meet unusual objectives."

The next generation of BI will be what finally delivers the long promised benefits of BI to the masses.  And Dynamic Warehousing will be the AIR that enables companies to finally start realizing significant business value from their investements.

Dynamic Warehousing and the Next Generation of Business Intelligence

This past week, IBM announced a comprehensive strategy to enable dynamic warehousing, a new generation of business intelligence capabilities that enable organizations to gain real-time insight and value from their business information.  So what exactly is dynamic warehousing?  And what is this next generation of business intelligence?

Well, dynamic warehousing is about making information AND analysis available as needed to address ALL of the different demands organizations have for business intelligence.  This includes the traditional needs for historical reporting and analysis, but also extends to generate increased business insights from all information across the enterprise, and enables that knowledge to be delivered in real-time when needed - often referred to as operational intelligence.  Dan Everett, at Ventana Research has been talking about this concept and some of the challenges organizations face in achieving this lofty goal for over a year.  Let' s break it down by looking at what companies started out doing with BI tools to better understand what this next generation of cabilities is about.

Initial efforts at generating business insight were focused on query and reporting to understand what happened.  Police agencies reported on crime statistics to learn about frequent criminal activities, insurance companies reported on fraud to determine how much money to write off, and retailers reported on store sales to find out which locations were underperforming.  The second wave focused on technologies such as online analytical processing (OLAP) and data mining for historical analysis to understand why things happened and recommend future action – strategic and tactical planning.  Police agencies began analyzing criminal activity to optimize deployment of personnel, insurance companies analyzed characteristics of fraud for better customer segmentation, and retailers analyzed customer purchases to improve merchandising and better manage inventory.

Dynamic Warehousing provides a new approach to companies who want the ability to rapidly analyze and act upon the hidden benefits of their business information.  This third generation approach is about making available and analyzing information on demand to optimize each transaction – in the call center, in the field, when helping customers, or when taking orders.  Dynamic Warehousing is helping the New York City Police Department (NYPD) identify related incidents upon receiving an emergency call and deliver a list of potential suspects in real-time to detectives before they arrive at the crime scene (see the video).  Likewise, insurance companies can identify potentially fraudulent claims prior to approval and payment, while retailers can rapidly leverage buying patterns and changes in consumer behavior to identify the most effective cross sell and up sell opportunities at the point of sale.

There are four key things required to enable dynamic warehousing:

1. Real-time access, in context - being able to access and work with aggregated, cleansed information in the context of the business process or activities being performed
2. Analytics, as part of the business process - going beyond just providing access to information and providing embedded analytics capabilities that can be leveraged as part of a real-time business process
3. Incorporating knowledge from unstructured information - leveraging all types of information, including the ability to extract additional insights buried in unstructured forms of content, such as free form text fields, documents, web pages, audio recordings, etc.
4. Extended capabilities, tightly integrated - providing a complete set of integrated capabilities that extend beyond the warehouse to enable use of information on demand.  This includes search and text analysis, information integration, business process management, enterprise data modeling, master data management and industry specific models to make these technologies more applicable to a particular business area.

These are the key tenets of dynamic warehousing.  And these are the things needed by organizations to further innovate their businesses and drive the next level of competitive differentiation.