There was an interesting article in the International Herald Tribune / New York Times last week about how airlines are applying more analytics into the area of avoiding overbooking - leading to “bumping” of passengers off the flight. As a business traveller who seems to suffer this on occasions (apparently the solution is - where you can - to check in online as early as possible) I can appreciate any reduction in such deliberate service level reductions by the airlines.

Interestingly the article quotes one of the those people who exploits such poor airline service by deliberately booking on likely overbooked flights, just to volunteer to be bumped (with associated benefits). Perhaps the event-driven approach to solving these overbooking conundrums might be to track the flight bookings by these “Professional Bumpees” - if they are booked on a flight, then the airline needs to take care over overbooking!

Of course, if you starting avoiding bumping the professional bumpees, then they will soon fail to be a good indicator of your overbooking status…

In addition, of course, analytics are only a guideline based on “past events”. What you really want is to embed those “smarts” (knowledge based on experience) with the reactive “smarts” (CEP) that let you respond optimally to new situations - such as weather, aircraft maintenance, or even volcanic events!

So TIBCO Software’s CEO made a few mentions of CEP in his interview with CBR this month. But the interviewer surely made a small mistake when he claimed:

But it is by no means business as usual at Tibco as we enter 2010. Having bought grid and cloud computing software player DataSynapse last year, Insightful for Complex Event Processing in 2008 and SpotFire for business intelligence in 2007, Ranadive believes Tibco has the right portfolio to help companies turn themselves into “Enterprise 3.0” firms.

Of course,  TIBCO BusinessEvents is the TIBCO CEP product, and this is supported by the analytics provided by Insightful’s S+ (now Spotfire S+) product set.

Vivek went on to mention an airline CEP case study:

“Having a two second advantage is about being more effective,” Ranadive continues. “Take an example in the airline industry. If you lose someone’s bags and they only know about it after waiting at baggage claim for half an hour, you have angry customers on your hands. If you have the information at your fingertips, you can tell that customer via SMS as they step off the plane that their bag has been diverted accidentally, but that you will have it delivered to their hotel and deposit 1,000 air-miles in their loyalty card account. You’ll have fewer angry customers.”

Of course, this kind of foresight requires a new approach to the way that companies accumulate, manage and analyse not just data, but events. “This is what I call Enterprise 3.0,” Ranadive explains. “The key building block is the information bus. In Enterprise 2.0, the key application was ERP. In 3.0, that application is end-to-end business process. You need predictive technologies and you need complex event processing (CEP).”

So was the interviewer wrong on the Insightful-for-CEP claim? Or just displaying foresight ?

- - - - - -

Notes: in case anyone thinks that CEP (in Operational Intelligence) doesn’t make a difference, I’m reminded that last week we had a large (CEP) customer fly to visit TIBCO and remark on their flight experiences: one had flown a CEP-enabled airline and been impressed that their connecting flight had waited for them AND that they had changed the gate for their flight to be next to the waiting flight. Another had been on a non-CEP-enabled flight, and was still waiting for their baggage…

Two events last month showed indicators of a convergence between the analytics world and CEP world.

Firstly Louis Bajuk-Yorgan from TIBCO Spotfire attended the Predictive Analytics World conference in San Francisco. He reported that:

Three key themes showed up multiple times throughout the talks-the growing importance of text mining, the application of net lift modeling to determine the real results of a marketing campaign (ignoring those people who would have bought anyway), and (most interesting to me) the importance of operationalizing predictive analytics.

In his opening keynote speech, Eric Siegel (the conference chair) saw the most important innovation in the field of Predictive Analytics focused on applying predictive analytics to operational decisions (as opposed to more established application areas such as customer churn & product recommendations). In a later talk, James Taylor of Decision Management Solutions (and co-author of the great book “Smart (Enough) Systems”), echoed Eric’s emphasis on operational results, encapsulated in the phrase “Action support, not just decision support.” James advised building an analytic platform that focused on the end game: the need to operationalize analytic decisions.

This is great validation for us, since operationalizing analytics is at the heart of TIBCO’s vision for its combined platform with Spotfire and S+ (as shown in products like Operations Analytics).

Then a week later Andreas Gerst from the TIBCO BusinessEvents team presented at cepconf in Munich, Germany. Andreas presented on CEP and Data Mining, and in particular how both these complement each other for advanced operational intelligence around customer management. Andreas used TIBCO BusinessEvents and TIBCO Spotfire Miner as his example technologies, mentioning techniques like PMML for moving from analytics to real-time event processing technologies.

Intelligent Enterprise invites a few “talking heads” to give their “Intelligent Enterprise Editors’ Choice Awards” every year, so its the time now for the 2010 edition. They selected a “top 12″ and then 12 each in categories of BI, Enterprise Applications and Information Management. Complex Event Processing got a fair share of references:

• Top 12 companies: CEP mentions were for 2 for of the 12 (Informatica and Sybase) who both acquired CEP technologies in the last 12 months; in addition 3 other companies in this list have CEP technology (IBM, Oracle, Microsoft).
• BI companies: CEP gets one mention in this list, under the TIBCO banner:

TIBCO could just as easily be listed among our Information Management Ones to Watch. But this firm sets itself apart by helping companies interpret — as well as manage and integrate — data and events. TIBCO Spotfire provides trend-spotting analysis through easily understood data visualizations. The company also offers complex event processing (CEP) for the broad market, where insight into events will make real-time BI a reality.

• Enterprise Applications: one mention here too, under Progress’ Apama division.
• Information Management: CEP gets a mention here, albeit for Streambase as a merger (aka “closer ties”) candidate for Vertica (although this seems unlikely IMHO).

So how did CEP do on the hypemeter here?
CEP:  5 company summaries included “CEP” or “Complex Event Processing”
Cloud: 7 company summaries included “cloud”
BPM: 2 mentions!
SOA: 1 mention!!!

But the winner was:
Analytics: 9 mentions, and they didn’t even mention this under TIBCO (whose Spotfire S+ is a pretty good contender)!

The CEP-BI-analytics theme was also recognised by Forrester’s James Kobelius in a recent blog posting. James identifies CEP under “analytics”, and says:

In this race, the vendor that has integrated BI, CEP, and PA/DM most effectively for the mass market is TIBCO Spotfire.

He goes on: Just as important,TIBCO’s offerings enable pushdown of predictive models to CEP, data warehouses, business process platforms, and other application platforms.

So if CEP should be counted in the same boat as analytics, then indeed there is a new world order for software technology in 2010:

Cloud, SOA and BPM: 10 mentions
CEP and Analytics: 14 mentions

One of the probable future trends in event processing will be the merger of Predictive Analytics (per Wikipedia: “analyze current and historical facts to make predictions about future events“) and Complex Event Processing… or basically the increasing use of advanced statistical techniques in event processing. Of course, for “current facts” (in Wikipedia) read “events” in the Event Processing community…

As an example, consider the diagram on the right, which details some of the algorithms available in such a predictive analytics tool (TIBCO Spotfire Miner), and how they relate to event processing (as when embedded in a CEP tool like TIBCO BusinessEvents). One thing to consider is that these algorithms are not likely to be specified by Subject Matter Experts or Business Analysts - who might ordinarily be relatively happy with the logical constructs in rule-based event patterns or queries.