I don’t normally have much to say on BPEL (as a language for orchestrated processes). A BPEL engine can be used to execute business processes and actions post-”complex event detection”, but at the end of the day its just a particular - albeit common and standards-based - technology for executing process orchestrations [*1].

On the other hand, state management of entities is particularly interesting for Complex Event Processing. In a nutshell, the state of some entity will often determine some event pattern detection, as well as the relevance of certain business decisions, operations / processes, and actions. CEP tools handle state, and some (like TIBCO BusinessEvents) actually allow developers to model state explicitly - usually for entity lifecycle management, including complex event pattern lifecycles…

It was therefore somewhat fascinating to find a new IBM initiative called BPEL4Data and an associated Business Entity Definition Language (BEDL) comprising of entities and state models (and indeed, almost MDM-like data management operations) [*2].

One trick I think the IBM team have missed is in tying themselves to (another reinvention of) BPEL. State models involve states and event-driven state transitions: to me this means that states are part of the behavior of entities, an active not a passive attribute. And conditional state transitions mean rules rather than processes. Ergo its no accident that TIBCO BusinessEvents state models are defined in an environment that combines them with events and declarative rules.

Of course BPEL - and orchestrated BPM - have their place and use cases - indeed TIBCO has many successful BPM customers, some of whom are combining entity models and states (in TIBCO BusinessEvents) to control workflow processes (in TIBCO BPM). There are indeed many patterns to combining event-based state changes with process orchestrations, and IBM’s focus on just one will be interesting to watch.

Notes:

[1] TIBCO’s BPM group of course has much more to say on BPM and BPEL …

[2] TIBCO followers will be bemused by the acronym overloading here:
- IBM BEs (Business Entities) are directly equivalent to TIBCO BE (BusinessEvents) concepts, with UML-based state models applicable to both.
- IBM BEs map via IBM BPEL4Data to BPEL business processes, whereas TIBCO BE concepts and states are directly executed in the TIBCO BE rule engine (with no BPEL intermediate step, if you like), with BPM processes as optional (downstream) activities.

IDC’s Stephen Hendrick gave the keynote on the 2nd day of BRForum, titled “BRMS at a Crossroads”. The gist of Stephen’s talk was the need for BRMSs to evolve to the next level. Interestingly this seemed to be the first conference mention of “cloud computing” - a welcome respite from the hype about remote deployment platforms - as “cloud” was one of the future trends that businesses needed to exploit, along with open source, visualization platforms, and, naturally, decision management…

Stephen started with an overview (based on IDC research) of the BRMS market: in 2008 this was worth $285M with a “10.5%” annual growth, with the 2 leading BRMS vendors taking 40% of that market. Of the 7 BRMS vendors he mentioned, TIBCO was rated as being joint 2nd due to its “strong legacy in rules and being well positioned to execute”…

He then introduced the “IDC Decision Framework”: measuring the “scope of decision” vs “degree of automation” vs “no of decisions” vs “level of collaboration” for any application area.

Onto the vision of a future Decision Management platform: this  needs better “data preparation” (which maybe means MDM)  and “decision refinement” (covering predictive analytics). But CEP events needed to drive the decisions, whose decisioning context was defined as handled by “state“. All these management constructs obviously mapped to a runtime platform for “active decisioning” / “always on” behavior…

Justifying the CEP connection, Stephen mentioned that in 2008 at least 20% of BRMS/decisioning deals had some kind of “real time” orientation which was expensive to handle in the “passive BRE” environments.

The key concept Stephen described was the move from the “process centric to information centric” approach. Sharp intake of breath from the BPM community present… however Stephen explained that fine-grain rule and event control and parallel processing were key, and useful, features of the CEP world.

Stephen ended with the comment that “cloud computing” will also be event-driven - so decision management platforms that are event-based will makes sense in the cloud deployment world. [And funnily enough, CEP certainly plays a role in TIBCO's Silver cloud offering].

From a TIBCO perspective, clearly the concept of state management and modeling, with event processing, decision management, MDM and analytics are all part of the “best practice” decision platform. Probably the “information centric” world will not replace the “process centric” world any time soon, but for those customers want to take this route, its good to know there are already vendor solutions…

Meanwhile, Sandy’s view of the talk can be found on her blog…

iGoogle duly delivered a reference (but not the referencing blog post) to an interesting IBM Research paper on “providing a new foundation for the Management of Business Operations and Processes”. To summarise (or at least, my interpretion was) :

• activity diagrams (and by extension, process diagrams) are problematic due to the effect of silo-based viewing and cross-process communications
• the key business-level model is the business artifact state model
• state models should be supported by forward chaining business rules to match events to business goals and strategies.

The good news: state models supported by production rules is available today to TIBCO BusinessEvents customers.

The bad news: it seems we have a new IT acronym for such systems: Business Operations Management or BOM. All abbreviations have to be overloaded it seems, so this time it is the turn of Business Object Models to be re-used. In some ways I like the BOM term in the context of operations as (a) it differentiates event-driven tools like BusinessEvents from the more traditional BPEL-focused BPM engines, and (b) supports the idea of Operational Intelligence that CEP tools provide.

Not surprisingly the paper omits to mention TIBCO in the “related work” slides, but it does cover healthcare case management - which hopefully explains our interest in the current discussions on an OMG Case Management standard… (case management being a special case itself of “artifact centric BPM”, a conference on which was the source for the research slides referenced earlier…).

I had promised Prof David Luckham at the last EPTS meeting I would post some notes from a BCS event I attended a while back, given by the IT department of JET. The topic came up again in a response to Tim Bass’ Linkedin CEP blog where Tim implied the CERN LHC’s event processing was something the EPTS Languages Working Group should have (or should in future be) taken into account - a fair criticism provided you think the LHC and its ilk’s event processing needs are “typical” or repeatable elsewhere.

Back to my notes on JET: they used custom hardware (>21,000 rack mount modules of >400 types) for event-data compression and collection - what could be described as a “hardware tree database” with event channels leading to a “lightweight” software mapping system controlled via APIs. However, the IT systems were mainly concerned with collecting valid events and exceptions for post-experiment analysis - each experiment having something like 22,000 parameters to deal with! Events were not just from the Torus, either - with such expensive experiments they were monitoring things like the data collection hardware. The control systems used a state model and a control loop executing at 500 iterations per second. The experiments were defined as state models that were maintained in a custom content management solution based on RDF. If I recall, their main programming languages were C++ and Fortran.

Interestingly, state management was one of the language aspects covered by the EPTS Languages working group… and state modeling is also a feature of TIBCO Business Events…

Tim Bass recently challenged [*1] the assertion that Complex Event Processing provides “situation awareness” and quoted a Wikipedia article covering this topic from the usual human element. The article interestingly includes a generalized diagram of situation awareness  [reference Endsley, M. R. (1995b). Toward a theory of situation awareness in dynamic systems. Human Factors 37(1), 32-64] that maps quite nicely to CEP (although it is obviously targeted at a more human-oriented system).

So lets review the Wikipedia definition of Situation Awareness (SA) and compare to CEP:

The most common theoretical framework of SA is provided by Dr. Mica Endsley (1995b). Endsley’s model illustrates three stages or steps of SA formation: perception, comprehension, and projection.

Perception (Level 1 SA): The first step in achieving SA is to perceive the status, attributes, and dynamics of relevant elements in the environment. Thus, Level 1 SA, the most basic level of SA, involves the processes of monitoring, cue detection, and simple recognition, which lead to an awareness of multiple situational elements (objects, events, people, systems, environmental factors) and their current states (locations, conditions, modes, actions).

• CEP comparison:  this is about monitoring events and the states of entities - basic CEP functionality. For example, TIBCO BusinessEvents uses events from various channels that can be monitored, and a state model / engine to represent any spatial or temporal aspect of the elements being monitored.

Comprehension (Level 2 SA): The next step in SA formation involves a synthesis of disjointed Level 1 SA elements through the processes of pattern recognition, interpretation, and evaluation. Level 2 SA requires integrating this information to understand how it will impact upon the individual’s goals and objectives. This includes developing a comprehensive picture of the world, or of that portion of the world of concern to the individual.

• CEP comparison:  this is about recognizing patterns of events and using reasoning tools (such as inference rules) to interpret and evaluate these patterns. Goals and objectives are usually represented as end states in a CEP system. For example, TIBCO BusinessEvents uses rules and (optionally continuous) queries to define patterns, and an inference engine and state model for reasoning to goals and objectives.

Projection (Level 3 SA): The third and highest level of SA involves the ability to project the future actions of the elements in the environment. Level 3 SA is achieved through knowledge of the status and dynamics of the elements and comprehension of the situation (Levels 1 and 2 SA), and then extrapolating this information forward in time to determine how it will affect future states of the operational environment.

• CEP comparison:  this is about associating certain complex events with predictions about likely future states and reporting or making decisions based on them. For example, TIBCO BusinessEvents uses rules and managed decisions to take certain actions based on the detection of these complex events.

SA also involves both a temporal and a spatial component. Time is an important concept in SA, as SA is a dynamic construct, changing at a tempo dictated by the actions of individuals, task characteristics, and the surrounding environment. As new inputs enter the system, the individual incorporates them into this mental representation, making changes as necessary in plans and actions in order to achieve the desired goals. SA also involves spatial knowledge about the activities and events occurring in a specific location of interest to the individual. Thus, the concept of SA includes perception, comprehension, and projection of situational information, as well as temporal and spatial components.

• CEP comparison:  this is about the importance of time and location. Indeed, you could substitute CEP for SA and still have a meaningful paragraph above. For example, TIBCO BusinessEvents uses time events and rules as well as states to manage the temporal aspect of reasoning.

* * *

To conclude, it seems there is a close association between CEP and situational awareness. Although automated tools may never have the perspective or detailed semantics of human awareness, CEP technologies can certainly provide a lot of the functionality required to assist operations. Indeed there are a few BusinessEvents applications that do just this - provide oversight of operational systems such as BPM and SOA. Are all possible situations detected? Probably not; however many are and that provides good business value to our customers.

 Notes:

[1] Tim clarified in the comments that he was claiming that “CEP software”, not the “concept of CEP”, didn’t “much” support Situation Awareness. As there is no common-use checklist as to what “support for SA” implies for software, we’ll just have to agree to differ on that point!