There were some customer discussions recently on the conceptual relationships between the CEP and BPM and SOA “software stacks”. This coincided with the announcement of the OMG-backed Event Processing Consortium being set up (alongside the merger of the SOA and BPM consortia), events which themselves can be interpreted as that event processing has some special role to play alongside BPM and SOA.

[Disclosure: note TIBCO is not a member of either the OMG's EP or BPMSOA consortia - they seem to be focused on end-user rather than vendor participation - but is an OMG member and participant in standards development. We currently see the EPTS as the main advocacy group across vendors, academics, analysts and end-users, but will be monitoring the progress of the EP Consortium].

So here are a few simplistic patterns on how CEP (event processing) relates to BPM (processes) and SOA (services)…

CEP Pattern	Example	Diagram
1a. Standalone CEP This is a bit of a misnomer - you don’t identify event patterns without some intent to use them, and at the very least such a use would be in a BAM type role displaying interesting correlations for some business person - who of course is engaged in some kind of business process, which may or may not be managed by BPM…	Monitor a production process to provide an “additional view” or dashboard for the process control manager.
1b. CEP enriching BPM processes and/or  SOA services. This is the conventional view of CEP, detecting the complex events that are of interest to, and useful in, appropriate processes and services. Complex events in these cases can be as straightforward as deducing that a deliverable has been completed, or some process truly initiated. Typically the CEP system is transforming source events into business events, for onward use in (the) business processes.	Identifying exception events in a business that need to handed to a workflow or case management system for resolution.
2a. CEP monitoring processes and services This is where the sources of events are the managed processes and services themselves. This process and service monitoring is used to detect exceptions, disparities across systems, and system performance… Note that effectively this pattern is a combination of patterns 1a and 1b above.	Detect when response times exceed some metrics and suggest corrective actions such as reallocating resources.
2b. CEP-based decisions for processes and services This is where I need to make intelligent decisions for the process and service layers, using the CEP layer as a monitoring, shared decision management component. Note that effectively this is a slight extension of pattern 2a above.	A BPMN “rule activity” sends a decision request to the CEP engine to get a valid decision for a process decision point; the CEP engine monitors the decisions made.
3a. Dynamic process and service control This is where the events from processes and services, and external services, are combined to select which processes and services are relevant to use for the current context. In effect, the CEP engine becomes the controlling agent for the business processes and service engine, handling for example dynamic process selection. Note that this pattern is a further evolution of patterns 2a and 2b.	In a complex business process for ever-changing fulfillment problems, CEP-based rules determine which sub-processes are valid based on incoming information.
3b. Embedded processes and services within CEP The final evolution of the above is when you argue that the functions of the BPM and SOA stacks can be subsumed into the CEP layer. In reality this is usually only a partial subsumption, as otherwise the centralization of services into just 1 layer could be perceived as contrary to the very idea of SOA! So this covers things like event-based policy implementations being embedded as CEP rules rather than as external services, but alongside some external services such as an operational database. Indeed, one could argue that in this case the CEP event processing agents are themselves really part of the SOA layer, not the other way round!	A service gateway controlling access to existing services, but embedding decisions, service policies, and business rules.

So, did I miss anything?

In a previous post we commented on IBM’s Conceptual Model for Event Processing Systems, but failed to observe that IBM’s rule engine group, Ilog, was noticably absent from its contributor list. So perhaps it was both fortuitous and timely that Ilog’s Daniel Selman recently added another “viewpoint” on the position of event processing (aka the rules viewpoint).

Daniel’s take on the “primary users (sic) of rules technology” are the (use cases for) automating decisions, event responses, processes and inferences. I think these might be better classified by renaming then as decision processing, event processing, and business process processing. But inferencing, which Daniel notes as being a technology to support Artificial Intelligence, is not so much a (user or) use case, but a means of providing knowledge-based reasoning to support any of decisions, processes, etc - “AI” is not (or should not be) a means unto itself.

Standard Event Processing Design Pattern

Indeed, one could probably argue that:

• all business processes are driven by events of some kind, and involve decisions and (re)actions;
• all processes, decisions and event processing are context (i.e. state) driven; however, some processes and algorithms are used in a subordinate fashion to a stateful process (consider a typical decision service that is subordinate to its application server and database layers);
• inferencing can be used to enhance any part of the event-handling process as a form of declarative rule control.

In TIBCO’s experience, an event driven rule engine (like TIBCO BusinessEvents) can be used to provide dynamic business processes, event-driven decisions, and real-time control mechanisms - and often all 3 - exploiting and building on the fundamentals of complex event processing.

So, on the utility of rule engines, we totally agree with Daniel!

I enjoyed the CIO UK article (mis)titled “Why BPM Should Be On The CIO’s Agenda in 2010″, where Dr Giles Nelson of Apama points out the benefits of event processing technology to “detect situations” and aid business processes. Giles’ article is of course part of a post-BPM-acquisition marketing campaign, and would be more accurately titled “Why Event Processing should be on BPM’s Agenda in 2010″, but he makes some valid points nonetheless:

…we need to take a different view of BPM technology and try to see how it can be used to make knowledge-based business more ‘operationally responsive’…

…the potential for creating real business value by bringing together the two disciplines of event processing and BPM is substantial…

…BPM will get really interesting when it is combined with event processing and can therefore detect situations that are occurring and automatically begin a process that takes action on that information…

…It’s about delivering real-time insight into what’s going on in processes across the organisation and, using that insight, to drive efficiencies…

All very true. Many BPM folk are of course still mostly concerned with defining what happens today, not what needs to happen tomorrow: in some ways event processing is just another step up the real-world business process maturity model.

An email from Layna Fischer, who is editor of the annual WfMC’s “BPM and Workflow Handbook”, tells us that the theme for the 2010 handbook is a “Spotlight on Process-Driven Business Intelligence (BI)”. To wit:

In 2010 we include a special spotlight on “Process-Driven BI” to illustrate how Business Process Management (BPM) and Business Intelligence (BI) are increasingly intertwined. Linking business intelligence and business process management creates stronger operational business intelligence.

Users seek more intelligent business process capabilities in order to remain competitive within their fields and industries. BPM vendors realize they need to improve their business processes, rules and event management offerings with greater intelligence or analytics capabilities.

We welcome submissions that include issues, case studies, guidance, solutions or research such as:

* BPM integration with collaboration, portals and BI
* BPM-BI with application assembly and deployment
* Dynamic event-based BI driven by BPM
* Embedding BI applications into business processes

Taking the 4 points above:

1. BPM integration with BI - this is quite an easy one as reporting (BI) on the process statistics gives insight into the long term performance of your processes - and thence areas for improvement. An obvious TIBCO example is TIBCO iProcess Spotfire - although this is more analytics than simple paper reporting…
2. BPM-BI with application assembly and deployment - I have to confess I have no idea what this phrase means! Is this BPM and BI for the software development process, perhaps? I have a feeling it is something to do with the aspect of BPM that is “visual coding” of straight-through-processes (for the event community: using BPMN to write single-event-handling applications). Traditionally there is usually insufficient data around to justify regular changes to traditional business processes, but this changes with the advent of analysis. Indeed the need to change processes often can lead to the use of things like rule-driven processes… a pretty common use case for TIBCO BusinessEvents in fact!
3. Dynamic event-based BI driven by BPM - this too can have a number of interpretations!
   • “Event-based BI” could be “event-based analytics”, through CEP. You would normally expect such “events” to drive process changes or business processes directly (e.g. “oooh look, the market trend for “wimbles” has started to go up!”). So this would be detecting events to “drive BPM”, not the other way round.
   • One could have a workflow to control manual views of your BI reports on incoming events - and this is again “BI as a process”.
4. Embedding BI into business processes - although this implied some “BI process” in a business process, I would expect it to be the simpler case of embedding BI results (decision changes, for example) into an existing business process - surely the classic use of BI in BPM… a slightly specialised case of the first point above.

But the above analysis is somewhat moot. More interesting is Layna’s assumption as indicated by the phrase:

BPM vendors realize they need to improve their business processes, rules and event management offerings …

So at least one BPM expert has come to believe that business process management encompasses managing events, rules, and processes… not just “process diagrams”!

If anyone is interested in submitting something on this, note abstracts are due Feb 17.

Gartner has published a Press Release containing a list of “Five Business Process Management Predictions for 2010 and Beyond”. These include:

• By 2012, 20 per cent of customer-facing processes will be knowledge-adaptable and assembled just in time to meet the demands and preferences of each customer, assisted by BPM technologies.
• By 2013, dynamic BPM will be an imperative for companies seeking process efficiencies in increasingly chaotic environments.

Possibly I am being dim here, but I just can’t see how to map “knowledge-adaptable” and “just-in-time” onto the current process orchestration diagramming (e.g. via BPMN) that is de rigeur in “BPM technology” circa 2010. For sure, TIBCO has BPM customers doing adaptable, event-driven, dynamic processes - but these are also using technologies like goal-driven BPM and state-driven rule-based CEP, assembling or re-using (BPMN or state model) process fragments on the fly. So Gartner reckons that 20% of BPM applications - OK, they let’s assume they mean 20% of new BPM projects - are going to be using such techniques in 2012? Well, they could be right… but its going to need tomorrow’s BPM3.0….

Other predictions were:

• Through 2014, the act of composition will be a stronger opportunity to deliver value from software than the act of development.
• By 2014, business process networks (BPNs) will underpin 35 per cent of new multienterprise integration projects.
• By 2014, 40 per cent of business managers and knowledge workers in Global 2000 enterprises will use comprehensive business process models to support their daily work, up from 6 per cent in 2009.

I can’t really comment on these - composition of business processes is a heavily constrained activity that requires a lot of knowledge of the state of the process and associated cases. As for “business process networks”, Gartner means these are preconfigured solutions or BAM applications - which again are extremely context (state) sensitive. As for use of business processes - well I’m sure many people use processes that started off as a process model today - every time you fill in a time card, log a customer visit, etc… but do you use process models? And how many of those models will be dynamic, rule-driven etc?

• About Gartner Business Process Management Summit 2010…

These predictions also happened to publicise the Gartner BPM events in the USA and Europe in March 2010… attendees can stop by the TIBCO booth for more info on TIBCO BPM+ … including CEP.