Traditionally (i.e. for the past 2-3 years!) CEP has mainly been covered by the (mostly academic) DEBS event [3rd is this year in Nashville, July 6-9] and the EPTS Symposium [5th is this year in Trento Italy, Sept 21-23]. Interestingly there is increasing spin-off of CEP into other events - such as (interesting for rule-driven CEP tools like TIBCO BusinessEvents) the Business Rules Forum [12th this year in Vegas - presumably as an antidote to this year's economic news - Nov 1-5]. BRForum has acknowledged 2 very-CEP-related agenda entries by 2 very-respected-experts:

Event Processing 2010: Past, Present and Future
David Luckham, Emeritus Professor, Electrical Engineering, Stanford University

This tutorial on Complex Event Processing (CEP) will cover six topics.
1. Developing markets for event processing - a short survey of the growth of CEP in enterprise management applications and Business Activity Monitoring.
2. History. - Event processing 1950 ─ 2000.
3. Adopting event processing — how to analyze your event processing requirements and plan a solution.
4. A survey of basic CEP concepts and their applications.
5. Crossing the Chasms - the four stages in the development of event processing from 2000 to 2050. The need to improve the CEP technology in commercial tools and applications.
6. The age of ubiquitous CEP - event processing goes global and disappears under the hood. Scenarios of current and future applications.
What you will learn:
• What Complex Event Processing is
• How to apply CEP to solve business problems and improve your BI operations
• How CEP enhances Service Oriented Architectures, Business Process Management, and Business Rules systems

A Facilitated Peer-to-Peer Workshop: Semantic Processes, Services and Events
Paul Haley, Founder, Automata, Inc.

Semantic technology provides the most general and flexible form of data modeling along with logical and rule-based capabilities. A new wave of semantic tools and standards, including models of time, events, and processes promise to align enterprise data modeling, application development, service-oriented architecture and business process management more closely with the perspectives of knowledge management and business rules practitioners.
What we will discuss:
• How semantic standards extend model-driven architecture to knowledge management
• How semantic architectures and models unify SOA and BPM, including events
• How semantics increases the impact of business intelligence and activity monitoring
• How BPMN, SBVR, PRR and complex event processing do or don’t intersect

Just the week before BRForum there is the rule-developer-focused October Rule Fest (which I keep wanting to write as Oktober Rule Fest, for some reason) [in Dallas, Oct 26-30] which, apart from a fascinating agenda for rule IT folk, has CEP topics such as:

Early Alert System at Southwest Airlines
Greg Barton: Southwest Airlines, Senior Software Engineer

Southwest Airlines is venturing into the rules development space with the Early Alert System. EAS enables SWA to have a real-time model of it’s entire aircraft fleet, tracking such activities as taxi in, taxi out, and in gate turn. It does this by maintaining a data structure representing physical assets and the activities they perform. Incoming data from those assets update the data structure, and rules react to the changes. We hope to use this paradigm going forward to use rules to monitor other aspects of the enterprise, enabling a more agile and efficient response to the airline’s daily operating challenges. Our main points will be the Overview, Feature Review, Design, Other Uses of Rules at present by SWA and the future of rules at SWA. [Note: this is a TIBCO CEP application in production at SWA]

ET2: Temporal Reasoning: a requirement for CEP
Edson Tirelli: Drools, CEP Designer

As Complex Event Processing grows in popularity and applicability, the convergence between modeling paradigms demand more and more functional requirements from the available tools. One key requirement for CEP use cases (and standard business scenarios) is the ability to write rules and queries that require some degree of temporal awareness, from simple constraints to actual data inference. More than that, temporal reasoning is a feature on top of which the actual platform can leverage internal optimizations, aiming for resource savings and improved scalability. [Note: DROOLS has now become the 2nd CEP inference engine]

A Survey of Complex-Event Processing Models
Charles Young: Solid Soft, Principal Consultant

Prof. David Luckham defined an EPN (Event Processing Network) as a network of ‘lightweight rules engines’ which act as Event Processing Agents (EPAs). He contrasted this with the exploitation of rules-based inference engines as ‘heavyweight EPAs’. Complex-event processing (CEP) is inherently rule-based and centres on pattern matching based, in large part, on temporal constraints. CEP, today, is broadly characterised by the use of diverse processing models embodied within different technologies. What are these models? What are their major differentiators, strengths and weaknesses, and how do they compare with Rete engines and other rules processing approaches? Are some models truly more ‘lightweight’ or ‘heavyweight’? What are the underlying differences and similarities and how might each approach best be exploited in building scalable and agile EPNs?

Still to be announced is the RuleML09 event’s agenda [conveniently co-located with BRForum, Nov 5-7] and targeting the rule representation community… we’ll have to wait and see if CEP gets represented here too!

Disclaimer: TIBCO is presenting at BRF and ORF on rule-CEP topics, too.

Not really too CEP-related, except that events and event processing can take place anywhere: TIBCO’s Cloud Computing initiative is an announced agenda item for next week’s TIBCO NOW global webevent. I haven’t seen the content, but clearly the main area of interest is likely to be Corporate Cloud Computing (i.e. your distributed-managed data center), and there is an obvious connection to virtual service management (i.e. TIBCO’s ActiveMatrix product set).

TIBCO NOW also includes some EP/CEP content under the Enterprise Optimization thread:

• • Supporting Dynamic Change Management with CEP

  • Philippe Goujard, Senior Manager, Accenture
  • For a major micro-electronics manufacturer any change request is a long and complex process: it impacts machinery, production, logistics, training, and documentation. By introducing a complex event processing solution based on TIBCO BusinessEvents the client optimized monitoring of a complete “change request” project, improved time to market, and reduced risks associated with conflicting projects.

• • Keeping the Lights On while Managing the Complexities of Compliance

  • Lyndon Wilkin, Solution Manager, EnergyAustralia
  • EnergyAustralia (EA) has over 100 years of experience providing broad-based energy solutions to over 1.6 million homes and businesses. Hear how EA is using TIBCO software to gain real-time visibility into customer service levels, to detect and respond to events that could affect compliance, market obligations, and providing a foundation technology for enabling the intelligent grid.

• • The Future of Enterprise Optimization

  • Alan Lundberg, Senior Product Marketing Manager, TIBCO Software
  • Visibility into every facet of your business is key to improving quality, customer service, and cost control. TIBCO provides the ability to harness the power of existing data and in-flight information across your many customer touch points in real time. Managing your business at the speed of events is essential to be a leader in today’s market. This session will discuss the event processing technology that SouthWest, QWEST and CitiGroup use to drive competitive advantage and that analyst firm IDC ranked number one in the market.

Registration is, by the way, free.

Here’s a quick announcement that might be of interest to those of you reading our blog.

We have recently been informed that our most favorite CEP product, TIBCO BusinessEvents, has been nominated, and by virtue of independent judging and votes from the public, has now made it to the Finals of the American Business Awards aka The Stevies.

So if, like me, you have a strong affinity for BE, or would like to see any CEP product gain more positive non-industry exposure, you ought to head over to cast your vote at: http://peopleschoice.stevieawards.com/default.cfm

The category is for - New Product or Service of the Year – Computer Software, New Version: TIBCO CEP BusinessEvents 3.0

Final judging begins today through June 1st. I’m told that the race in these categories is tight and every vote counts.

The ABA plans to publicly announce the winners on Tuesday, June 9. The winners will be invited to attend the 7th annual American Business Awards banquet in New York on Monday, June 22 to accept their special People’s Choice trophy(s).

We will keep you posted on any developments.

Work on the EPTS Reference Architecture is moving forward, with excellent contributions from the event processing industry and academia (EPTS members can observe the work-in-progress on the EPTS working group wiki). One interesting angle of discussion revolves around the compatibility and overlap of “event processing” with conventional “data processing” technologies, especially when it is possible to view “events” as “data” and process them as such. Of course, critics of CEP will counter that all event processing tools simply re-use existing data processing techniques (such as extended versions of SQL engines, or inference engines, or neural nets etc…), and to some extent they are right - what’s new is the idea of re-arranging these techniques to exploit higher-performance CPUs and larger amounts of available memory for in-situ event pattern detection in everyday business problems.

Defining a generalized event processing (and complex event processing) architecture that covers all use cases, even of a single (albeit ambidextrous) tool like TIBCO BusinessEvents, is rather challenging. There are a number of different architecture patterns possible, covering different functional and performance requirements. One generalized effort is included here, based on TIBCO’s work for customers in this area:

The various tiers can be described as:

Tier 0 (not displayed):  The sources of events could be anything from SCADA control systems, RFID detectors, BPM or ERP systems, or whatever.

Tier 1: Event channel (or transmission medium). In TIBCO’s world this is usually EMS (topic-based or queue-based) or RV message buses, although it could also easily be some other middleware mechanism.

Tier 2: Event PreProcessor. Optional. Handles basic event filtering, and roles such as concept enrichment (when the event is just adding information to an existing event object), concept creation (the event is transformed into an event object, or concept in BusinessEvents), and forwarding to local and/or distributed memory (a.k.a. event store, for processing elsewhere).

Tier 3: Distribution Agent(s) or Event Store. Optional - used for high-performance, high volume, fault-tolerence and large-scale CEP. Provides storage for events (and event objects) required for the event patterns representing complex (aggregate, abstract, derived) events - where these event patterns may occur over periods of time. Needed (in conjunction with Tier 4…n) where the volume of events, or amount of event processing, exceeds the limits of a single process node’s memory or CPU limits. Used also when information needs to be replicated across nodes to support fault tolerance and failovers. For performance reasons this is often some kind of high-performance cache or data grid (as is the case of BusinessEvents’ cache component), but it could also be a common-or-garden file-oriented database… in TIBCO BusinessEvents we support both approaches.

Tier 4…n: Event Processing Agent(s). In the simplest case, a single EP Agent can attach to a channel and process events (2-tier). Or it can involve a Preprocessor either in-process or as a separate agent (2.5-tier). Or be used with the cache (4-tier). Ultimately these can be organised as a hierarchy of EP services, with these services themselves constituting multiple, load-balanced, identical agents. These EP Agents define event patterns (through rule conditions, state transitions, and queries) and communicate either directly to event channels [Tier 1] or, more often, through the event store [Tier 3]. Large workloads can be distributed through queries (i.e. pull model) or through shared event / event object updates (i.e. push model). In BusinessEvents, EP Agents can be one of 2 base types: inference agents (for rules and states), and query agents.

Tier n+1: Services. Optional. Existing SOA services may also contribute data to the event processing agents, either as-required or cached in the (event) store to be used and updated as needed. Typically service invocation (which could also be a BPM workflow invocation)  involves a call out to some mechanism like TIBCO BusinessWorks, and possible some TIBCO Adapter.

Tier n+2: Persistence.  Optional. When the event store [Tier 3] is using high-performance cache there is a need to provide back-up persistence for tasks like archiving, start-up load and shut-down un-load, and standard analytics. You might also have operational data stores you need to query directly (rather than through the services layer [Tier n+1]) to enrich event objects for processing. If you want to visualise all your events over time to identify new patterns you would typically use something like TIBCO Spotfire against this layer, and perhaps S+ Miner to extract some rules to update some Tier 4…n agent with…

The UK’s Register had an interesting white paper advertised as “University Hospitals Bristol gain real-time view of bed availability”. As healthcare is one of the next IT wastelands that could really benefit from event processing (think real-time medical monitoring, patient track and trace, specialist resource real-time optimization, etc), I was disappointed to find  this was an advertorial for… Sharepoint! It’s a bit like TIBCO claiming its Portal product provides the smarts for a real-time dashboard, rather than just being the container for such displays. One hopes that UH Bristol is doing a bit more than just having some web part polling some database with “end-of-day” ward bed reports and then relying on doctors telephone the wards to try and book any available beds…

[For non-UK readers, UK hospitals are usually managed under the National Health Service: like any government body, it is widely considered as heavy on bureaucracy and light on "customer focus", with a common budget-saving mechanism of closing patient wards leading to situations where hospital beds are scarce resources.]