Situation Awareness in CEP…
Posted by Paul Vincent
Tim Bass recently challenged [*1] the assertion that Complex Event Processing provides “situation awareness” and quoted a Wikipedia article covering this
topic from the usual human element. The article interestingly includes a generalized diagram of situation awareness [reference Endsley, M. R. (1995b). Toward a theory of situation awareness in dynamic systems. Human Factors 37(1), 32-64] that maps quite nicely to CEP (although it is obviously targeted at a more human-oriented system).
So lets review the Wikipedia definition of Situation Awareness (SA) and compare to CEP:
The most common theoretical framework of SA is provided by Dr. Mica Endsley (1995b). Endsley’s model illustrates three stages or steps of SA formation: perception, comprehension, and projection.
Perception (Level 1 SA): The first step in achieving SA is to perceive the status, attributes, and dynamics of relevant elements in the environment. Thus, Level 1 SA, the most basic level of SA, involves the processes of monitoring, cue detection, and simple recognition, which lead to an awareness of multiple situational elements (objects, events, people, systems, environmental factors) and their current states (locations, conditions, modes, actions).
- CEP comparison: this is about monitoring events and the states of entities - basic CEP functionality. For example, TIBCO BusinessEvents uses events from various channels that can be monitored, and a state model / engine to represent any spatial or temporal aspect of the elements being monitored.
Comprehension (Level 2 SA): The next step in SA formation involves a synthesis of disjointed Level 1 SA elements through the processes of pattern recognition, interpretation, and evaluation. Level 2 SA requires integrating this information to understand how it will impact upon the individual’s goals and objectives. This includes developing a comprehensive picture of the world, or of that portion of the world of concern to the individual.
- CEP comparison: this is about recognizing patterns of events and using reasoning tools (such as inference rules) to interpret and evaluate these patterns. Goals and objectives are usually represented as end states in a CEP system. For example, TIBCO BusinessEvents uses rules and (optionally continuous) queries to define patterns, and an inference engine and state model for reasoning to goals and objectives.
Projection (Level 3 SA): The third and highest level of SA involves the ability to project the future actions of the elements in the environment. Level 3 SA is achieved through knowledge of the status and dynamics of the elements and comprehension of the situation (Levels 1 and 2 SA), and then extrapolating this information forward in time to determine how it will affect future states of the operational environment.
- CEP comparison: this is about associating certain complex events with predictions about likely future states and reporting or making decisions based on them. For example, TIBCO BusinessEvents uses rules and managed decisions to take certain actions based on the detection of these complex events.
SA also involves both a temporal and a spatial component. Time is an important concept in SA, as SA is a dynamic construct, changing at a tempo dictated by the actions of individuals, task characteristics, and the surrounding environment. As new inputs enter the system, the individual incorporates them into this mental representation, making changes as necessary in plans and actions in order to achieve the desired goals. SA also involves spatial knowledge about the activities and events occurring in a specific location of interest to the individual. Thus, the concept of SA includes perception, comprehension, and projection of situational information, as well as temporal and spatial components.
- CEP comparison: this is about the importance of time and location. Indeed, you could substitute CEP for SA and still have a meaningful paragraph above. For example, TIBCO BusinessEvents uses time events and rules as well as states to manage the temporal aspect of reasoning.
* * *
To conclude, it seems there is a close association between CEP and situational awareness. Although automated tools may never have the perspective or detailed semantics of human awareness, CEP technologies can certainly provide a lot of the functionality required to assist operations. Indeed there are a few BusinessEvents applications that do just this - provide oversight of operational systems such as BPM and SOA. Are all possible situations detected? Probably not; however many are and that provides good business value to our customers.
Notes:
[1] Tim clarified in the comments that he was claiming that “CEP software”, not the “concept of CEP”, didn’t “much” support Situation Awareness. As there is no common-use checklist as to what “support for SA” implies for software, we’ll just have to agree to differ on that point!
Complex Event Processing (CEP), Queries, Rules, Situation Awareness, State | 25 February 2009 |
21 Comments
Other Links to this Post
-
CEP The Concept Is Not CEP The Software | Cyberstrategics Complex Event Processing Blog — February 25, 2009 @ 10:43
RSS feed for comments on this post. TrackBack URI
By Peter Lin, February 25, 2009 @ 07:08
I’m not an expert in situational awareness, so my comments are probably skewed or wrong.
From an expert systems perspective, for a system to “perceive” in a “human-like” manner, a system has to have a knowledge base. For me, this means a model of the domain, some pertinent facts a human expert would have for the domain and inference rules that reason over the first two.
Event stream processing and the examples I’ve seen so far from CEP products look nothing like an expert system. One key aspect of building a knowledge base is the format one uses to capture the knowledge. It takes years of experience to learn how to do that effectively. Even after 8 years of research and study, I still find it very challenging.
I agree that one could substitute SA with CEP and have a meaningful definition, but I feel it borders on hyperbole to claim CEP supports SA. It would be better to say CEP supports business decision automation, which doesn’t require an expert system or a knowledge base.
By vincent, February 25, 2009 @ 09:13
Hi Peter:
“Event stream processing and the examples I’ve seen so far from CEP products look nothing like an expert system”.
Speaking as one reasonably versed in both (er, too many years to quote for expert systems), my answer is that ESP solutions are of course a subset of CEP, and some CEP solutions like TIBCO’s are in fact very expert system like:
- knowledgebase of rules, queries and possible states
- facts defined as objects and transient events.
Like modern BREs, TIBCO BE does not have an out-of-the-box explanation facility (although one can be added through rule extensions, profiler advisory events etc), so it cannot fulfil the basic definition of an expert system (in explaining its behavior). Also, “learning” is, like most expert systems were, a manual maintenance process (although event-based analytics can be used). And the knowledgebase may not be defined from “known human authorities”, so BE applications may just be “knowledge-based systems”.
Situation awareness in automated systems is more about being able to track events and reason over them over time - and this is CEP functionality. Hence we consider SA a close match, a thesis supported by the CEP applications we tend to see…
Cheers
By Tim Bass, February 25, 2009 @ 10:12
Hi Paul,
I did not assert that CEP does not provide SA. I said that the current tools on the market that call themselves CEP do not provide SA.
There is a very big difference, don’t you think.
I agree with you that CEP and SA are a very close match, what I disagree with is that the current softare on the market that calls itself “CEP software” is useful in most SA applications.
Yours sincerely, Tim
By Tim Bass, February 25, 2009 @ 10:17
Hi Again Paul,
The trouble with your posts is that do not not clearly distingush between “CEP the concept,” and “CEP the software”.
I was the person who was the leader, before you came on the CEP scene, to express how CEP is a means to achieve SA. I also said, many years ago, that SA requires much more sophisticated methods and algorithms than what software sellers are selling as CEP.
TIBCO has never disputed that, BTW. Few, if any, experts at TIBCO would claim that TIBCO’s BE as evolved to the point of being “SA capable”. I am very happy to elaborate on my blog why; however most of that elaboration would be repeating what I said in my first keynote on this topic in 2006.
Yours sincerely, Tim
By vincent, February 25, 2009 @ 11:30
Hi Tim,
Lets see if I can address your points…
The “Event Driven CEP Systems”, as you call them, do not provide much situational awareness.
Actually I didn’t make any claim to identify such systems, even as software :); however it is an interesting argument that “event driven CEP systems” are somehow aloof from the concepts of CEP. I *think* this is due to your unique ideas about how none of the current CEP technologies satisfy the criterion of being “CEP”…
The trouble with your posts is that do not not clearly distingush between “CEP the concept,” and “CEP the software”..
Generally “XYZ software implements XYZ concepts” (replace XYZ with SOA, BPM, BRE, CEP etc). You may argue that specific software S that claims to do CEP does not do a good job because it excludes the capability to do Y type of event pattern, but then we have to go into specifics. Hopefully the post above argued that feature X in BE, for example, maps to requirement R for Situation Awareness. A better counterargument, by the way, would be to identify feature Y, not in current CEP tools, which is proven to better solve requirement R…
I was the person … to express how CEP is a means to achieve SA. I also said, many years ago, that SA requires much more sophisticated methods and algorithms than what software sellers are selling as CEP.
To avoid any accusations of false modesty against Tim, due credit indeed goes to him for describing some of the fundamentals on CEP (e.g. see the very first posts on this blog or his own blog). One could argue that SA can never be fully automated, or that it requires AI capabilities. Nonetheless, CEP does provide situation awareness capabilities - its just that it might not be the full capabilities some domains might want. Again, this is true for any domain: SOA, BPM etc never provide *all* the features one might want in their domain, but nonetheless do provide useful SOA and BPM etc capabilities. Likewise for CEP and SA.
keep your eyes on the blog-o-rinkbecause it looks like there may be a lot of sparks
I doubt it - these sorts of discussions are interesting but ultimately somewhat moot compared to the real-world problems being solved by CEP vendors today…
Cheers
By Peter Lin, February 25, 2009 @ 14:27
I’ve only played with Business Events a little, so my perspective is probably wrong and totally inaccurate. For me, when I think of expert system shells, I think of Clips, Haley and JESS. A shell which provides the functionality to build, create and manage knowledge base. It also means the system is able to create new facts, rules and object definitions at runtime from the shell or through API.
How does BE compare to CLIPS, Haley, Art and JESS? I ask because my experience and understanding of Tibco BE is superficial at best.
thanks
peter
By vincent, February 25, 2009 @ 16:08
Hi Peter,
I would describe Clips, Haley and JESS as production rule engines, not expert system shells of the MYCIN variety. One of my favorite references is Expert Systens: AI in Business by Harmon and King - see http://www.amazon.com/Expert-Systems-Artificial-Intelligence-Business/dp/0471808245 …
Note that the main difference between the BE rule engine and these other production rule engines is that facts in BE are primarily events and concept instances (aka event objects). As an OO system it makes no sense to assert and retract logic-type facts (which are more akin to Prolog type thinking). In this way BE is more like the BREs from FIC and Ilog. Of course, there are a few other differences too
Cheers
By Peter Lin, February 25, 2009 @ 19:33
Ahh ok. I see where you’re coming from. Although I’ve heard of Mycin, I’ve never used it. I’ve mainly used JESS (Java expert system shell), CLIPS and my own engine Jamocha.
From my experience developing Jamocha, a rule engine that takes a strong OO approach is not well suited to machine learning due to the cost of generating and compiling java classes at runtime.
I’m curious, does Tibco BE use an interpreted design like CLIPS and JESS or a strong OO design, which compiles the rules to java objects?
The last time I read the BE documentation, I couldn’t see any details about the low level design of the rule engine.
peter
By Tim Bass, February 25, 2009 @ 21:37
Hi Paul,
You may consider being a big smug on this topic as “cute”; but it might me more constructive to actually be more truthful.
You say, “I doubt it - these sorts of discussions are interesting but ultimately somewhat moot compared to the real-world problems being solved by CEP vendors today…”
When was the last time a TIBCO customer announced a “real-world problem being solved by a CEP vendor”?
Give us a break
The “CEP vendors” are doing very little to solve customer problems because “CEP vendors” are selling little more than rule-based engines which most customers can solved with PERL scripts, or open source rules engines like Drools.
Customers are solving real-world problems with a lot of things these days, but CEP software is not one of them, Paul.
Yours faithfully, Tim
By vincent, February 26, 2009 @ 01:06
Hi Peter,
Interpretation vs compilation of rules affects dynamic rule (as opposed to dynamic fact) useage. For BE we use compiled rules by the way; however we utilize a multi-agent architecture so you could bring online a new compiled rulebase with new rules, sync memory, and redeploy… this is why AI tools that are about knowledge / rule *creation* tend to use tools like Prolog…
Cheers
By vincent, February 26, 2009 @ 01:16
Hi Tim, please try and keep your comments at a professional level and avoid insults (?) like “big smug” etc. Thanks!
When was the last time a TIBCO customer announced a “real-world problem being solved by a CEP vendor”?
For customer statements on CEP, using CEP software like TIBCO BusinessEvents, see previous blog entries and TUCON announcements. We’ve had a few interesting projects in the last few months and we’ll have to see what and when we can release as case studies. For a Situation Awareness case study from last year consider the Airline Operations example mentioned in this past post on TUCON 2008 …
The “CEP vendors” are doing very little to solve customer problems because “CEP vendors” are selling little more than rule-based engines
Well, you could consider some of the pattern matching mechanisms in, for example, pure ESP tools as “rules”: but I’m not sure all CEP tools qualify as “rule engines” per the standard use of the term. See this past post on the differences between BREs and rule-driven CEP - Part 1 and - Part 2 for a discussion on the differences between CEP tools and rule engines per se. Also, I’m surprised you believe CEP vendors are “selling” while doing “very little to solve customer problems” - it would be interesting to see your data (if any) about any CEP tools being mis-sold, which this statement implies.
…which most customers can solved with PERL scripts, or open source rules engines like Drools.
Notwithstanding the fact that PERL or rule engines are not the same as event processing platforms doing CEP, this statement would seem to be contradicted by the large year-on-year increase in the CEP software tools market. As ultimately all software compiles down to machine code, and these 2 tools can be extended, the statement may be intrinsically correct - but not very useful (you don’t see many people writing databases or BPM tools in PERL either).
Note: Interestingly, we’ve had at least one good BusinessEvents application come from a user “experimenting” with open source rules technology and determining that their requirements need to be solved by BusinessEvents.
Cheers
By Peter Lin, February 26, 2009 @ 10:43
Thanks for taking time to respond Paul. A few years back Said Tabet and I looked at multi-agent techniques. Although they are powerful, they are still limited for dynamic environments.
Say I start up 4 agents with the object model and 4 different rulesets. At T2 some time later, I start up 2 agents with a new model, the communication between the first 4 agents and the second set run into issues. Communication from the first 4 to the second 2 are fine, but communication from the 2 new agents to the 4 initial agents may run into issues if they don’t have those object definitions.
I agree that’s why prolog style systems are used for AI and machine learning.
thanks.
peter
By vincent, February 26, 2009 @ 14:13
Hi Peter: you are correct, multi-agent systems are generally non-trivial. BusinessEvents is helped by the fact that TIBCO is probably the most experienced company in certain types of middleware - which distributed agents / blackboard type systems rely on.
On your example, note that typically in BE:
- each message type is typically handled by at most one agent type.
- the agent will consume (filter / initial transforms and processing) the event and share the event information (via the persistence agents / data grid)
- shared event information is then used by any other / multiple rule (and query) agents.
Clearly synchronization (/verification of duplicate rule conditions across agents) needs to be taken care of: but using the data grid as the blackboard seems to work well. In other words, you start with the shared fact model / view when partitioning your rules into separate agents.
In your example: if you add new fact (object) types in your 2 new agents that are not known (or used) by the 4 existing agents, then this is not a big deal (*their* rules will obviously ignore any data they don’t reference). Or if you have added more classification information via subclasses, the rules operating on the superclass will still operate. But you could then selectively update the 4 agents with new / updated rules to use the new information as you need…
FYI the Agent Metamodel and Profile group in OMG under Jim Odell is doing some interesting work on agent organization…
Back on topic: the distributed data grid acts as the application fact base, including history, for “situation representation” and, through rule-based interpretation, “situation awareness”. Conceptually and in software…
Cheers
By Peter Lin, February 27, 2009 @ 06:55
I agree that many business cases the system is only dealing with new facts and not new object definitions or rules at runtime. From my experience, partitioning the rules into smaller manageable sets works in some cases, but not all. Ideally, a large ruleset should be divided into small manageable sets regardless of whether agents are used.
System that compile rules down to pure java code run into JVM PermGen memory issues. Having seen the PermGen issue crop up in rule engines, servlet containers and ejb containers, it’s an important consideration for enterprise applications. For example, some rule engines can’t load 20K rules without increasing the PermGen setting to 512Mb. Loading 100K rules would require using 64bit JVM from Sun or IBM.
Does BE have the option of running in interpreted mode?
thanks
peter
By vincent, February 27, 2009 @ 07:48
Hi Peter:
“partitioning the rules into smaller manageable sets works in some cases, but not all. Ideally, a large ruleset should be divided into small manageable sets regardless of whether agents are used.”
Yes, normal rule development best practices apply…
“System that compile rules down to pure java code run into JVM PermGen memory issues. Having seen the PermGen issue crop up in rule engines, servlet containers and ejb containers, it’s an important consideration for enterprise applications.”
Note that decisions (that in BE map from its DecisionManager to rulefunctions (aka procedures / sequential mode code) are optimized (for exclusive rulesets) so that you can get say 50K row decision tables into a small amount of code. And of course, distibuting across multiple agents helps here too
“Does BE have the option of running in interpreted mode?”
Not in the sense of a “rule interpreter”, no.
Full documentation is available to signed-up partners at download.tibco.com, of course.
Cheers
By Peter Lin, February 27, 2009 @ 17:40
Thanks for taking time to explain. I have an older version of BE. i will have to go back and re-read the documentation.
peter
By Neil Raden, March 3, 2009 @ 08:33
Paul,
This is all very interesting, but what’s the point? It reminds me of the early days of relational databases when DBMS’ struggled to meet all of Codd’s “rules.” We ended up with 20 years of stagnant, stale lookalike database systems and have only recently broken out of it.
Like the late, great software visionary Mao Zadong said, “I don’t care if a cat has a tail or not, so long as it catches mice.”
Keep focusing on solving your customers problems, and don’t be distracted by this pedantic nonsense.
-NR
By vincent, March 4, 2009 @ 02:34
Thanks Neil -
“We ended up with 20 years of stagnant, stale lookalike database systems and have only recently broken out of it. “
I’m pretty sure the CEP tool world is not being constrained by the SA classification from Wikipedia / Dr Endsley, nor indeed the EPTS definitions. Indeed one could argue that with in-memory data stores and dynamic time-based queries, CEP is part of the “break-out” from the constraints of the DBMS world on IT.
“Keep focusing on solving your customers problems, and don’t be distracted by this pedantic nonsense.”
I’m pretty sure we haven’t lost any focus on our customers! Nontheless, we find that CEP applications are very often used for operational intelligence roles that overlap with situation awareness concepts, making situation awareness a reasonable topic…
Some other interesting links on Situation Awareness:
- Stephen Few on SA Dashboards from his Visual BI blog
- Situation Awareness 101 for management from CIO.com
Cheers
By Mica Endsley, February 17, 2010 @ 13:57
All,
I just ran across this interesting conversation and thought I’d chime in. I am not familiar with your specific CEP products. It sounds like they are indeed trying to provide situation awareness to their users. If users of the systems are complaining that they do, it most likely because either the information they really need is not there, or it is being communicated and presented in a manner that is sub-optimal. To overcome these difficulties, product vendors may want to check out “Designing for Situation Awareness” (Endsley, Bolte & Jones, 2003), available on Amazon.com. This book provides a 3 stage process for helping to ensure that your systems are effectively delivering this much needed commodity. It also includes 50 design principles, based on over 20 years of research findings, on how to present the information effectively. It was written for exactly this type of situation, to translate the research base into practical solutions that engineers need.
On the other points, unfortunately expert systems (or automation or AI) does not necessarily enhance SA. In fact, the research shows it can subtly undermine in some cases. The idea of using advanced algorithms (of any type) to help provide comprehension (level 2 SA) and projection (level 3 SA) is definitely a good one. However much care needs to be taken to make sure that many pitfalls are avoided and that the information is really being integrated effectively. This also is covered in the book for those who are interested
cheers,
Mica Endsley
By Paul Vincent, February 17, 2010 @ 14:13
Thanks Mica - sounds like an interesting book reference - will check it out! Cheers