The use of Complex Event Processor to monitor the status of a Smart City
CitiSim is an open smart city platform that incorporates a series of advanced services that provide the platform with extra functionalities such as 3D visualization, augmented reality, advanced data analysis, reporting services, energy efficiency services, emergency services, Complex Event Processor, etc. This initial set of advanced services can be extended thanks to the interoperability layer that allows the incorporation of new services.
In this entry, we will focus on the service Complex event Processor (CEP) , as an advanced monitoring system of all devices/sensors and services included in the platform. Thanks to the CEP, all the information transmitted by the platform can be monitored with a single system. Next we are going to explain how the CEP Component works.
Common Architecture of a CEP Component
The goal of Complex Event Processor (CEP) system is to process streams of event data in identifying some pattern within those streams, using techniques such as detection of relationships between multiple events, event correlation, event hierarchies, and other complex aspects.
In this context, a Rule Engine (RE) is usually referred to as a component that applies simple rules to an incoming stream of data and automatically executes the predefined consequences. A Complex Event Processor (CEP) would take into account not only the data just received, but also the historical data (events that happened in the past and are somehow relevant). Figure 4 shows an overview of the structure of a CEP as it is currently understood.
Figure 1: overview of the structure of a CEP
Dynamic Data usually consists of the recent input data and possibly some useful intermediate derived data. It is stored in memory in order to apply the various rules on it. There is usually a window time frame to determine for how long the historical data must be kept.
Static Data is some other data that describes the business context and is generally unmodified during the execution. Some examples could be data describing the infrastructure of an organization, unmodifiable parameters that depend on the country’s laws, etc. Rules are expressions, usually with an if-then structure, written in some formal language. They are permanently being checked, in order to at some point determine the existence of a new event. The Rule Engine is the core component. Its task is determining when the rules match their condition and acting accordingly. Optimizing the amount of computations needed to achieve this task is usually not easy.
A Clock is added to the diagram to stress the fact that some rules use time frames in their conditions, which means that the current time is also relevant for rule evaluation. Useful Intermediate Events Detected are auxiliary pieces of data that the CEP optionally stores temporarily (dynamic data) which are relevant in order to work with the rules. Data Stream is the input to the CEP. It often needs a pre-process in order to convert it to the CEP internal data model. Usually a timestamp is associated to it. Detected Events is the output of the CEP. It is usually a series of timestamped data structures that describe some kind of event that is relevant in the business context. Other components will take it as input.
Rule Administration Tool is an optional external component that establishes the rules to be used by the CEP. It usually acts before the execution begins, though sometimes it is possible to update the rules on the fly, and therefore the behavior of the CEP evolves dynamically by applying new rules to the current historical dynamic data. Some well-known examples of CEP-related software are: Drools, Esper, WSO2 CEP, SQLStream, JBoss BRMS ,Oracle Event Processing
CITISIM CEP Service
The objective of the CEP service is to provide a development environment for event processing. This is a method of tracking and analysing streams of information about things that happen (events) and deriving a conclusion from them. Complex event processing, or CEP, is event processing that combines data from multiple sources to infer events or patterns that suggest more complicated circumstances. In CitiSim, the input of these events is the sensor data that comes through the Message Broker and other information provided by other sources.
These events may be happening across the layers of CitiSim (sensor layer, pre-processed data, etc.). An event may also be defined as a “change of state”, when a measurement exceeds a predefined threshold of time, temperature or another parameter. CEP will provide to CitiSim a new way to analyze patterns in real-time and help the business side communicate better with IT and service layers.
On the other hand, a top layer provides REST services for other IoT clients or the analysis layer. Figure 2 shows how the information flows from the bottom layer where the sensors generate events to the top layer where this information is consumed.
Figure 2. The flow of information from the sensor layer to the consumer layer
The CEP component is subscribed to a broker data where sensors, smart services and other CitiSim services publish data. When a new data is published in the broker (Tansport Layer), the CEP component receives the new data (CEP layer) and analyse it to check if any event occurs. If an event occurs it publish this event through the broker (Transport layer). If any component is interested in these CEP events, it should subscribe to a specific type of event to receives the new alert (Consumer) and process it internally.
An example of a full process could be:
- A Thermometer sensor publish a new value: 60ºC into the broker
- The CEP component receives the value and check internally if the rule “danger of fire” is accomplish.
- The CEP component sends the event detected to the Broker
- The Smart city website receive the event and show the event as an alert in the 3D web client, moreover it sends an email to an emergency account
The previous example is a simple one, but the CEP service is designed to implement more complex rules. Several verifications from different data sources and locations can be included in a single rule, for instance:
- The average temperature during the last 5 minutes of a specific room is over 50ºC
- The temperature of a room is 15ºC higher of the temperature of the adjacent rooms.
- The heating is off
- The temperature outside the building is less than 35ºC
- The temperature has risen 10 degrees in the last hour
Displaying the alerts and notifications generated by the CEP service
The CEP service does not provide any visualization by itself, but we can see the alerts and notifications produced by the CEP using the 3d Viewer of Cititim that visualize the rules activated using different visual resources (icons, popups, message windows and so on)
When a new reading of a sensor occurs, or a new earthquake has been detected, the corresponding message is sent to the CEP (Complex Event Processing). In the CEP there are implemented a set of rules that evaluate these data and generate a response/notification message if needed.
The CEP can generate two kind of alert messages: ‘warning’, when there is not an immediate risk for persons; and ‘danger’, when there is an immediate risk for persons.
A ‘warning’ example could be a rain accumulation between 16mm and 40mm in the last hour (1 mm = 1 L/m²).
A ‘danger’ example could be a rain accumulation greater than 250mm in the last hour (Figure 3: Alert popup window (type danger)).
Depending on whether the alert is a ‘warning’ or a ‘danger’, the sensor icon animation goes from green to orange or red respectively, and at the same time a pop-up alert window with the corresponding text will be displayed.
Following the previous example, if 280mm rain accumulation in the last hour is detected, the next alert of type ‘danger’ will be displayed:
Figure 3: Alert popup window (type danger)
As can be seen in the previous capture, two actions have taken place: on the one hand the animation under the corresponding icon has become red, indicating that the last reading has produced a ‘danger’ type alert message; and on the other hand, a pop-up window with a descriptive text has been opened to alert the user about the danger of this alert
If several alerts were generated at same time, these pop-up windows would be shown as cascading windows, where the most recent one would be above the rest. In this case, the user could click on older one to push it to top:
Figure 4: Several Alert popup windows showed in cascade
These popup windows will be closed automatically after few seconds, if the user does not close them by clicking on the ‘Ok’ button.
Author: Ismael Torres
Company: Prodevelop