http://esper.codehaus.org/

http://www.infoq.com/cn/news/2007/10/esper

Esper（InfoQ曾在一年前報道其1.0版本的發布消息）是一個事件流處理（Event Stream Processing，ESP）和復雜事件處理（Complex Event Processing，CEP）的系統，它可以監測事件流并當特定事件發生時觸發某些行動——可看作是把數據庫反過來，語句是固定的，而數據流進進出出。事件處理是軟件行業的一個發展趨勢，已有數家大廠商以及許多初創企業加入到該市場中。其常有的應用例子包括系統自動交易、BAM、RFID、高級監測系統、欺詐檢測，甚至直接集成進SOA。InfoQ恰遇Esper的創始人，向他了解了項目的近況，以及最近的基準測試問題。

如Esper開發小組所說，Esper現在是僅有的純Java開源ESP/CEP引擎，由EsperTech公司提供商業支持服務，而這個公司也在維護一個同樣的.Net項目。

BEA得到了Esper授權，將在修改后在加入到六月發布的WebLogic Event Server中。根據多方面的反饋，Thomas跟InfoQ談道：

我相信Esper在BEA的產品中占一席位的事實，在多個方面都有助于Esper的發展。首先，我們所獲得的反饋的聲音對于Esper的改進有很重要的作用。其次，BEA的產品從總體上提高了CEP/ESP技術的知名度，并且因此擴大了市場的共識。第三，這是Esper技術的開放性，可擴展性，適應企業級應用的最好的證明。Esper社區和用戶群都真的為此而感到自豪。

隨著市場空間的擴大，多種實現之間出現的競爭，標準化能給該行業帶來一定的好處。Thomas對CEP語言標準化的潛力和背景作了評價：

CEP社區顯然把CEP和ESP看作是互補的，并且認為其他手段（如貝葉斯網絡或神經網絡）也可應用于CEP的問題。由于存在各種實現技術，各廠商又各執己見，ANSI SQL標準化委員會在擴展SQL基礎上所提供的“行序列的模式匹配”似乎成為最重要的曙光。

對于這個初步的課題當然會有進一步的研究，并且標準化很可能會超出ESP/CEP語言標準化的范圍。

Esper近期最突出的消息是在八月中旬發布了一個性能基準測試工具及公布了性能測試結果：

Esper在雙2GHz CPU的Intel系統測試環境下，處理超過500 000個事件/秒；在VWAP基準測試中在有1000語句的情況下，引擎延時平均小于3微妙（在10us時超過99%的預測準確率）——最高時有70 Mbit/s流量并占用85%的CPU資源。

雖然這個基準測試是基于一個相當簡單的用例，其發表的目的是震動整個行業，因為它提供了完整的工具集來重現測試結果。Esper事件服務器監聽遠端客戶端通過網絡傳送過來的股票市場事件信息。Esper引擎是通過一個滑動的時間窗口或事件窗口，來實時計算輸入信息的成交量加權平均價。當被問及該基準測試的必要性時，Esper回應道：

整個CEP市場已被含糊不清的信息所包圍，每個廠商都在各自的宣傳單上做文章，避開詳細地交待實際性能和延時。在這個領域中還沒有對它們作比較的基準測試。

在這個行業中含糊的性能信息已經受到Progress Apama和其他人的批評。以下是來自于Apama的博客中的抱怨的聲音：

……Skyler處理速度高達200,000條/秒……主要特征：Coral8每秒處理從數千到百萬計的事件……StreamBase性能領先，每秒處理超過1百萬個事件，反應時間接近零……Aleri Labs打破了亞毫秒反應時間的障礙…… Apama自己也說過“一個能處理數千事件每秒的高性能、可伸縮的引擎”這種話。同樣的措詞在BEA也能找到，WebLogic Event Server公告了似乎較差但較為精確的指標“當我們的產品準備好，它將提供50,000復雜事件／秒的處理速度”。

那些測試結果似乎確定了在這個領域里“數十萬”事件每秒是普遍的，毫無例外。同時也正顯示了Esper在特定場景中的性能表現。它同樣給了用戶群有價值的工具來更好地得知實際性能，而不是聽信廠商任意的令人充滿疑惑的宣傳，對有價值的開源軟件普遍懷有的偏見。

Esper小組已經在其wiki上發布了所有運行的詳情，并且已更新了網頁的性能部分和性能最佳實踐部分。另一個基準測試的來源是最近成立的STAC基準測試委員會，該委員會的目的是為技術的交易而提供由客戶推動的基準測試標準。

請點擊這里獲取InfoQ之前有關Esper和CEP背景的相關報道：http://infoq.com/esper。

查看英文原文：Catching up with Esper: Event Stream Processing Framework

?

Event Processing with Esper and NEsper

Esper is a component for complex event processing (CEP), available for Java as Esper, and for .NET as NEsper.

Esper and NEsper enable rapid development of applications that process large volumes of incoming messages or events. Esper and NEsper filter and analyze events in various ways, and respond to conditions of interest in real-time.

Technology Introduction

Complex event processing (CEP) delivers high-speed processing of many events across all the layers of an organization, identifying the most meaningful events within the event cloud, analyzing their impact, and taking subsequent action in real time (source:Wikipedia).

Esper offers a Domain Specific Language (DSL) for processing events. The Event Processing Language (EPL) is a declarative language for dealing with high frequency time-based event data.

Some typical examples of applications are:

• Business process management and automation (process monitoring, BAM, reporting exceptions, operational intelligence)
• Finance (algorithmic trading, fraud detection, risk management)
• Network and application monitoring (intrusion detection, SLA monitoring)
• Sensor network applications (RFID reading, scheduling and control of fabrication lines, air traffic)

?

?

Quick Start

This quick start guide provides step-by-step instructions for creating a first event-driven application using Esper.

Installation

Esper is easy to install and run: The first step is to download and unpack the distribution zip or tar file. Provided you have a Java VM installed, you may then run an example as described in the online documentation

Esper consists of a jar file named "esper-version.jar" that can be found in the root directory of the distribution.

Dependent libraries to Esper are in the "lib" folder.

Esper includes several examples and a benchmark kit that are documented in the reference manual. These can be run from the command line. Esper does not include a GUI or a server middleware itself, other then the benchmark client and server components.

Creating a Java Event Class

Java classes are a good choice for representing events, however Map-based or XML event representations can also be good choices depending on your architectural requirements.

A sample Java class that represents an order event is shown below. A simple plain-old Java class that provides getter-methods for access to event properties works best:

package org.myapp.event;public class OrderEvent {private String itemName;private double price;public OrderEvent(String itemName, double price) {this.itemName = itemName;this.price = price;}public String getItemName() {return itemName;}public double getPrice() {return price;} }

Creating a Statement

A statement is a continuous query registered with an Esper engine instance that provides results to listeners as new data arrives, in real-time, or by demand via the iterator (pull) API.

The next code snippet obtains an engine instance and registers a continuous query. The query returns the average price over all OrderEvent events that arrived in the last 30 seconds:

EPServiceProvider epService = EPServiceProviderManager.getDefaultProvider(); String expression = "select avg(price) from org.myapp.event.OrderEvent.win:time(30 sec)"; EPStatement statement = epService.getEPAdministrator().createEPL(expression);

Adding a Listener

Listeners are invoked by the engine in response to one or more events that change a statement's result set. Listeners implement the UpdateListener interface and act on EventBean instances as the next code snippet outlines:

public class MyListener implements UpdateListener {public void update(EventBean[] newEvents, EventBean[] oldEvents) {EventBean event = newEvents[0];System.out.println("avg=" + event.get("avg(price)"));} }

By attaching the listener to the statement the engine provides the statement's results to the listener:

MyListener listener = new MyListener(); statement.addListener(listener);

Sending events

The runtime API accepts events for processing. As a statement's results change, the engine indicates the new results to listeners right when the events are processed by the engine.

Sending events is straightforward as well:

OrderEvent event = new OrderEvent("shirt", 74.50); epService.getEPRuntime().sendEvent(event);

Configuration

Esper runs out of the box and no configuration is required. However configuration can help make statements more readable and provides the opportunity to plug-in extensions and to configure relational database access.

One useful configuration item specifies Java package names from which to take event classes.

This snippet of using the configuration API makes the Java package of the OrderEvent class known to an engine instance:

Configuration config = new Configuration(); config.addEventTypeAutoAlias("org.myapp.event"); EPServiceProvider epService = EPServiceProviderManager.getDefaultProvider(config);

In order to query the OrderEvent events, we can now remove the package name from the statement:

String epl = "select avg(price) from OrderEvent.win:time(30 sec)";EPStatement statement = epService.getEPAdministrator().createEPL(epl);

?

?

?

?

Tutorial

Introduction

Esper is an Event Stream Processing (ESP) and event correlation engine (CEP, Complex Event Processing). Targeted to real-time Event Driven Architectures (EDA), Esper is capable of triggering custom actions written as Plain Old Java Objects (POJO) when event conditions occur among event streams. It is designed for high-volume event correlation where millions of events coming in would make it impossible to store them all to later query them using classical database architecture.

A tailored Event Processing Language (EPL) allows expressing rich event conditions, correlation, possibly spanning time windows, thus minimizing the development effort required to set up a system that can react to complex situations.

Esper is a lightweight kernel written in Java which is fully embeddable into any Java process, JEE application server or Java-based Enterprise Service Bus. It enables rapid development of applications that process large volumes of incoming messages or events.

Introduction to event streams and complex events

Information is critical to make wise decisions. This is true in real life but also in computing, and especially critical in several areas, such as finance, fraud detection, business intelligence or battlefield operation. Information flows in from different sources in the form of messages or events, giving a hint on the state at a given time such as stock price. That said, looking at those discrete events is most of the time meaningless. A trader needs to look at the stock trend over a period, possibly combined with other information to make the best deal at the right time.

While discrete events when looked one by one might be meaningless, event streams--that is an infinite set of events--considered over a sliding window and further correlated, are highly meaningful, and reacting to them with the minimal latency is critical for effective action and competitive advantage.

Introduction to Esper

Relational databases or message-based systems such as JMS make it really hard to deal with temporal data and real-time queries. Indeed, databases require explicit querying to return meaningful data and are not suited to push data as it changes. JMS systems are stateless and require the developer to implement the temporal and aggregation logic himself. By contrast, the Esper engine provides a higher abstraction and intelligence and can be thought of as a database turned upside-down: instead of storing the data and running queries against stored data, Esper allows applications to store queries and run the data through. Response from the Esper engine is real-time when conditions occur that match user defined queries. The execution model is thus continuous rather than only when a query is submitted.

Such concepts are a key foundation of EDA, and have been under active research in more than the last 10 years. Awareness of the importance of such systems in real-world architectures has started to emerge only recently.

In Esper, a tailored EPL allows registering queries in the engine. A listener class--which is basically a POJO--will then be called by the engine when the EPL condition is matched as events flow in. The EPL enables to express complex matching conditions that include temporal windows, joining of different event streams, as well as filtering, aggregation, and sorting. Esper statements can also be combined together with "followed by" conditions thus deriving complex events from more simple events. Events can be represented as JavaBean classes, legacy Java classes, XML document or java.util.Map, which promotes reuse of existing systems acting as messages publishers.

A trivial yet meaningful example is as follow: assume a trader wants to buy Google stock as soon as the price goes below some floor value-- not when looking at each tick but when the computation is done over a sliding time window--say of 30 seconds. Given a StockTick event bean with a price and symbol property and the EPL "select avg(price) from StockTick.win:time(30 sec) where symbol='GOOG'", a listener POJO would get notified as ticks come in to trigger the buy order.

Developing event-driven applications

Developing event-driven application is not hard using Esper. You may want to roughly follow these steps:

Define the mission of your application by analyzing your business domain and defining the situations to be detected or information to be reported

Define your performance requirements, specifically throughput and latency

Identify where events are coming from

Identify lower level event formats and event content that is applicable to your domain

Design the event relationships leading to complex events

Instrument your sources of events

Design how you want to represent events: as Java classes, as Maps, or as XML events

Define EPL statements for patterns and stream processing

Use the CSV adapter as an event simulation tool, to test situations to be detected, or to generate to load

Test against your performance requirements: throughput and latency in your target environment

With "Instrument your sources of events" we mean to plan for, design and implement hooks in the event source systems so that they can generate the events in the format defined in 4. Instrumentation is roughly placing hooks that don't change the nominal execution flow. There are several techniques for that, from custom code to aspect-oriented technology with in the middle a whole range of component-dependant and framework-dependant technology (servlet filter, proxy objects, decorators, topic-based architecture etc.). This can also be implemented at a more coarse grained level (pipeline derivation in an enterprise service bus or a BPEL/BPM process for example).

Designing event representations

Java classes are a simple, rich and versatile way to represent events in Esper. Java classes offer inheritance and polymorphism via interfaces and super-classes, and can represent a complex business domain via an object graph. Maps and XML are an alternative way of representing events.

Event Stream Analysis

EPL statements derive and aggregate information from one or more streams of events, to join or merge event streams, and to feed results from one event stream to subsequent statements.

EPL is similar to SQL in it's use of the select clause and the where clause. However EPL statements instead of tables use event streams and a concept called views. Similar to tables in an SQL statement, views define the data available for querying and filtering. Views can represent windows over a stream of events. Views can also sort events, derive statistics from event properties, group events or handle unique event property values.

This is a sample EPL statement that computes the average price for the last 30 seconds of stock tick events:

select avg(price) from StockTickEvent.win:time(30 sec)

A sample EPL that returns the average price per symbol for the last 100 stock ticks.

select symbol, avg(price) as averagePricefrom StockTickEvent.win:length(100) group by symbol

This example joins 2 event streams. The first event stream consists of fraud warning events for which we keep the last 30 minutes (1800 seconds). The second stream is withdrawal events for which we consider the last 30 seconds. The streams are joined on account number.

select fraud.accountNumber as accntNum, fraud.warning as warn, withdraw.amount as amount,MAX(fraud.timestamp, withdraw.timestamp) as timestamp, 'withdrawlFraud' as desc from FraudWarningEvent.win:time(30 min) as fraud,WithdrawalEvent.win:time(30 sec) as withdraw where fraud.accountNumber = withdraw.accountNumber

Event Pattern Matching

Event patterns match when an event or multiple events occur that match the pattern's definition. Patterns can also be temporal (time-based). Pattern matching is implemented via state machines.

Pattern expressions can consist of filter expressions combined with pattern operators. Expressions can contain further nested pattern expressions by including the nested expression(s) in round brackets.

There are 5 types of operators:

Operators that control pattern finder creation and termination: every

Logical operators: and, or, not

Temporal operators that operate on event order: -> (followed-by)

Guards are where-conditions that filter out events and cause termination of the pattern finder, such as timer:within

Observers observe time events as well as other events, such as timer:interval, timer:at

A sample pattern that alerts on each IBM stock tick with a price greater then 80 and within the next 60 seconds:

every StockTickEvent(symbol="IBM", price>80) where timer:within(60 seconds)

A sample pattern that alerts every 5 minutes past the hour:

every timer:at(5, *, *, *, *)

A sample pattern that alerts when event A occurs, followed by either event B or event C:

A -> ( B or C )

An event pattern where a property of a following event must match a property from the first event:

every a=EventX -> every b=EventY(objectID=a.objectID)

Combining Patterns Matching with Event Stream Analysis

Patterns match when a sequence (or absence) of events is detected. Pattern match results are available for further analysis and processing.

The pattern below detects a situation where a Status event is not followed by another Status event with the same id within 10 seconds. The statement further counts all such occurrences grouped per id.

select a.id, count(*) from pattern [every a=Status -> (timer:interval(10 sec) and not Status(id=a.id) ] group by id

Named windows

A named window is a global data window that can take part in many statement queries, and that can be selected-from, inserted- into and deleted-from by multiple statements. Named windows are similar to a table in a relational database system.

One can create a named window for example as follows:

create window AlertNamedWindow as (origin string, priority string, alarmNumber long)

One can trigger a select, update or delete when an event arrives. Here we show a select that simply counts the number of rows:

on TriggerEvent select count(*) from AlertNamedWindow

Named windows can as well be queried with fire-and-forget queries through the API and inward-facing JDBC driver.

Match-Recognize Pattern Matching

A match-recognize pattern is a regular-expression-based pattern-matching syntax proposed for inclusion in SQL standards.

The below query is a sample match-recognize pattern. It detects a pattern that may be present in the events held by the named window as declared above. It looks for two immediately-followed events, i.e. with no events in-between for the same origin. The first of the two events must have high priority and the second of the two events must have medium priority.

select * from AlertNamedWindowmatch_recognize (partition by originmeasures a1.origin as origin, a1.alarmNumber as alarmNumber1, a2.alarmNumber as alarmNumber2pattern (a1 a2)definea1 as a1.priority = 'high',a2 as a2.priority = 'medium' )

Variables

A variable is a scalar, object or event value that is available for use in all statements including patterns. Variables can be used in an expression anywhere in EPL.

?

?

總結

以上是生活随笔為你收集整理的Esper的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。