東華大學圖書館 |

State Management for Efficient Event Pattern Detection.

紀錄類型:	書目-電子資源 : Monograph/item
正題名/作者:	State Management for Efficient Event Pattern Detection./
作者:	Zha, Bo.
面頁冊數:	1 online resource (163 pages)
附註:	Source: Dissertations Abstracts International, Volume: 84-03, Section: A.
Contained By:	Dissertations Abstracts International84-03A.
標題:	Sensitivity analysis. -
電子資源:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=29280678click for full text (PQDT)
ISBN:	9798845431646

State Management for Efficient Event Pattern Detection.
Zha, Bo.

State Management for Efficient Event Pattern Detection. - 1 online resource (163 pages)

Source: Dissertations Abstracts International, Volume: 84-03, Section: A.

Thesis (Ph.D.)--Humboldt Universitaet zu Berlin (Germany), 2022.

Includes bibliographical references

S ystems for event stream processing continuously evaluate queries over high-velocity event streams to detect user-specified patterns with low latency. Since the patterns become less important over time, it is crucial to detect them as quickly as possible. However, low-latency pattern detection is challenging, because query processing is stateful and the set of partial matches maintained by common algorithms for query evaluation grows exponentially in the size of the processed event data.Handling this state during query evaluation is further complicated by the dynamicity of streams and the potential need to integrate remote data. First, heterogeneous event sources may yield streams with dynamic, unpredictable input rates and data distributions, and hence, query selectivities. During short peak times, exhaustive processing is no longer reasonable, or even infeasible, and systems shall resort to best-effort query evaluation: They shall strive for optimal result quality while staying within a latency bound. Second, some queries require access to remote data from external sources to determine whether a specific event is part of a pattern. Such dependencies are problematic, since waiting for remote data to be fetched interrupts the evaluation of queries over the streams. Yet, without event selection based on remote data, the growth of the number of partial matches maintained during query evaluation is amplified.In this dissertation, we present strategies for optimised state management in event pattern detection. First, we enable best-effort query evaluation with load shedding that discards both input events and partial matches. We carefully select the partial matches and input events to drop in order to satisfy a latency bound while striving for a minimal loss in result quality. Second, to efficiently integrate remote data, we decouple the fetching of remote data from its use in query evaluation through a caching mechanism. Based thereon, we hide the transmission latency of remote data by prefetching data based on anticipated use and by lazy evaluation that postpones the event selection based on remote data to avoid interruptions. A cost model is proposed to determine when to fetch which remote data items and how long to keep them in the cache.We evaluated our techniques for load shedding and the integration of remote data with queries over synthetic and real-world event data. We show that our load shedding technique significantly improves the recall of pattern detection over baseline approaches under different latency bounds, while our technique for remote data integration can drastically reduce the detection latency. Moreover, we report on a case study on smart grid management where event stream processing is employed to alleviate stress on the grid during peak demand hours. This is achieved by monitoring how consumers alter their consumption as requested by a utility, and by predicting potential non-compliance in real-time. Our simulation results show that load shedding and remote data integration enable the design of a system that scales up to 1.6 million residents.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2023

Mode of access: World Wide Web

ISBN: 9798845431646Subjects--Topical Terms:

3560752
Sensitivity analysis.
Index Terms--Genre/Form:

542853
Electronic books.

State Management for Efficient Event Pattern Detection.
LDR:07532nmm a2200337K 4500 001 2361789
005 20231027102313.5
006 m o d
007 cr mn ---uuuuu
008 241011s2022 xx obm 000 0 eng d
020 $a 9798845431646
035 $a (MiAaPQ)AAI29280678
035 $a (MiAaPQ)Humboldt_1845225352
035 $a AAI29280678
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Zha, Bo. $3 3702472
245 1 0 $a State Management for Efficient Event Pattern Detection.
264 0 $c 2022
300 $a 1 online resource (163 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 84-03, Section: A.
500 $a Advisor: Weidlicho, Matthias;Rothermel, Kurt;Koldehofe, Boris.
502 $a Thesis (Ph.D.)--Humboldt Universitaet zu Berlin (Germany), 2022.
504 $a Includes bibliographical references
520 $a S ystems for event stream processing continuously evaluate queries over high-velocity event streams to detect user-specified patterns with low latency. Since the patterns become less important over time, it is crucial to detect them as quickly as possible. However, low-latency pattern detection is challenging, because query processing is stateful and the set of partial matches maintained by common algorithms for query evaluation grows exponentially in the size of the processed event data.Handling this state during query evaluation is further complicated by the dynamicity of streams and the potential need to integrate remote data. First, heterogeneous event sources may yield streams with dynamic, unpredictable input rates and data distributions, and hence, query selectivities. During short peak times, exhaustive processing is no longer reasonable, or even infeasible, and systems shall resort to best-effort query evaluation: They shall strive for optimal result quality while staying within a latency bound. Second, some queries require access to remote data from external sources to determine whether a specific event is part of a pattern. Such dependencies are problematic, since waiting for remote data to be fetched interrupts the evaluation of queries over the streams. Yet, without event selection based on remote data, the growth of the number of partial matches maintained during query evaluation is amplified.In this dissertation, we present strategies for optimised state management in event pattern detection. First, we enable best-effort query evaluation with load shedding that discards both input events and partial matches. We carefully select the partial matches and input events to drop in order to satisfy a latency bound while striving for a minimal loss in result quality. Second, to efficiently integrate remote data, we decouple the fetching of remote data from its use in query evaluation through a caching mechanism. Based thereon, we hide the transmission latency of remote data by prefetching data based on anticipated use and by lazy evaluation that postpones the event selection based on remote data to avoid interruptions. A cost model is proposed to determine when to fetch which remote data items and how long to keep them in the cache.We evaluated our techniques for load shedding and the integration of remote data with queries over synthetic and real-world event data. We show that our load shedding technique significantly improves the recall of pattern detection over baseline approaches under different latency bounds, while our technique for remote data integration can drastically reduce the detection latency. Moreover, we report on a case study on smart grid management where event stream processing is employed to alleviate stress on the grid during peak demand hours. This is achieved by monitoring how consumers alter their consumption as requested by a utility, and by predicting potential non-compliance in real-time. Our simulation results show that load shedding and remote data integration enable the design of a system that scales up to 1.6 million residents.
520 $a Systeme zur Event Stream Processing uberwachen kontinuierliche Datenstrome, um nutzerdefinierte Patterns zu detektieren. Diese Patterns sollen schnellstmoglich erkannt werden, um die Latenz einer Reaktion auf ein Pattern zu minimieren. Dies ist sehr herausfordernd, weil die Verarbeitung der entsprechenden Queries zustandsbasiert erfolgt und die Grose des Zustands durch partielle Patternubereinstimmungen exponentiell wachst. Die Handhabung dieser Zustande wahrend der Evaluation einer Query wird zusatzlich durch die moglicherweise existierende Notwendigkeit Remote-Daten fur die Evaluation heranzuziehen verkompliziert. Als weiteres Hindernis sorgen heterogene Datenstrom-Quellen fur sich dynamisch andernde Datenstrome mit unvorhersehbaren Inputraten und Datenverteilungen. Entsprechend variieren die Query Selektivitaten. Wahrend kurzer Peaks ist eine vollumfassende Query Evaluation nicht praktikabel und teilweise unmoglich. Somit wird eine best-effort Query Evaluation notwendig, welche eine optimale Query Evaluation anstrebt, bei gleichzeitiger Einhaltung von Latenzbedingungen. Desweiteren erfordern einige Queries die Abfrage von Remote-Daten um die Relevanz einzelner Events fur ein Pattern zu bestimmen. Diese Abhangigkeit ist problematisch, weil die Wartezeit, welche beim Zugriff auf Remote-Daten entsteht, die Evaluation der Queries verzogert. Dennoch ist eine solche Uberprufung notwendig, weil sonst die Anzahl der partiellen Patternubereinstimmungen weiter wachsen wurde. In der vorliegenden Disseration werden verschiedene Strategien fur das optimierte Zustandsmanagement wahrend der Event Pattern Detection vorgestellt. Als erstes wird ein Ansatz fur die Best-effort Query Evaluation mittels Load Shedding vorgestellt, welcher sowohl partielle Patternubereinstimmungen, als auch Input Events selektiert. Diese Selektion erfolgt mit den Ziel der Einhaltung einer Latenzgrenze, bei gleichzeitiger Minimierung des Qualitatsverlustes bei der Query Evaluation. Auserdem stellen wir eine Strategie zur Integration von Remote-Daten vor, welche Abfragen jener von der Query-Evaluation durch einen Caching Mechanismus entkoppelt. So konnen Ubertragungslatenzen bei der Abfrage von Remote-Daten mittels Prefetching und Lazy Evaluation Techniken vermieden werden. Mit Hilfe eines Kostenmodells wird bestimmt, welche Remote-Daten wann abgerufen werden sollen und wie lange sie zwischengespeichert werden mussen. Die Evaluation unserer Ansatze fur Load Shedding und die Integration von Remote-Daten erfolgt auf synthetischen und real-world Datensatzen. Dabei wird aufgezeigt, dass die in dieser Arbeit vorgestellten Ansatze eine signifikante Verbesserung gegenuber dem Stand der Technik ist. Beispielsweise kann der Ansatz fur die Remote-Daten Integration die Detektionslatenz drastisch reduzieren. Auserdem wird eine Fallstudie aus dem Smart Grid Bereich vorgestellt, welche einen Anwendungsfall illustriert, bei dem mit Event Stream Processing Lastspitzen abgeschwacht werden. Im Rahmen der Fallstudie kann aufgezeigt werden, wie unsere Optimierungstrategien diesen Ansatz fur ein System mit bis zu 1,6 Millionen Konsumenten skalieren.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2023
538 $a Mode of access: World Wide Web
650 4 $a Sensitivity analysis. $3 3560752
650 4 $a Queries. $3 3564462
650 4 $a Demand side management. $3 3681811
650 4 $a COVID-19. $3 3554449
655 7 $a Electronic books. $2 lcsh $3 542853
690 $a 0454
710 2 $a ProQuest Information and Learning Co. $3 783688
710 2 $a Humboldt Universitaet zu Berlin (Germany). $3 3480542
773 0 $t Dissertations Abstracts International $g 84-03A.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=29280678 $z click for full text (PQDT)