Consider the following series of events:
4:00 GMT Detection runs for interval 3:30--4:00
4:30 GMT Detection runs for interval 4:00--4:30
4:35 GMT Event 1 occurs on an endpoint
4:45 GMT Event 1 is indexed
5:00 GMT Detection runs for interval 4:30--5:00
5:05 GMT Event 1 finding is added to ES with timestamp 4:35
5:24 GMT Event 2 occurs on an endpoint
5:30 GMT Detection runs for interval 5:00--5:30
5:35 GMT Event 2 is indexed
6:00 GMT Detection runs for interval 5:30--6:00
What is the problem with the detection schedule chosen and how can it be solved?
The issue is ingestion latency. Event 2 occurs at 5:24 but is not indexed until 5:35. The 5:30 scheduled search covers event time 5:00--5:30, but the event is unavailable when that search executes. By the next execution at 6:00, the search window has advanced to 5:30--6:00, so an event timestamped 5:24 can fall outside the new window even though it is now searchable.
The solution is to increase the detection's lookback window sufficiently to accommodate expected indexing delay. Operationally, scheduled detections frequently use overlapping windows---for example, executing every 30 minutes while looking back farther than 30 minutes---to prevent late-arriving events from being missed. Appropriate deduplication or throttling can then address repeated matches caused by overlap.
Decreasing the time window would make the latency problem worse. The scenario also does not indicate that an excessively large window is causing duplicate alerts; it demonstrates a late-arriving event that misses the search interval.
Study Guide topics: detection scheduling, ingestion latency, event time versus index time, lookback windows, late-arriving telemetry.
Currently there are no comments in this discussion, be the first to comment!