Quickstart

Coincidence System backend for snews alert trigger

The backend tools that needs to run in order for the observation messages to be cached and compared. The latest snews coincidence logic allows for efficiently searching coincidences on a pandas dataframe. The coincidence script needs to be running for this.

Furthermore, we have a slack bot which can be enabled within the coincidence script (requires channel token). This bot listens to the alert topics and publishes a slack-post as soon as it receives an alert. This way, we are aiming to reach subscribed members faster, and with a slack notification.

The observation messages submitted to a given kafka topic using the SNEWS Publishing Tools. The SNEWS Coincidence System listen this topic and caches all the messages submitted. These messages then assigned to different sublists depending on their neutrino_time.

The first message in the stream makes the first sublist as sublist=0, and sets the coincidence window. Next, the second message is compared against the first and if the neutrino_time differences are less than the defined coincidence_threshold (default is 10sec) it is added to the same sublist and an alert is triggered. This alert is sent to all subscribed users through the Kafka Alert Channels, and a slack bot message is published on the relavant SNEWS Slack channel. In case if the second message is earlier than the first one and there were no alerts triggered before (the first message was alone), then the second message is replaced as the initial message of that sublist and the coincidence window is started from the neutrino time of the second message. The alert is still published.

If there are more incoming observation messages, they are compared against the initial message of each sublist i.e. if the neutrino time difference of the 3rd (or later) messages are within +10 seconds they are added to the same sublist, and another alert message is published, only this time stating that it is an “UPDATE” on the previous alert message.

In case the new message is not coincident with the initial message of a sublist, it is assigned a new sublist number e.g. sublist=1. Since, even though it might not coincide with the initial message of the sublist=0, it can, in principle, still be coincident with the later messages in the sublist=0. Thus, once a new sublist is created, we go over all the messages in the cache to see if non-initial messages in other sublists coincides with this new sublist. This allows for same messages being a part of two nearby coincidence windows.

Let’s look at a simple example, imagine having three detectors submitting the following messages in this order;

detector1 = {'neutrino_time': "01/01/2022 12:30:00:000000"}
detector2 = {'neutrino_time': "01/01/2022 12:30:09:000000"}
detector3 = {'neutrino_time': "01/01/2022 12:30:11:000000"}

detector1 sets a new sublist; sublist=0 and opens a coincidence window. Later, detector2 submits and it is a coincidence with detector1 thus it is added in the same sublist and a SNEWS alert is published, everyone is happy, popping champaigns, celebrations are in order. Then, detector3 publishes their message. The delta time with the initial message of the sublist=0 is 11 seconds, thus it does not enter this sublist. It creates a new sublist sublist=1. Since there is a new sublist created, we go back and look if any of the existing messages would also satisfy the coincidence condition with this sublist. Indeed, the detector2 has a delta time of -2 seconds! Since this is an earlier message we assign detector2 to be the initial message of sublist=2 and keep the detector3 as a second message in sublist with a delta time=+2. Finally, we sent another message with this new coincidence sublist. At the end, there will be two alerts corresponding to sublist=0 and sublist=1. Notice, if there were a 4th message at a much later time, it would also create a sublist=3 and look through the cache. However, since nothing would match, it would sit alone and wait for more messages to come without triggering any alert.

In reality, Supernova time window is expected to be much smaller than 10 seconds, thus it is already a conservative time boundary. However, we wanted to have a stable and bullet proof logic that could also prove useful during stres tests and firedrills.

All the observation messages are kept in cache for 24 hours after their neutrino_time and unless they are involved in any coincidence they are discarded. These messages together with the triggered alert messages are also locally stored in the Purdue servers through a Mongo Database connection.