Notifications "stuck" and can't be closed anymore

I've developed a plugin that shows balloon notifications for GitlabCI pipelines. When the plugin is running while the computer is locked or otherwise idle (not sure) and it continually shows notifications, when the computer is unlocked then the notifications cannot be closed. I've already added code that prevents more than three notifications to be shown at any time. I've tried analyzing the notifications with the UI inspector but I cannot select them. You can't click them either, the mouse click just "goes through". The only way to get rid of the notifications is by restarting IntelliJ. Apparently there's no option to automatically expire the notificiations after a given time.

I'm posting this because it only affects notifications made by my plugin. Is there any way to debug this problem or, better, prevent it in the first place? I didn't find any "best practices" regarding notifications.

You can find the code for the notifications here: https://gitlab.com/ppiag/intellij_gitlab_pipeline_monitor/-/blob/master/src/main/java/de/sist/gitlab/notifier/NotifierService.java

Thank you.