Plugin development questioning if inlays are the appropriate tool

I am writing a plugin that will add custom markers above certain characters within the editor. These markers, will be specific shapes and colors. The end goal is to be able to use a speech to text system to instruct the IDE to select, copy, cut… portions of the file using these markers. I have a rudimentary plugin implemented but I have used inlays for it to date. This works but has generated some problems with navigation and changing the file as the inlays are somewhat navigable. This has led me to question whether I should be using inlays in the first place but I have not seen another appropriate tool that is available. Is this the appropriate tool or is there something else that I should be doing to draw these markers above characters within the file?