Safe Delete on PsiElement(XML_ATTRIBUTE_VALUE) fails because BaseRefactoringAction handles PomTargetPsiElementImpl class

I want to implement Safe Delete for our Custom XML contents.
Something like 
<concept id="LinkViolations">
    <description>Link Violations to source code nodes.</description>
</concept>

I have a DOM-Structure for our specific XML-Schema like in:

https://intellij-support.jetbrains.com/hc/en-us/community/posts/206778105-PsiReferences-and-DOM

interface RuleBase : DomElement {

 @get:Stubbed 

@get:NameValue(referencable = true) 

@get:Attribute("id") 

val id: GenericAttributeValue<String>

 }

Calling a Safe-Delete action on the value of <concept id="value"> results in a PomTargetPsiElementImpl passed to BaseRefactoringAction via DataContext.

But it extends FakePsiElement which always returns false for isPhysical(), which is checked by SafeDeleteProcessor.validElement().

Result: Intellij disables the action

I did some more debugging and while I am not 100% sure it seems as the PSI Tree is first instantiated with the PomTargetPsiElementImpl and later on our plugin creates XmlAttributeValueTokenImpl instances. But obviously they don't replace the PomTargetPsiElementImpl. What is needed for that?