Run configurations with macro $ProjectFileDir$ do not work properly after starting the IDE/opening a new project

In our project, we are sharing run configurations via Git. These run configurations are part of one of our application's modules and get imported by IntelliJ IDEA automatically when a project with this module is opened.

This worked fine with 2021.3, but since 2022.1 and greater the run configurations get annotated with a red cross after Idea was started or the project was opened: 

This only happens with run configurations which contain the $ProjectFileDir$ macro. When opening these run configurations, the dialog complains that the directory containing the $ProjectFileDir$ macro does not exist:

The problem here is that the $ProjectFileDir$ variable does not seem to be assigned at this moment. In some cases, it was assigned but pointed to the project file directory of a different project that had been opened by the IDE before. (This can lead to very weird errors when the files of two different projects get mixed up.)

The only way we found to fix this is to click the "Insert Macros" icon

and press "Cancel" in the following dialog. Then Idea seems to remember the correct setting for the $ProjectFileDir$ macro and all the red crosses shown on the launch configurations disappear.

This is quite annoying as you have to do this each time you open the IDE or open a new project. And it is even more annoying as files of different projects can get mixed up when the $ProjectFileDir$ macro accidentally points to the project file directory of a different project so that the launch config of your current project A is using a working directory which belongs to project B.