How to override file read/write behavior?

Hello IDEA community! I hope everyone was able to enjoy the holidays and celebrate welcoming in the new year :)

I’m working on a plugin to support a new file/data format that can either appear in a human-readable ASCII text format or in its own custom binary file format. I’ve successfully been able to implement what I want for the human-readable text file format, but I’m having trouble with the binary files. 

My end goal is for whenever the user opens a binary file, my plugin will (1) convert the binary file to ASCII text and present that text in the editor to the user. Whenever the user edits the ASCII text in the editor and subsequently saves the file, I’d like my plugin to (2) override the IDE’s file save behavior and perform its own method of saving to ensure it’s saved back into the binary file format.

I’ve created a partially functional form of feature #1 by supplying + implementing a filetype.decompiler extension. Through that method I could successfully convert my binary files to ASCII, but the resulting editor text was neither editable nor syntax highlighted (seemingly because the file contents were now marked read-only and weren’t associated with my custom language), so I scrapped that approach.

I was pointed in the direction of this thread by the JetBrains Platform slack community for a potential solution and tried implementing Peter Gromov’s suggestions to provide my own FileViewProviderFactory extension. However this also did not seem to work for my plugin; anytime I opened a binary file, the editor opened up the raw file contents and not the converted text I hoped it would.

I did some more experimenting and found I could “hack” a solution to both features #1 and #2 by providing my own FileEditorProvider and overriding its createEditor method to supply my own custom VirtualFile class (which has an overridden contentsToByteArray method to convert the binary contents to ASCII) into a newly constructed TextEditor. This gave me the behavior I wanted where opening a binary file would display its converted ASCII text in the editor, the text would be editable, and saving the file would correctly save back into the binary format. However this resulted in other new strange behaviors (like clicking on elements in the structure view resulted in the file being re-opened in a new editor tab) that was likely the result of this improper implementation, so I scrapped that attempt too.

From that last experiment however, I have a hunch that the “right” way to go about implementing my desired behavior would be to provide my own VirtualFileSystem extension and have it supply instances of my custom binary VirtualFile class. I’ve begun looking into how to do this and I’ve been a bit at a loss as to what the connections are between my custom VirtualFileSystem extension and the IDE and how/where the methods in my VirtualFileSystem are called from.

Is the custom VirtualFileSystem extension the right approach to what I’m attempting? If so, how would I specify that I’d want my VirtualFileSystem to supply the custom VirtualFiles for my custom binary file format? I’m hoping someone can guide me in the right path here - any help would be greatly appreciated. Thank you!