Strange document behaviour in some light platform tests

Answered

In a few tests running under LightPlatformCodeInsightFixtureTestCase I started getting strange failures. If I run the single test class all pass. When I run a test suite of multiple test classes these 4 tests in one test class fail consistently. Including getting caret offset in editor > document text length.

These tests get document content to check results of actions. The same as all other tests in the class. The only difference is that the failing tests add \n into the document text.

Tracing the document content showed that when run in a test suite document content after action modification was reverted to original content on creation, with modification time stamp becoming of -1.

More logging to see if I was cleaning up the editor and file too soon, I added trace on document content and time stamp right after creation and before cleanup. This somehow fixed the issue.

Now I get document text right after creating the file in my test cases and everything is back to normal.

The test initializes file/editor by using the fixture to configure from a file name and text:

configureByText(getExampleFileName(myExample), testInput);
// CAUTION: getting document char sequence seems to be needed, without it some tests have document 
//   content reverted to original after action has modified it causing tests to fail and caret 
//   offset to be out of sync with content, including > textLength()
getEditor().getDocument().getCharsSequence();

Run time and other tests do not have an issue and this only started happening lately with snapshot build of community edition. Is this expected for documents in light platform tests?

9 comments
Comment actions Permalink

Could you please share your test class sources (privately)? Thanks.

0
Comment actions Permalink

Yann, I don't mind sharing but there is a lot of hacking of the LightPlatformCodeInsightFixtureTestCase to get around the standard fixture and tests using junit assertions which I cannot catch. There are also tons of delegated protected methods to give public access adding to the noise.

The test case class in question is a subclass of a subclass of a subclass of LightPlatformCodeInsightFixtureTestCase so all would have to be included to make sense.

It is not pretty or easy on the eyes.

I think what I should do is boil it down to a class which extends the LightPlatformCodeInsightFixtureTestCase directly and only has methods which are used by the tests. That way it will be decipherable without needing the all the classes in the test framework.

Also, the test, when run on its own, works without the document getCharsSequence() hack. Only when run as part of a test suite was the hack needed. So I should probably isolate which other test when run with this one causes the issue. There are only about a dozen of them so commenting all out and uncommenting one at a time until the issue shows up should not take long.

I should be able to get the boiled down test done by this evening.

Where should I upload or post a link?

0
Comment actions Permalink

Thanks, you can upload it to https://uploads.services.jetbrains.com/ and write me the filename to yann.cebron @@ jetbrains.com, Thanks.

0
Comment actions Permalink

Creating a minimal test case that failed was a worthwhile exercise. Found some optimization issues in my code learned a lot from deconstructing the tests.

Managed to create a test which is self-contained except for the IntentionAction which is used in the test. Simulating it with a dummy intention action that does the same document modification does not exhibit the reversion of document content to original.

I discovered what causes the failure but still do not quite grasp why the document reverts to original content under these circumstances.

The cause is a non-platform test case which populates the plugin’s own extension points with instances of the implementation of these extensions, bypassing the plugin.xml defined extensions and instantiation. Effectively, instead of EP_NAME getExtensions() using the IDE instantiated extensions, it uses test instances which were created using new ExtensionPoint....(). Code for that is included in the MinimalCoTestCase.java.

From the point of view of the extension implementation classes there is no difference. Neither is there a difference from the point of view of the class using these extension points. Both are unaware that the IDE was not involved.

The zipped file has the following files:

  • FailTrace.txt : complete console and debug log from the failed test.
  • MinimalCoTestCase.java : the minimal test needed to run in the same test suite to make the other test fail.
  • MinimalIntentionSpecTest.java: minimal intention test case which only fails when run with the other test case.
  • MinimalPlatformTestSuite.java: test suite running both test cases.
  • Description.md: copy of this description in markdown.

The file was uploaded and named DocumentContentRevertInTest.zip

0
Comment actions Permalink

If it wasn't for the fact that the test works if only getEditor().getDocument().getCharSequence() is added right after creating the file using the test fixture I would just shrug it off to an interaction caused by my use of bypass EPs. But that little action changing the outcome seems strange in itself.

0
Comment actions Permalink

It bothered me that the failure only occurred in specific tests and only ones adding EOL into the document were failing so I added variation of replacement text with two EOLs, one and none. Only if two EOLs are inserted does the issue occur.

I added new full traces and uploaded a new version:

  • complete console and debug logs from the failed and passed test versions.
    • FailTraceTwoEOLs.txt: two EOLs inserted
    • PassTraceNoEOLs.txt: no EOLs
    • PassTraceOneEOL1.txt: one at start of string
    • PassTraceOneEOL2.txt: one at end of string

The file was uploaded and named DocumentContentRevertInTest2.zip

 

0
Comment actions Permalink

I added the full trace for passing test when doc chars are obtained on file creation to help isolate the issue:

  • complete console and debug logs from the failed and passed test versions.
    • FailTraceTwoEOLs.txt: two EOLs inserted
    • PassTraceNoEOLs.txt: no EOLs
    • PassTraceOneEOL1.txt: one at start of string
    • PassTraceOneEOL2.txt: one at end of string
    • PassTrace2EOLsDocGetChars.txt: two EOLs added but getEditor().getDocument().getCharsSequence()was used when file was created.

The file was uploaded and named DocumentContentRevertInTest3.zip

0
Comment actions Permalink

Hello Vladimir. If you meant "DocumentContentRevertInTestMoreTraces3.zip" then I was unable to reproduce the issue because some sources are missing (all these "MdPlugin" classes). 

But from what you've said, the reason of exception can be that you've overwritten some extensions in not compatible manner.

The intellij platform doesn't have especially strong protection against overwriting its internal data structures, so it's always advisable to use standard extension mechanism.

E.g. use `<codeStyleSettingsProvider implementation="MdCodeStyleSettingsProvider"/>` in plugin.xml instead of ```

.addExtensions(MdCodeStyleSettingsProvider.EP_NAME, () -> new MdCodeStyleSettingsProvider[] {
new MdEnhCodeStyleSettingsProvider(),
})```

 

 

 

0
Comment actions Permalink

Alexey, just to clarify. I do not overwrite any extensions or IDE internal structures. What I do is wrap the EP_NAME.getExtensions() in my own code so that if RUNNING_TESTS is true it does not call EP_NAME.getExtensions() but uses the extensions added via the test extensions mechanism. 

This way the only side-effect is that the IDE never instantiates the extensions but the rest of the code is untouched and unmodified. There is nothing overwritten or modified in the IDE structures. I have enough work without risking corruption of IDE structures adding to my work. ;)

Duplicating this would probably require the plugin source because it only occurs if the real intention is invoked and only if it inserts two "\n" in the document. Trying to make it happen with simulated intention action or just document mods does not make it fail.

I was hoping that the difference between full trace of the failure and the full trace of it passing when getEditor().getDocument().getCharSequence() right after creating the file would help show where the problem could lie.

There is definitely a difference but to my untrained eye nothing major until the last step of getting document chars for comparison. When it fails the document returns text which was used to create it, even though in the trace I can see that the intention action modified it and the document content reflected the change correctly. Somewhere between the mod and after the highlight pass the document contents get reverted to original. 

Again, the fact that this happens only when two "\n" are inserted and is fixed by getting document chars before any changes are applies says that something different in the highlight pass or internal processing is involved.

I will see if I can get a stand-alone test to duplicate this but do not hold too much hope because of all previous attempts at reducing the activity resulted in not having the issue come up.

0

Please sign in to leave a comment.