remote testing/codecoverage using codeception

Hi there,

No -- you cannot add your own stuff to the Run/Debug just like that -- it has to be properly supported via plugin.

http://youtrack.jetbrains.com/issue/WI-16350 -- star/vote/comment to get notified on progress.

What you could do is:

• Write a batch/shell script (or whatever) that will do all the testing/path transformations etc
  
• Add it as an Extrenal Tool (Settings | External Tools)  .. or maybe as Remote Extermal Tools and call it when necessary
• Once called and sucessfully executed, you will need to get the results back to your local computer (unless they will be available via mount/network path)
• Manually load coverage xml file

The xml file is available locally as a vagrant folder. Does phpstorm keep track of changes in that file?  So if i manually do "Show Code Coverage Data", and it's a 'converted' xml file that has proper code coverage info, and this file changes because I do another shell run, will phpstorm reload the info?

If not, what if I set up an empty phpunit.xml and run phpunit from the Run menu, will that reload the xml file?

Well .. I cannot answer for 100% .. but as far as I'm aware (99.99% -- based on my experience working with PHPUnit integration):

The xml file is available locally as a vagrant folder. Does phpstorm keep track of changes in that file?  So if i manually do "Show Code Coverage Data", and it's a 'converted' xml file that has proper code coverage info, and this file changes because I do another shell run, will phpstorm reload the info?

No -- info is cached and original file is not monitored for changes. At very least this is how it was 2-3 years ago when PHPUnit coverage was not yet implemented (but coverage feature already existed -- it's a platform feature after all).

If not, what if I set up an empty phpunit.xml and run phpunit from the Run menu, will that reload the xml file?

No:

1. Coverage info will ONLY be loaded if "Run with Coverage" action was used
2. PhpStorm tells PHPUnit to ALWAYS generate separate copy of coverage.xml (regardless of what phpunit.xml (or whatever phpunit config file you will use) says about that)

Maybe ... if you record a macros with loading coverage file it will do the job ... but considering that macro support in IDE is very weak ... I really doubt that it will work.

So here's what im thinking. Replace phpunit with a bash script that detects if a specific option was passed along, lets say --codeception, then it runs codeception with whatever I need to do to put a clover.xml in the right spot for phpstorm.

If the bash script doesnt see that option, it just runs the real phpunit.

Wonder if that would work. Bit of a hack,and it wont survive upgrades of phpunit, but maybe until there's a codeception plugin.

Or i just load the file manually whenever I need to see code coverage :)

So here's what im thinking. Replace phpunit with a bash script that detects if a specific option was passed along, lets say --codeception, then it runs codeception with whatever I need to do to put a clover.xml in the right spot for phpstorm.

Which "phpunit" are you intended to replace?

PhpStorm does not use "phpunit" command itself -- it uses PHPUnit's classes directly via special helper php script (which is responsible for IDE integration).

You will have to do quite a lot of work here:

• to write your own helper that would behave exactly the same
• to replace that helper script inside of the PHP plugin .jar file (as this helper script is always extracted from plugin file; written into your system temp folder and gets deleted once it's over)

Or i just load the file manually whenever I need to see code coverage

For me, considering the above, this will be the easiest option (although not that convenient for sure).

I do not know how codeception works (never used it -- only PHPUnit; but would definitely try once proper support will be implemented) but if it generates HTML reports like PHPUnit does, I would use such html version of coverage instead -- just keep it open in browser and refresh it when necessary (at least I find PHPUnit's coverage report a bit more convenient over PhpStorm's one -- possibly because it's optimized just for that task only while in PhpStorm coverage representation has to cover other details as well, like ability to conveniently edit source code while having coverage info).

I guess the best option is to try your idea first and see how it goes (just instead of Run use "Debug" and tell IDE to stop on first line -- debugger should take you to that helper file so you will have a chance to look at it)

I meant the phpunit on the vagrant box. I assumed 'PHPunit on Server' would just run phpunit(1) on the server. But come to think of it, I think i read it uploads some file so probably uses the phpunit classes from that file.

I think just running it manually is the way to go for now :)

Yes -- it's all done via helper script and not actual "phpunit" command/executable file.

What I ended up doing is added a gulp watcher for the codeception coverage.xml on my vagrant box (gulp is running anyways), replace paths so they work on my host system, and i just have to reload the code coverage manually.