PyCharm unable to find fixtures in conftest.py

Answered

I recently updated a project from Python 3.10 to 3.11 which broke my ability to run tests through PyCharm. The issue is that fixtures in conftest are no longer recognized, and nearly all of the tests I work with are organized in that way. In fact, it doesn't look like the conftest files are even executed now. I've tried adding break points and print statements within conftest files in several file locations and none of them hit.

The following code works if it's all in one file
```
@pytest.fixture(scope="session")
def work():
    print('work')
    return 'work'

def test_with_fixture(work):
    print(f'Fixture work -- {work}')
```
However if I move the fixture to a conftest.py file the test fails with 

```
test setup failed
file project/path/test_file.py, line 27
  def test_with_fixture(work):
E       fixture 'work' not found
>       available fixtures: _session_faker, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, class_mocker, cov, doctest_namespace, faker, mocker, module_mocker, monkeypatch, no_cover, package_mocker, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, session_mocker, testrun_uid, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, worker_id
>       use 'pytest --fixtures [testpath]' for help on them.
```

I have confirmed that the run config is using pytest. 

I have also double checked the settings and ensured that the Project Structure > Content Root is correct, and that Python Integrated Tools > Testing is set to `pytest`.

If I revert back to Python 3.10 the tests once again pass. If I run the tests outside of PyCharm they also pass. So it seems specific to PyCharm + Python 3.11. I've tried this with several versions of PyCharm including the current latest (2023.2) and all have the same issue.

Losing the ability to run tests through PyCharm is a big loss for me and I'm hoping that there is a solution.

 

Edit: After some further debugging I find few more things of note.

1. Putting a conftest file in the exact same directory as the test being run *does* work, but no other locations do.
2. I tried running a test from both a terminal and through pycharm with `--trace-config`, which revealed two differences. The non-PyCharm version registered all of the conftest files, as expected. The PyCharm version also has two lines in the trace that do not appear when run outside of it. 

```
PLUGIN registered: <module 'teamcity.pytest_plugin' from '/Applications/PyCharm 3.app/Contents/plugins/python/helpers/pycharm/teamcity/pytest_plugin.py'>
PLUGIN registered: <class '__main__.Plugin'>
```

Is it possible that one of those modules are somehow breaking the discovery of conftest files? I'm not yet sure how to confirm that but it seems like a potential clue.

3
27 comments

When you run the tests in the terminal, do you run them from the same working directory that you have in the run configuration in PyCharm?
Would it be possible to provide an example project that we can use for reproducing the problem?

0

Ah, so the working directory does make a difference. From the terminal I was running the test from the project root and passing the path to the test file. The run config that PyCharm generates automatically sets the working directory to the test file directory. If I update the config to switch the working directory to the project root and update the Target field the test now works as expected.

If I edit the pytest configuration template and set the working directory to the project root, new run configs will fill in the correct target and work. I typically work with a number of different repos and it will be somewhat inconvenient to have to set this for every project. Is there a way to configure PyCharm to always set working directly to the project root for pytest run configs?

2

I still haven't found an answer to my last question. If anyone knows a way to update the default behavior to use the correct working directory in new run configs I would appreciate it. Having to fix this in every single code repo is a pain.

1

Hey, I can confirm the bug and it is so annoying.
Running all the tests (right click on the tests folder and “Run pytest in tests…” works perfectly.
But running a test from a subfile directly doesn't load the conftest anymore. I have started noticing this for 2 months I think.

0

Sergey Karpov, you can find a very simple / quick / dirty project to reproduce the bug here: https://github.com/FloChehab/pycharm_bug 
 

Here are the infos: 

PyCharm 2023.2 (Professional Edition)
Build #PY-232.8660.197, built on July 26, 2023
Licensed to Florent Chehab
Subscription is active until July 13, 2024.
Runtime version: 17.0.7+7-b1000.6 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Linux 6.4.11-200.fc38.x86_64
GC: G1 Young Generation, G1 Old Generation
Memory: 2048M
Cores: 12
Non-Bundled Plugins:
   cn.haojiyou.CodeGlance3 (2.1.1-2021.1.3)
   YAML/Ansible support (0.11.2)
   ru.adelf.idea.dotenv (2023.2)
   net.ashald.envfile (3.4.1)
   com.koxudaxi.pydantic (0.4.7)
   Key Promoter X (2023.2.0)

Current Desktop: GNOME

1

This works

The following doesn't work (the test run but the conftest is not loaded so the test actually fails).

1

Today's update, right clicking on `tests` on “Mark Directory as … Tests Sources root” also works. It feels like in the past Pycharm would find this on its own in the past.

4

Thanks Florent Chehab for message above. It helped me. 

 

1. Mark Directory tests as Test Sources Root

2. Also I deleted .pytest_cache folder(s). 

0

I am also experiencing this issue (fixtures defined in `tests/conftest.py` are not available to tests written in sub-directories under `tests/`, even though the same tests work fine when running pytest on the commandline) with the latest version of Pycharm on MacOS.  The above fix (Mark Directory as -→ Test Sources Root) is not working for me, even after deleting .pytest_cache dirs.  

 

This seems like a significant regression - it is certainly impeding my work.  Is this issue filed/being tracked in YouTrack?  I found several issues there related to pytest and conftest (seems like some recent change must have caused several regressions?) but I can't find this one specifically.

For reference, my environment:

  • MacOS Monterey (12.6.9)
  • PyCharm 2023.2.1 (Community Edition)
  • Build #PC-232.9559.58, built on August 22, 2023
  • running pytest 7.4.2
  • running Python 3.8.16


 

1

My colleagues and I are also experiencing this issue.

PyCharm 2023.2.1 (Professional Edition)
Build #PY-232.9559.58, built on August 22, 2023
Licensed to ----
Subscription is active until May 19, 2024.
Runtime version: 17.0.8+7-b1000.8 aarch64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 13.5.2
GC: G1 Young Generation, G1 Old Generation
Memory: 2048M
Cores: 10
Metal Rendering is ON
Registry:
   debugger.new.tool.window.layout=true
   ide.experimental.ui=true
Non-Bundled Plugins:
   commit-prefix-plugin (1.2.1)
   com.atlassian.bitbucket.references (2022.1.242)
   com.koxudaxi.pydantic (0.4.8)
   com.intellij.bigdatatools.core (232.9559.62)
   com.intellij.bigdatatools.binary.files (232.8660.206)
   com.intellij.bigdatatools.rfs (232.9559.62)
   mobi.hsz.idea.gitignore (4.5.1)
0

Hi, please vote for https://youtrack.jetbrains.com/issue/PY-61040
The change in the fixtures resolve was intentional, but we are tracking other use cases in that issue.

0

Just wanted to follow this as I am facing the same issue as well. Like other users, the fixtures are found and utilized correctly when run from the directory marked as “Test Sources Root” but not when run from any subdirectories.

0

Sergey Karpov When you say the change to fixtures is intentional, does this mean that we shouldn't expect the issue to be resolved? This is very much hurting my ability to develop and run tests efficiently. 

For context on our use case for this, we use a fixture in the test root to define a common spark session configured for the test session. We have 100s of the tests in our various subdirectories that are dependent on this this spark fixture. Running the whole test suite takes about 10 minutes, so it's incredibly inefficient for us to run them all when testing small changes that will only impact a small subset of the tests.

Can this be fixed by reverting to earlier version of PyCharm? If not, I'll have to use VS Code or CLI for testing until this is fixed.

1

It wasn't an unexpected change, but there are some cases (as those in the comments) that we didn't consider. That's why the ticket wasn't closed. Feel free to comment about your specific case to make sure the developer is aware of it.
Yes, you can downgrade to a previous PyCharm version to see the old behavior.

0

Sergey Karpov , the ticket you referenced is specifically about the “go to usages” UI in PyCharm.  That is not what this thread is about - here, we are finding that PyCharm is failing to find `pytest` `conftest.py` fixtures for tests in subdirectories, even though `pytest` itself has no trouble locating those fixtures.

This is a PyCharm regression, pure and simple.

2

When you run it in the terminal, do you run it from the same working directory that you have in your run configuration in PyCharm?

0

Confirmed the issue is caused by the working directory configuration: 

An example project folder like this one where `test_completions_service.py` uses a fixture available in `conftest.py` will fail running in pycharm.

Trying to run any individual test (or test folder) will generate a run configuration like this, where the working directory points to the location of the test file/folder itself and it will not recognize the root `conftest.py`

As a temporal workaround, modifying the run configuration to point to the root of the project works, but it has to be done for every single run configuration:

 

 

1

Pablo Orgaz thank you so much for your answer! Your answer helped me find a more persistant solution by modifying the working directory in my PyTest and Autodetect run/debug configuation templates!

0

Thank you Pablo Orgaz. That bandaid fix resolved my issue as well. For other folks you can go to the "Run/Debug Configuration > Edit Configuration Templates" and change the pytest working directory to the root of the project. Be sure to delete all existing test configurations. The change should persist for new tests. Unfortunately, you have to do this for every project.

I will say this was a very unexpected change for PyCharm. I wasted far too long trying to figure out why conftest.py was not being picked up.

1

I don't believe I ever hit a problem with the old behavior, whereas this regression hurts nearly every single test in every single repo I work in. The pytest template bandaid fix helps for one repo, but for anyone who works in many different projects it's still a large hassle, on top of the original debugging of the regression that many have wasted time on.

 

I hope this issue gets fixed. I'm already reluctant to update JetBrains products given past regressions and this thread strengthens that concern.

0

FWIW, for me the resolving of fixtures in a higher-level configuration file (`tests/conftest.py` ) breaks after upgrading `pytest` from 7.3.2 to 7.4.0.

I don't know how to fix it but pinning`pytest` on 7.3.2. for the time being, keeps things working in PyCharm.

0

I can also confirm that this is related to the version of pytest, not PyCharm. But I imagine PyCharm needs some kind of update to make it play nicer with pytest. In my case, when using the latest pytest, specifying Working Directory is not a viable workaround because if you do so, then running individual tests by clicking the green arrow next to them in the editor won't copy in the additional args from the template. Something about specifying both working directory and additional args is not compatible on individual tests.

0

It should be fixed in the latest release (2023.3.2) of PyCharm.

0

for me using current intellij with python plugin 233.14475.28 it is not fixed. not sure how pycharm and this plugin are related version wise

0

I hit this issue when using a docker-compose interpreter. After adding the local path to my project as the `Working directory` in my run config it worked:

0

Please sign in to leave a comment.