Apparent BUG in Linux PyCharm's built-in support of pipenv

IN BRIEF: PyCharm correctly processes most Packages in a Pipfile, but chokes on installing one, mysql_config

(perhaps because it expects a BASH execution of the command /bin/sh mysql_config, but Linux Mint processes /bin/sh as DASH shell commands)


OVERVIEW:


I have a Pipfile, reproduced at the bottom.  My PyCharm is configured for pipenv, as directed in https://www.jetbrains.com/help/pycharm/pipenv.html

All seems good at first.  PyCharm detects the Pipfile and imports all packages, EXCEPT mysql_config , which remains uninstalled - screen shot:

Asking PyCharm to proceed with the "Install requirements from Pipfile.lock" (i.e. to install the mysqlclient package) leads to an error. 

The full error message is in the Details section below, but of particular interest seem statements like these:

------------------

I'm seeing statements like

"/bin/sh: mysql_config: command not found', '    /bin/sh: mariadb_config: command not found', '    /bin/sh: mysql_config: command not found',"

--------------------

 

On my Linux Mint 19.1 system, I indeed have mysl_config installed, and it runs just fine if I type it into the shell, but NOT if I type /bin/sh/mysl_config into the shell.  See screenshot:

 

On Linux Mint, /bin/sh links to DASH.   And /bin/dash mysql_config  fails.    By contrast, /bin/bash mysql_config works.

So, presumably, the script internally used by PyCharm is expecting a BASH shell when it invokes /bin/sh - and it chokes when it instead gets a DASH.

Scripts ought to specify which shell they need, rather than rely on /bin/sh , whose implementation may change over time - as it indeed has.

I found a source that stated it well:


"/bin/sh is an executable representing the system shell. Actually, it is usually implemented as a symbolic link pointing to the executable for whichever shell is the system shell. The system shell is kind of the default shell that system scripts should use. In Linux distributions, for a long time this was usually a symbolic link to bash, so much so that it has become somewhat of a convention to always link /bin/sh to bash or a bash-compatible shell. However, in the last couple of years Debian (and Ubuntu) decided to switch the system shell from bash to dash - a similar shell - breaking with a long tradition in Linux (well, GNU) of using bash for /bin/sh. "

 

I created a StackOverflow thread to try to find a workaround:  https://stackoverflow.com/questions/57964618/how-to-make-the-dash-shell-run-some-commands-meant-for-sh-bash

 

But in the short term, I had to ditch PyCharm's built-in support of pipenv, and instead use pipenv directly.

==>  Using pipenv directly (without PyCharm) worked just fine, and in particular installed the mysqlclient package that PyCharm kept choking on.



DETAILS:


The Pipfile:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
django = "*"
djangorestframework = "*"
mysqlclient = "*"
django-mysql = "*"
djangorestframework-simplejwt = "*"
pyyaml = "*"
uritemplate = "*"
django-extensions = "*"
graphviz = "*"
django-rest-swagger = "*"
pydot = "*"

[requires]
python_version = "3.7"

 

The error message from PyCharm's attempt to install mysqlclient (boldface added by me):

Installing dependencies from Pipfile.lock (9c896a)…
Installing initially failed dependencies…

An error occurred while installing mysqlclient==1.4.3 --hash=sha256:0d3eeb97534c9f42e9b65bd35c1a35fadc2f6d7ea5d9fdbda5339a569156344b! Will try again.
[pipenv.exceptions.InstallError]: File "/app/lib/python3.7/site-packages/pipenv/core.py", line 1874, in do_install
[pipenv.exceptions.InstallError]: keep_outdated=keep_outdated
[pipenv.exceptions.InstallError]: File "/app/lib/python3.7/site-packages/pipenv/core.py", line 1253, in do_init
[pipenv.exceptions.InstallError]: pypi_mirror=pypi_mirror,
[pipenv.exceptions.InstallError]: File "/app/lib/python3.7/site-packages/pipenv/core.py", line 859, in do_install_dependencies
[pipenv.exceptions.InstallError]: retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[pipenv.exceptions.InstallError]: File "/app/lib/python3.7/site-packages/pipenv/core.py", line 763, in batch_install
[pipenv.exceptions.InstallError]: _cleanup_procs(procs, not blocking, failed_deps_queue, retry=retry)
[pipenv.exceptions.InstallError]: File "/app/lib/python3.7/site-packages/pipenv/core.py", line 681, in _cleanup_procs
[pipenv.exceptions.InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: ['Collecting mysqlclient==1.4.3 (from -r /tmp/pipenv-ukg97i_b-requirements/pipenv-p6uu66ff-requirement.txt (line 1))', ' Using cached https://files.pythonhosted.org/packages/13/37/6c941aabeb78464a538b7d43266bfae645993779e3ca3e12fe82b4590f52/mysqlclient-1.4.3.tar.gz']
[pipenv.exceptions.InstallError]: ['ERROR: Command errored out with exit status 1:', ' command: /home/juls/.var/app/com.jetbrains.PyCharm-Community/data/virtualenvs/cloud-hcs-O6XmIN9f/bin/python3.7 -c \'import sys, setuptools, tokenize; sys.argv[0] = \'"\'"\'/tmp/pip-install-mf8uxtsq/mysqlclient/setup.py\'"\'"\'; __file__=\'"\'"\'/tmp/pip-install-mf8uxtsq/mysqlclient/setup.py\'"\'"\';f=getattr(tokenize, \'"\'"\'open\'"\'"\', open)(__file__);code=f.read().replace(\'"\'"\'\\r\\n\'"\'"\', \'"\'"\'\\n\'"\'"\');f.close();exec(compile(code, __file__, \'"\'"\'exec\'"\'"\'))\' egg_info --egg-base pip-egg-info', ' cwd: /tmp/pip-install-mf8uxtsq/mysqlclient/', ' Complete output (12 lines):', ' /bin/sh: mysql_config: command not found', ' /bin/sh: mariadb_config: command not found', ' /bin/sh: mysql_config: command not found', ' Traceback (most recent call last):', ' File "<string>", line 1, in <module>', ' File "/tmp/pip-install-mf8uxtsq/mysqlclient/setup.py", line 16, in <module>', ' metadata, options = get_config()', ' File "/tmp/pip-install-mf8uxtsq/mysqlclient/setup_posix.py", line 61, in get_config', ' libs = mysql_config("libs")', ' File "/tmp/pip-install-mf8uxtsq/mysqlclient/setup_posix.py", line 29, in mysql_config', ' raise EnvironmentError("%s not found" % (_mysql_config_path,))', ' OSError: mysql_config not found', ' ----------------------------------------', 'ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.']
ERROR: ERROR: Package installation failed...

4 comments
Comment actions Permalink

Interestingly, another user in these forums reported, a few months ago, the same broad issue of "pipenv project working on command line but not in pycharm", but his report was never fully answered: https://intellij-support.jetbrains.com/hc/en-us/community/posts/360003455060-pipenv-project-working-on-command-line-but-not-in-pycharm 

0
Comment actions Permalink

Hi,

I seem to reproduce the issue, but it also reproduces for me in the terminal. Can you describe how did you worked around it without using PyCharm?

1
Comment actions Permalink

@Andrey, in the terminal (Linux Mint 19.1), I did:

1) I Installed the needed version of Python.   I needed 3.7 for this project, while I had 3.6 as my system version.

2) sudo apt install libmysqlclient-dev   [to install mysql_config]

3) cd <my directory where the repository files - incl. the Pipfile - are>

4) pipenv --python /usr/local/bin/python3.7  install     [here I specified an alternate version of Python with the --python option; I have 3.6 installed as my system version]

5) pipenv shell   (to activate this project's virtual environment: it says "Launching subshell in virtual environment…", and it prints out . followed by the name of the virtual env folder, followed by /bin/activate)

6)  python --version   This ought to show the Python version specified to pipenv

0
Comment actions Permalink

Please see the issue I've created: https://youtrack.jetbrains.com/issue/PY-38052

If everything is correct, vote and follow. If you would like to make some corrections, please let me know.

1

Please sign in to leave a comment.