PyCharm Runserver command issues

Answered

I'm trying to make thumbnails from pdfs. To do it, I need to import wand.

It runs fine if I do `./manage.py runserver` from the command line, but if I run from PyCharm, it breaks.

When I step through the code, the problem is that the code used to open the blob is always empty. It's the right class (wand.image) but it's an empty one. The object I pass it is a pdf, but the blob conversion, which produces no error at all, is empty.

Again, if I launch the server from the command line, it works, but if I launch from PyCharm, it breaks.

Here's the code I'm running:

from wand.image import Image as WandImage
from wand.color import Color


def convert_to_thumb(pdf_path, slug):
with open(pdf_path) as f:
image_binary = f.read()
all_pages = WandImage(blob=image_binary) #<-- Here image_binary is a pdf
single_image = all_pages.sequence[0] #<-- BOOM! all_pages is a wand.image, but it's empty. Gives an Index error
with WandImage(single_image) as i:
i.format = 'png'
i.background_color = Color('white')
i.alpha_channel = 'remove'
i.transform(resize='x100')
save_name = slug + '_pdf_preview.png'
i.save(filename='/foo/bar/' + save_name)

return i
19 comments
Official comment

SysPath sometimes affects some libraries. E.g. library may need some external plugin and fail to find it due to incorrect sys.path.

You can also check environment variables, they may be different when tool launched from PyCharm and from terminal. That is all difference between PyCharm and terminal versions.

After it you may use debugger to debug WandImage internals. 

Hello.

Not sure what prevents WandImage from working (you probably need to debug it), but it could be syspath issue, since it is the only thing that may be different in PyCharm. Please try to evaluate "sys.path" when debugging or simply print it and compare output when launched from console and from PyCharm. It may give us a clue.

0

I actually doubt it's a syspath, since I would get an error when calling or even importing wand, right? There's no error at all until the attempt to sequence the empty wand.image object, which is just a standard Python error.

Here are the sys.path from each environment, the only difference I see is a couple are defined twice in PyCharm and the addition of PyDev.

PyCharm sys.path:

['/Users/rcladd/PycharmProjects/pycharm_myapp',
'/Applications/PyCharm.app/Contents/helpers/pydev',
'/Users/rcladd/PycharmProjects/pycharm_myapp',
'/Applications/PyCharm.app/Contents/helpers/pydev',
'/Users/rcladd/venv/myapp/lib/python27.zip',
'/Users/rcladd/venv/myapp/lib/python2.7',
'/Users/rcladd/venv/myapp/lib/python2.7/plat-darwin',
'/Users/rcladd/venv/myapp/lib/python2.7/plat-mac',
'/Users/rcladd/venv/myapp/lib/python2.7/plat-mac/lib-scriptpackages',
'/Users/rcladd/venv/myapp/lib/python2.7/lib-tk',
'/Users/rcladd/venv/myapp/lib/python2.7/lib-old',
'/Users/rcladd/venv/myapp/lib/python2.7/lib-dynload',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/Users/rcladd/venv/myapp/lib/python2.7/site-packages',
'/srv/web/django_apps/myapp/myapp',
'/srv/web/django_apps/myapp']

CLI sys.path:



['/Users/rcladd/PycharmProjects/pycharm_myapp',
'/Users/rcladd/venv/myapp/lib/python27.zip',
'/Users/rcladd/venv/myapp/lib/python2.7',
'/Users/rcladd/venv/myapp/lib/python2.7/plat-darwin',
'/Users/rcladd/venv/myapp/lib/python2.7/plat-mac',
'/Users/rcladd/venv/myapp/lib/python2.7/plat-mac/lib-scriptpackages',
'/Users/rcladd/venv/myapp/lib/python2.7/lib-tk',
'/Users/rcladd/venv/myapp/lib/python2.7/lib-old',
'/Users/rcladd/venv/myapp/lib/python2.7/lib-dynload',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/Users/rcladd/venv/myapp/lib/python2.7/site-packages',
'/srv/web/django_apps/myapp/myapp',
'/srv/web/django_apps/myapp']
0

Can you run it as bare Python script in PyCharm instead of Django "run server" and see if it still does not work?

0

When I run it as a bare python script in PyCharm it fails with exactly the same error.

Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Users/rcladd/PycharmProjects/pycharm_myapp/publication/make_thumb.py", line 10, in convert_to_thumb
single_image = all_pages.sequence[0] # Just work on first page
File "/Users/rcladd/venv/myapp/lib/python2.7/site-packages/wand/sequence.py", line 112, in __getitem__
index = self.validate_position(index)
File "/Users/rcladd/venv/myapp/lib/python2.7/site-packages/wand/sequence.py", line 82, in validate_position
'out of index: {0} (total: {1})'.format(index, length)
IndexError: out of index: 0 (total: 0)

When I run the same python command from the shell (same virtualenv) it runs fine.

0

Here's some more info:

When I run from CLI and use `pdb.set_trace()` to check the value of `all_pages` i get this

(Pdb) p all_pages
<wand.image.Image: 3da0549 'PDF' (612x792)>

But when I do the same thing from the PyCharm console, I get:

(Pdb) >? p all_pages
<wand.image.Image: (empty)>
0

@Ilya,

Thanks for your reply.

"SysPath sometimes affects some libraries. E.g. library may need some external plugin and fail to find it due to incorrect sys.path."

If the library were not found, it would give an error on import, yes? Regardless, the libraries are there when I debug the script.

"You can also check environment variables..."

They are the same in both cases.

"After it you may use debugger to debug WandImage internals."

Unfortunately, since wand is a wrapper for ImageMagick, there's no obvious way to step through the code.

What's clear is that PyCharm is somehow breaking the functionality of the script. Pretty frustrating.

 

0

>If the library were not found, it would give an error on import, yes?

It may catch (using try / except) and suppress exception. 

>They are the same in both cases.

os.environ are the same, including PATH? If you use native library, PATH  could be an issue.

0

Hmm, I'll check for try/except in wand.

The PATH appears to be the same, as far as I can tell. I posted sys.path above.

In both cases, the ImageMagic library is located at /usr/local/lib/libMagickWand.dylib. I've verified this with PDB.

0

You also can try to run it from terminal window in PyCharm to check it works. If not, it could be some kind of permissions issue

0

I've run from the PyCharm console, and it fails (see above)

When I try to open the terminal window from PyCharm, I get an error:

Can't Open Local Terminal
java.io.IOException: Exec_tty error:Unknown reason

0

About the terminal: what do you have in the Settings | Terminal -> Shell path ?

Could you please attach the logs ( Help | Show logs..) ?

0

I don't have a Settings | Terminal.

But if I go to File | Default settings | Tools | Terminal I see that Shell path is pointing to a file that doesn't exist: /venv/app_name/bin/terminalactivate

What should it be?

Here are the logs for today:

2017-01-31 08:23:27,778 [404069822] INFO - .intellij.util.EnvironmentUtil - loading shell env: /bin/bash -c . '/Users/robladd/venv/xrdb3/bin/activate';'/Applications/PyCharm.app/Contents/bin/printenv.py' '/private/var/folders/p9/qkmlq50557sgbymmp_vzstv40000gn/T/intellij-shell-env.tmp'
2017-01-31 08:23:27,891 [404069935] INFO - .intellij.util.EnvironmentUtil - shell environment loaded (14 vars)
2017-01-31 08:23:36,031 [404078075] INFO - .intellij.util.EnvironmentUtil - loading shell env: /bin/bash -c . '/Users/robladd/venv/xrdb3/bin/activate';'/Applications/PyCharm.app/Contents/bin/printenv.py' '/private/var/folders/p9/qkmlq50557sgbymmp_vzstv40000gn/T/intellij-shell-env.tmp'
2017-01-31 08:23:36,141 [404078185] INFO - .intellij.util.EnvironmentUtil - shell environment loaded (14 vars)
0

This does not appear to be a sys.path issue in that it's finding. I suspect multiple installs of wand.  Try Intentionally inserting an Exception in line 112 of 

/Users/rcladd/venv/myapp/lib/python2.7/site-packages/wand/sequence.py

Line 112:

   raise SystemError("Bummer")
index
= self.validate_position(index)

Then validate you can get this error when you run ./manage.py outside of PyCharm.  If you don't sys.path has more than one version and that's the issue.

  

0

When I run from ./manage.py outside of PyCharm I get:

SystemError: Bummer
0

Confirmed no sys.path issue.  Something in PyCharm.  Can you create a simple test case to prove it?

0

OK, I created a new PyCharm project `wand_test` using a virtualenv I created in PyCharm.

I did

pip install wand

from the command line (since I cannot open a terminal in PyCharm).

Then this file:

from wand.image import Image as WandImage
from wand.color import Color


def convert_to_thumb(pdf_file, filename):
with open(pdf_file) as f:
image_binary = f.read()
all_pages = WandImage(blob=image_binary)
single_image = all_pages.sequence[0]
with WandImage(single_image) as i:
i.format = 'png'
i.background_color = Color('white')
i.alpha_channel = 'remove'
i.transform(resize='x100')
i.save(filename=filename + '.png')

And I ran it from the PyCharm console:

from make_pdf import convert_to_thumb
convert_to_thumb('diagnosis-of-strawberry-diseases.pdf', 'wand_test')

And got the (same) error:

Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Users/rcladd/PycharmProjects/wand_test/make_pdf.py", line 9, in convert_to_thumb
single_image = all_pages.sequence[0]
File "/Users/rcladd/venv/wand_test/lib/python2.7/site-packages/wand/sequence.py", line 112, in __getitem__
index = self.validate_position(index)
File "/Users/rcladd/venv/wand_test/lib/python2.7/site-packages/wand/sequence.py", line 82, in validate_position
'out of index: {0} (total: {1})'.format(index, length)
IndexError: out of index: 0 (total: 0)

 

0

I would post this to YouTrack  as this is clearly a bug.

0

OK, I'll do that. Can I reference this support ticket in YouTrack?

0

Please sign in to leave a comment.