PyCharm Runserver command issues

Answered
Avatar
Rcladd
Created

I'm trying to make thumbnails from pdfs. To do it, I need to import wand.

It runs fine if I do `./manage.py runserver` from the command line, but if I run from PyCharm, it breaks.

When I step through the code, the problem is that the code used to open the blob is always empty. It's the right class (wand.image) but it's an empty one. The object I pass it is a pdf, but the blob conversion, which produces no error at all, is empty.

Again, if I launch the server from the command line, it works, but if I launch from PyCharm, it breaks.

Here's the code I'm running:

from wand.image import Image as WandImage
from wand.color import Color


def convert_to_thumb(pdf_path, slug):
    with open(pdf_path) as f:
        image_binary = f.read()
    all_pages = WandImage(blob=image_binary) #<-- Here image_binary is a pdf
    single_image = all_pages.sequence[0]     #<-- BOOM! all_pages is a wand.image, but it's empty. Gives an Index error
    with WandImage(single_image) as i:
        i.format = 'png'
        i.background_color = Color('white')  
        i.alpha_channel = 'remove'  
        i.transform(resize='x100')
        save_name = slug + '_pdf_preview.png'
        i.save(filename='/foo/bar/' + save_name)

        return i

Votes

0

Share

19 comments
Avatar
Ilya Kazakevich
Created
Official comment

SysPath sometimes affects some libraries. E.g. library may need some external plugin and fail to find it due to incorrect sys.path.

You can also check environment variables, they may be different when tool launched from PyCharm and from terminal. That is all difference between PyCharm and terminal versions.

After it you may use debugger to debug WandImage internals. 

Avatar
Ilya Kazakevich
Created

Hello.

Not sure what prevents WandImage from working (you probably need to debug it), but it could be syspath issue, since it is the only thing that may be different in PyCharm. Please try to evaluate "sys.path" when debugging or simply print it and compare output when launched from console and from PyCharm. It may give us a clue.

0
Avatar
Rcladd
Created

I actually doubt it's a syspath, since I would get an error when calling or even importing wand, right? There's no error at all until the attempt to sequence the empty wand.image object, which is just a standard Python error.

Here are the sys.path from each environment, the only difference I see is a couple are defined twice in PyCharm and the addition of PyDev.

PyCharm sys.path:

['/Users/rcladd/PycharmProjects/pycharm_myapp',
'/Applications/PyCharm.app/Contents/helpers/pydev',
'/Users/rcladd/PycharmProjects/pycharm_myapp',
'/Applications/PyCharm.app/Contents/helpers/pydev',
'/Users/rcladd/venv/myapp/lib/python27.zip',
'/Users/rcladd/venv/myapp/lib/python2.7',
'/Users/rcladd/venv/myapp/lib/python2.7/plat-darwin',
'/Users/rcladd/venv/myapp/lib/python2.7/plat-mac',
'/Users/rcladd/venv/myapp/lib/python2.7/plat-mac/lib-scriptpackages',
'/Users/rcladd/venv/myapp/lib/python2.7/lib-tk',
'/Users/rcladd/venv/myapp/lib/python2.7/lib-old',
'/Users/rcladd/venv/myapp/lib/python2.7/lib-dynload',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/Users/rcladd/venv/myapp/lib/python2.7/site-packages',
'/srv/web/django_apps/myapp/myapp',
'/srv/web/django_apps/myapp']

CLI sys.path:



['/Users/rcladd/PycharmProjects/pycharm_myapp',
'/Users/rcladd/venv/myapp/lib/python27.zip',
'/Users/rcladd/venv/myapp/lib/python2.7',
'/Users/rcladd/venv/myapp/lib/python2.7/plat-darwin',
'/Users/rcladd/venv/myapp/lib/python2.7/plat-mac',
'/Users/rcladd/venv/myapp/lib/python2.7/plat-mac/lib-scriptpackages',
'/Users/rcladd/venv/myapp/lib/python2.7/lib-tk',
'/Users/rcladd/venv/myapp/lib/python2.7/lib-old',
'/Users/rcladd/venv/myapp/lib/python2.7/lib-dynload',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/Users/rcladd/venv/myapp/lib/python2.7/site-packages',
'/srv/web/django_apps/myapp/myapp',
'/srv/web/django_apps/myapp']
0
Avatar
Ilya Kazakevich
Created

Can you run it as bare Python script in PyCharm instead of Django "run server" and see if it still does not work?

0
Avatar
Rcladd
Created

When I run it as a bare python script in PyCharm it fails with exactly the same error.

Traceback (most recent call last):
 File "<input>", line 1, in <module>
 File "/Users/rcladd/PycharmProjects/pycharm_myapp/publication/make_thumb.py", line 10, in convert_to_thumb
 single_image = all_pages.sequence[0] # Just work on first page
 File "/Users/rcladd/venv/myapp/lib/python2.7/site-packages/wand/sequence.py", line 112, in __getitem__
 index = self.validate_position(index)
 File "/Users/rcladd/venv/myapp/lib/python2.7/site-packages/wand/sequence.py", line 82, in validate_position
 'out of index: {0} (total: {1})'.format(index, length)
IndexError: out of index: 0 (total: 0)

When I run the same python command from the shell (same virtualenv) it runs fine.

0
Avatar
Rcladd
Created

Here's some more info:

When I run from CLI and use `pdb.set_trace()` to check the value of `all_pages` i get this

(Pdb) p all_pages
<wand.image.Image: 3da0549 'PDF' (612x792)>

But when I do the same thing from the PyCharm console, I get:

(Pdb) >? p all_pages
<wand.image.Image: (empty)>
0
Avatar
Rcladd
Created

@Ilya,

Thanks for your reply.

"SysPath sometimes affects some libraries. E.g. library may need some external plugin and fail to find it due to incorrect sys.path."

If the library were not found, it would give an error on import, yes? Regardless, the libraries are there when I debug the script.

"You can also check environment variables..."

They are the same in both cases.

"After it you may use debugger to debug WandImage internals."

Unfortunately, since wand is a wrapper for ImageMagick, there's no obvious way to step through the code.

What's clear is that PyCharm is somehow breaking the functionality of the script. Pretty frustrating.

 

0
Avatar
Ilya Kazakevich
Created

>If the library were not found, it would give an error on import, yes?

It may catch (using try / except) and suppress exception. 

>They are the same in both cases.

os.environ are the same, including PATH? If you use native library, PATH  could be an issue.

0
Avatar
Rcladd
Created

Hmm, I'll check for try/except in wand.

The PATH appears to be the same, as far as I can tell. I posted sys.path above.

In both cases, the ImageMagic library is located at /usr/local/lib/libMagickWand.dylib. I've verified this with PDB.

0
Avatar
Ilya Kazakevich
Created

You also can try to run it from terminal window in PyCharm to check it works. If not, it could be some kind of permissions issue

0
Avatar
Rcladd
Created

I've run from the PyCharm console, and it fails (see above)

When I try to open the terminal window from PyCharm, I get an error:

Can't Open Local Terminal
java.io.IOException: Exec_tty error:Unknown reason

0
Avatar
Dmitry Trofimov
Created

About the terminal: what do you have in the Settings | Terminal -> Shell path ?

Could you please attach the logs ( Help | Show logs..) ?

0
Avatar
Rcladd
Created

I don't have a Settings | Terminal.

But if I go to File | Default settings | Tools | Terminal I see that Shell path is pointing to a file that doesn't exist: /venv/app_name/bin/terminalactivate

What should it be?

Here are the logs for today:

2017-01-31 08:23:27,778 [404069822] INFO - .intellij.util.EnvironmentUtil - loading shell env: /bin/bash -c . '/Users/robladd/venv/xrdb3/bin/activate';'/Applications/PyCharm.app/Contents/bin/printenv.py' '/private/var/folders/p9/qkmlq50557sgbymmp_vzstv40000gn/T/intellij-shell-env.tmp'
2017-01-31 08:23:27,891 [404069935] INFO - .intellij.util.EnvironmentUtil - shell environment loaded (14 vars)
2017-01-31 08:23:36,031 [404078075] INFO - .intellij.util.EnvironmentUtil - loading shell env: /bin/bash -c . '/Users/robladd/venv/xrdb3/bin/activate';'/Applications/PyCharm.app/Contents/bin/printenv.py' '/private/var/folders/p9/qkmlq50557sgbymmp_vzstv40000gn/T/intellij-shell-env.tmp'
2017-01-31 08:23:36,141 [404078185] INFO - .intellij.util.EnvironmentUtil - shell environment loaded (14 vars)
0
Avatar
Steven Klass
Created

This does not appear to be a sys.path issue in that it's finding. I suspect multiple installs of wand.  Try Intentionally inserting an Exception in line 112 of 

/Users/rcladd/venv/myapp/lib/python2.7/site-packages/wand/sequence.py

Line 112:

   raise SystemError("Bummer")
   index = self.validate_position(index)

Then validate you can get this error when you run ./manage.py outside of PyCharm.  If you don't sys.path has more than one version and that's the issue.

  

0
Avatar
Rcladd
Created

When I run from ./manage.py outside of PyCharm I get:

SystemError: Bummer
0
Avatar
Steven Klass
Created

Confirmed no sys.path issue.  Something in PyCharm.  Can you create a simple test case to prove it?

0
Avatar
Rcladd
Created

OK, I created a new PyCharm project `wand_test` using a virtualenv I created in PyCharm.

I did

pip install wand

from the command line (since I cannot open a terminal in PyCharm).

Then this file:

from wand.image import Image as WandImage
from wand.color import Color


def convert_to_thumb(pdf_file, filename):
    with open(pdf_file) as f:
        image_binary = f.read()
    all_pages = WandImage(blob=image_binary)
    single_image = all_pages.sequence[0]    
    with WandImage(single_image) as i:
        i.format = 'png'
        i.background_color = Color('white') 
        i.alpha_channel = 'remove'          
        i.transform(resize='x100')
        i.save(filename=filename + '.png')

And I ran it from the PyCharm console:

from make_pdf import convert_to_thumb
convert_to_thumb('diagnosis-of-strawberry-diseases.pdf', 'wand_test')

And got the (same) error:

Traceback (most recent call last):
 File "<input>", line 1, in <module>
 File "/Users/rcladd/PycharmProjects/wand_test/make_pdf.py", line 9, in convert_to_thumb
     single_image = all_pages.sequence[0]
 File "/Users/rcladd/venv/wand_test/lib/python2.7/site-packages/wand/sequence.py", line 112, in __getitem__
     index = self.validate_position(index)
 File "/Users/rcladd/venv/wand_test/lib/python2.7/site-packages/wand/sequence.py", line 82, in validate_position
    'out of index: {0} (total: {1})'.format(index, length)
IndexError: out of index: 0 (total: 0)

 

0
Avatar
Steven Klass
Created

I would post this to YouTrack  as this is clearly a bug.

0
Avatar
Rcladd
Created

OK, I'll do that. Can I reference this support ticket in YouTrack?

0

Please sign in to leave a comment.