How to enable syntax check for PySpark

I would like to use Pycharm to develop PySpark applications; I already added to the envirioment the PySpark "package", but in the editor I have syntax errors even if the code works fine.

For example I'm using this code line to print:

print "value of variable a: %i, value of variable b: %i" % (a, b);

it's highlited in red as an error, but it works when I launch the program with spark-submit

1 comment

Hi,

There is a known issue with related to PySpark: https://youtrack.jetbrains.com/issue/PY-28339

In the comments there are some users experiencing the same issue with syntax highlight, and suggested workaround is to change shell script name to python3. Also there are known issues with code completion.

Please leave a vote in that ticket, and also you can vote for the native PySpark support: https://youtrack.jetbrains.com/issue/PY-29811

0

Please sign in to leave a comment.