Cannot read packages in docker interpreter (Tar error?)

Recently, using a dockerized python environment with some installed packages has lead to problems. PyCharm does not recognize any installed packages any more. In the log, I found the following error. While I have rebuilt the container recently, nothing but some code changes have occurred. I'm using PyCharm 2018.1 on Arch Linux.

```

2018-05-03 10:26:59,889 [ 130796] INFO - .skeletons.PySkeletonGenerator - Retrieving binary module list took 1113 ms
2018-05-03 10:26:59,953 [ 130860] INFO - .CloudSilentLoggingHandlerImpl - Creating container...
2018-05-03 10:27:06,851 [ 137758] INFO - .skeletons.PySkeletonGenerator - Failed to download remote sources /usr/lib/python2.7 to local directory /home/dke/.PyCharm2018.1/system/remote_sources/-1088670127/
com.jetbrains.python.sdk.InvalidSdkException: Failed to download remote sources /usr/lib/python2.7 to local directory /home/dke/.PyCharm2018.1/system/remote_sources/-1088670127/
at com.jetbrains.python.docker.PyDockerSkeletonGeneratorBase.downloadSources(PyDockerSkeletonGeneratorBase.kt:148)
at com.jetbrains.python.docker.PyDockerSkeletonGeneratorBase.refreshGeneratedSkeletons(PyDockerSkeletonGeneratorBase.kt:95)
at com.jetbrains.python.sdk.skeletons.PySkeletonRefresher.regenerateSkeletons(PySkeletonRefresher.java:352)
at com.jetbrains.python.sdk.skeletons.PySkeletonRefresher.refreshSkeletonsOfSdk(PySkeletonRefresher.java:124)
at com.jetbrains.python.sdk.PythonSdkUpdater$2.run(PythonSdkUpdater.java:181)
at com.intellij.openapi.progress.impl.CoreProgressManager$TaskRunnable.run(CoreProgressManager.java:750)
at com.intellij.openapi.progress.impl.CoreProgressManager.a(CoreProgressManager.java:157)
at com.intellij.openapi.progress.impl.CoreProgressManager.a(CoreProgressManager.java:580)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:525)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:85)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:144)
at com.intellij.openapi.progress.impl.CoreProgressManager$4.run(CoreProgressManager.java:395)
at com.intellij.openapi.application.impl.ApplicationImpl$1.run(ApplicationImpl.java:305)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Error detected parsing the header
at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.getNextTarEntry(TarArchiveInputStream.java:280)
at com.intellij.docker.remote.run.runtime.RemoteDockerApplicationRuntime.extractTar(RemoteDockerApplicationRuntime.java:330)
at com.intellij.docker.remote.run.runtime.RemoteDockerApplicationRuntime.extractTar(RemoteDockerApplicationRuntime.java:320)
at com.intellij.docker.remote.run.runtime.RemoteDockerApplicationRuntime.downloadDirectory(RemoteDockerApplicationRuntime.java:306)
at com.intellij.docker.remote.run.runtime.RemoteDockerApplicationRuntime.downloadDirectoryAndRemove(RemoteDockerApplicationRuntime.java:227)
at com.jetbrains.python.docker.PyDockerSkeletonGenerator.doDownloadSources(PyDockerSkeletonGenerator.java:217)
at com.jetbrains.python.docker.PyDockerSkeletonGeneratorBase.downloadSources(PyDockerSkeletonGeneratorBase.kt:145)
... 17 more
Caused by: java.lang.IllegalArgumentException: Invalid byte 41 at offset 0 in ')�;q��I' len=8
at org.apache.commons.compress.archivers.tar.TarUtils.parseOctal(TarUtils.java:137)
at org.apache.commons.compress.archivers.tar.TarUtils.parseOctalOrBinary(TarUtils.java:168)
at org.apache.commons.compress.archivers.tar.TarArchiveEntry.parseTarHeader(TarArchiveEntry.java:1007)
at org.apache.commons.compress.archivers.tar.TarArchiveEntry.parseTarHeader(TarArchiveEntry.java:996)
at org.apache.commons.compress.archivers.tar.TarArchiveEntry.<init>(TarArchiveEntry.java:326)
at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.getNextTarEntry(TarArchiveInputStream.java:278)
... 23 more
2018-05-03 10:27:06,875 [ 137782] INFO - .skeletons.PySkeletonGenerator - Sources and skeletons are loaded in 8099 millis.
2018-05-03 10:27:06,878 [ 137785] INFO - ns.python.sdk.PythonSdkUpdater - Bogus sys.path entry /home/dke/.PyCharm2018.1/system/remote_sources/-1088670127/201544331
2018-05-03 10:27:06,879 [ 137786] INFO - ns.python.sdk.PythonSdkUpdater - Bogus sys.path entry /home/dke/.PyCharm2018.1/system/remote_sources/-1088670127/-2145795944
2018-05-03 10:27:06,879 [ 137786] INFO - ns.python.sdk.PythonSdkUpdater - Bogus sys.path entry /home/dke/.PyCharm2018.1/system/remote_sources/-1088670127/-154863933
2018-05-03 10:27:06,879 [ 137786] INFO - ns.python.sdk.PythonSdkUpdater - Bogus sys.path entry /home/dke/.PyCharm2018.1/system/remote_sources/-1088670127/918732847
2018-05-03 10:27:06,880 [ 137787] INFO - ns.python.sdk.PythonSdkUpdater - Bogus sys.path entry /home/dke/.PyCharm2018.1/system/remote_sources/-1088670127/1560306194
2018-05-03 10:27:06,880 [ 137787] INFO - ns.python.sdk.PythonSdkUpdater - Binary skeletons directory for SDK Remote Python 2.7.14 Docker (<container>:latest) (docker://<container>:latest/python)): /home/dke/.PyCharm2018.1/system/python_stubs/-1088670127
2018-05-03 10:27:06,880 [ 137787] INFO - ns.python.sdk.PythonSdkUpdater - User skeletons directory for SDK Remote Python 2.7.14 Docker (<container>:latest) (docker://<container>:latest/python)): /opt/pycharm-professional/helpers/python-skeletons
2018-05-03 10:27:06,883 [ 137790] INFO - ns.python.sdk.PythonSdkUpdater - Performing background scan of packages for SDK Remote Python 2.7.14 Docker (<container>:latest) (docker://<container>:latest/python)
2018-05-03 10:27:06,924 [ 137831] INFO - .CloudSilentLoggingHandlerImpl - Creating container...

```

0
5 comments
Avatar
Permanently deleted user

Test with other containers straight from the Docker registry have the same problem. What could be the issue here? Can I work around it somehow? The containers themselves work fine and contain all the packages.

0
Avatar
Permanently deleted user

I have the same exact issue (PyCharm 2018.1.2 on Manjaro):

```

2018-05-04 18:01:44,855 [ 102087] INFO - .skeletons.PySkeletonGenerator - Retrieving binary module list took 2278 ms
2018-05-04 18:01:46,561 [ 103793] INFO - .skeletons.PySkeletonGenerator - Failed to download remote sources /usr/src/app/eggs/argon2_cffi-18.1.0-py3.6-linux-x86_64.egg to local directory /home/jzvelc/.PyCharm2018.1/system/remote_sources/-1903142790/
com.jetbrains.python.sdk.InvalidSdkException: Failed to download remote sources /usr/src/app/eggs/argon2_cffi-18.1.0-py3.6-linux-x86_64.egg to local directory /home/jzvelc/.PyCharm2018.1/system/remote_sources/-1903142790/
at com.jetbrains.python.docker.PyDockerSkeletonGeneratorBase.downloadSources(PyDockerSkeletonGeneratorBase.kt:148)
at com.jetbrains.python.docker.PyDockerSkeletonGeneratorBase.refreshGeneratedSkeletons(PyDockerSkeletonGeneratorBase.kt:95)
at com.jetbrains.python.sdk.skeletons.PySkeletonRefresher.regenerateSkeletons(PySkeletonRefresher.java:352)
at com.jetbrains.python.sdk.skeletons.PySkeletonRefresher.refreshSkeletonsOfSdk(PySkeletonRefresher.java:124)
at com.jetbrains.python.sdk.PythonSdkUpdater$2.run(PythonSdkUpdater.java:181)
at com.intellij.openapi.progress.impl.CoreProgressManager$TaskRunnable.run(CoreProgressManager.java:750)
at com.intellij.openapi.progress.impl.CoreProgressManager.a(CoreProgressManager.java:157)
at com.intellij.openapi.progress.impl.CoreProgressManager.a(CoreProgressManager.java:580)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:525)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:85)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:144)
at com.intellij.openapi.progress.impl.CoreProgressManager$4.run(CoreProgressManager.java:395)
at com.intellij.openapi.application.impl.ApplicationImpl$1.run(ApplicationImpl.java:305)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Error detected parsing the header
at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.getNextTarEntry(TarArchiveInputStream.java:280)
at com.intellij.docker.remote.run.runtime.RemoteDockerApplicationRuntime.extractTar(RemoteDockerApplicationRuntime.java:330)
at com.intellij.docker.remote.run.runtime.RemoteDockerApplicationRuntime.extractTar(RemoteDockerApplicationRuntime.java:320)
at com.intellij.docker.remote.run.runtime.RemoteDockerApplicationRuntime.downloadDirectory(RemoteDockerApplicationRuntime.java:306)
at com.intellij.docker.remote.run.runtime.RemoteDockerApplicationRuntime.downloadDirectoryAndRemove(RemoteDockerApplicationRuntime.java:227)
at com.jetbrains.python.docker.PyDockerComposeSkeletonGenerator.doDownloadSources(PyDockerComposeSkeletonGenerator.java:328)
at com.jetbrains.python.docker.PyDockerSkeletonGeneratorBase.downloadSources(PyDockerSkeletonGeneratorBase.kt:145)
... 17 more
Caused by: java.lang.IllegalArgumentException: Invalid byte 75 at offset 0 in 'K���Dy�D' len=8
at org.apache.commons.compress.archivers.tar.TarUtils.parseOctal(TarUtils.java:137)
at org.apache.commons.compress.archivers.tar.TarUtils.parseOctalOrBinary(TarUtils.java:168)
at org.apache.commons.compress.archivers.tar.TarArchiveEntry.parseTarHeader(TarArchiveEntry.java:1011)
at org.apache.commons.compress.archivers.tar.TarArchiveEntry.parseTarHeader(TarArchiveEntry.java:996)
at org.apache.commons.compress.archivers.tar.TarArchiveEntry.<init>(TarArchiveEntry.java:326)
at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.getNextTarEntry(TarArchiveInputStream.java:278)
... 23 more

```

0
Avatar
Permanently deleted user

Hmm, looks like an issue with ArchLinux (and by proxy, Manjaro) then. I guess I'll have to take a look at what updates were installed in the last few days. Strange issue.

0
Avatar
Permanently deleted user

Same issue there (Arch/Manjaro x86_64). For PyCharm 2018.1.2, 2018.1.1, 2017.3.5, 2017.1.7

Doesn't work either for images with alpine or debian (not tested on others).

Downgrading local tar doesnt' work, also downgrading IDE and both to last known working versions (2017.3.5., tar 1.29-2).

Only difference to aboves are other invalid byte number:

```
Caused by: java.lang.IllegalArgumentException: Invalid byte 125 at offset 0 in '}
�րq' len=8
```

0
Avatar
Permanently deleted user

"The problem occurred when in the latest Docker versions ContainerArchive Docker Engine API operation started to return compressed stream of container resources."

Solution: downgrade docker to stable 18.03.0-ce version.

https://youtrack.jetbrains.com/issue/PY-29585#u=1523907633941

0

Please sign in to leave a comment.