Running jupyter notebook with remote Docker interpreter

已回答

Hello,

 

I cannot manage to run Jupyter notebook with a remote Docker interpreter. The notebook run configuration keeps asking: "Please select local python interpreter"... I have PyCharm Pro 2017.2.3.

The Remote interpreter works perfectly when I run a standard python program. It just doesn't work for notebooks. If I "force execute" the run configuration, it runs the server but I cannot manage to connect the notebook (it works through the provided link in a browser though but not with the notebook "Run cell" button) even though I can see the following log:

c4ae62f9f886:/usr/bin/python3 -u /opt/.pycharm_helpers/pycharm/pycharm_load_entry_point.py notebook --no-browser --ip 0.0.0.0 --port 8888 --port-retries=0 --allow-root
[TerminalIPythonApp] WARNING | Subcommand `ipython notebook` is deprecated and will be removed in future versions.
[TerminalIPythonApp] WARNING | You likely want to use `jupyter notebook` in the future
[I 15:01:19.080 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
[I 15:01:19.101 NotebookApp] Serving notebooks from local directory: /opt/project
[I 15:01:19.101 NotebookApp] 0 active kernels
[I 15:01:19.101 NotebookApp] The Jupyter Notebook is running at:
[I 15:01:19.101 NotebookApp] http://0.0.0.0:8888/?token=6ba64aba0f80d6ea1d4b0f35283530615b58778a7934e1bf
[I 15:01:19.101 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 15:01:19.102 NotebookApp]
    
    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://0.0.0.0:8888/?token=6ba64aba0f80d6ea1d4b0f35283530615b58778a7934e1bf

 

  • Here is my launch configuration (I tried with 127.0.0.1 and * too):

  • Here is my docker-compose file:
version: '2'
services:
python:
build: .
command: /usr/bin/python3
devices:
- "/dev/snd:/dev/snd"
environment:
- DISPLAY=$DISPLAY
image: gaia:latest
port:
- "127.0.0.1:8888:8888"
privileged: true
volumes:
- /dev/dri:/dev/dri
- /tmp/.X11-unix:/tmp/.X11-unix
- /dev/shm:/dev/shm
- /etc/machine-id:/etc/machine-id
- /run/dbus:/run/dbus

  • And my Dockerfile looks like this:
ARG BASE_IMAGE=ubuntu:artful
FROM $BASE_IMAGE

# System libraries
RUN apt-get update \
&& apt-get install --yes --no-install-recommends \
build-essential \
cmake \
curl \
fort77 \
gfortran \
git \
gstreamer1.0-alsa \
gstreamer1.0-clutter \
gstreamer1.0-fluendo-mp3 \
gstreamer1.0-libav \
gstreamer1.0-nice \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-base \
gstreamer1.0-plugins-base-apps \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-ugly \
gstreamer1.0-pulseaudio \
gstreamer1.0-tools \
gstreamer1.0-vaapi \
gstreamer1.0-x \
jq \
libatlas-base-dev \
libavcodec-dev \
libavformat-dev \
libdc1394-22-dev \
libboost-python-dev \
libjpeg-dev \
libgstreamer1.0-dev \
libgstreamer-plugins-base1.0-dev \
libgtk2.0-dev \
libgtk-3-dev \
libpng-dev \
libswscale-dev \
libtbb2 \
libtbb-dev \
libtiff-dev \
libv4l-dev \
libx11-dev \
libx264-dev \
libxvidcore-dev \
pkg-config \
python3-dev \
python3-numpy \
python3-pip \
python3-scipy \
python3-setuptools \
python3-wheel \
vim \
wget \
&& apt-get -y autoclean \
&& apt-get -y autoremove \
&& rm -rf /var/lib/apt/lists/*


# OpenCV
RUN cd /root \
&& wget https://github.com/opencv/opencv/archive/3.3.0.tar.gz -O opencv.tar.gz \
&& tar zxf opencv.tar.gz && rm -f opencv.tar.gz \
&& wget https://github.com/opencv/opencv_contrib/archive/3.3.0.tar.gz -O contrib.tar.gz \
&& tar zxf contrib.tar.gz && rm -f contrib.tar.gz \
&& cd opencv-3.3.0 \
&& mkdir build \
&& cd build \
&& cmake \
-D BUILD_DOCS=OFF \
-D BUILD_EXAMPLES=OFF \
-D BUILD_TESTS=OFF \
-D BUILD_opencv_python2=OFF \
-D BUILD_opencv_python3=ON \
-D CMAKE_BUILD_TYPE=Release \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D CMAKE_CXX_FLAGS="-O3 -funsafe-math-optimizations" \
-D CMAKE_C_FLAGS="-O3 -funsafe-math-optimizations" \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D OPENCV_EXTRA_MODULES_PATH=/root/opencv_contrib-3.3.0/modules \
-D WITH_JASPER=OFF \
-D WITH_LIBV4L=ON \
-D WITH_TBB=ON \
.. \
&& make \
&& make install \
&& cd /root \
&& rm -rf opencv-3.3.0 opencv_contrib-3.3.0 \
&& ldconfig


# Dlib
RUN wget http://dlib.net/files/dlib-19.7.tar.bz2 \
&& tar xvf dlib-19.7.tar.bz2 \
&& cd dlib-19.7/ \
&& mkdir build \
&& cd build \
&& cmake \
-D CMAKE_BUILD_TYPE=Release \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D USE_AVX_INSTRUCTIONS=1 \
.. \
&& cmake --build . --config Release \
&& make install \
&& ldconfig \
&& cd .. \
&& python3 setup.py install --yes USE_AVX_INSTRUCTIONS



# Python env
RUN pip3 install --help
RUN python3 -m pip install --upgrade pip \
&& pip3 install --user \
face_recognition \
imutils \
ipython \
jupyter \
matplotlib \
numpy \
pandas \
scikit-image \
scikit-learn

ENV PATH="~/.local/bin:${PATH}"

What am I doing wrong?

2
Avatar
Permanently deleted user

The only related bug I found is the following: https://youtrack.jetbrains.com/issue/PY-24739

I also have a weird error in the logs (knowing that I'm on Linux Arch):

2017-09-30 20:19:21,183 [  33266]   INFO - figurations.GeneralCommandLine - Cannot run program "docker-machine": error=2, No such file or directory 
java.io.IOException: Cannot run program "docker-machine": error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
    at com.intellij.execution.configurations.GeneralCommandLine.startProcess(GeneralCommandLine.java:420)
    at com.intellij.execution.configurations.GeneralCommandLine.createProcess(GeneralCommandLine.java:387)
    at com.intellij.execution.process.OSProcessHandler.<init>(OSProcessHandler.java:45)
    at com.intellij.execution.process.CapturingProcessHandler.<init>(CapturingProcessHandler.java:38)
    at com.intellij.docker.machine.DockerMachine.exec(DockerMachine.java:98)
    at com.intellij.docker.machine.DockerMachine.version(DockerMachine.java:252)
    at com.intellij.docker.machine.DockerMachineUtil.checkValidExecutable(DockerMachineUtil.java:70)
    at com.intellij.docker.machine.DockerMachineUtil.tryFindDockerMachineExecutable(DockerMachineUtil.java:40)
    at com.intellij.docker.DockerSettings.getDockerMachinePath(DockerSettings.java:50)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.util.xmlb.PropertyAccessor.read(PropertyAccessor.java:58)
    at com.intellij.util.xmlb.SkipDefaultsSerializationFilter.equal(SkipDefaultsSerializationFilter.java:32)
    at com.intellij.util.xmlb.BeanBinding.serializeInto(BeanBinding.java:101)
    at com.intellij.util.xmlb.BeanBinding.serialize(BeanBinding.java:85)
    at com.intellij.configurationStore.XmlSerializer.serialize(xmlSerializer.kt:55)
    at com.intellij.configurationStore.XmlSerializer.serialize$default(xmlSerializer.kt:49)
    at com.intellij.configurationStore.SaveSessionBaseKt.serializeState(SaveSessionBase.kt:57)
    at com.intellij.configurationStore.SaveSessionBase.setState(SaveSessionBase.kt:29)
    at com.intellij.configurationStore.StateStorageManagerImpl$startExternalization$1.setState(StateStorageManagerImpl.kt:434)
    at com.intellij.configurationStore.ComponentStoreImpl.a(ComponentStoreImpl.kt:229)
    at com.intellij.configurationStore.ComponentStoreImpl.save(ComponentStoreImpl.kt:159)
    at com.intellij.openapi.components.impl.stores.StoreUtil.save(StoreUtil.java:49)
    at com.intellij.openapi.application.impl.ApplicationImpl.saveSettings(ApplicationImpl.java:1437)
    at com.intellij.openapi.components.impl.stores.StoreUtil.saveDocumentsAndProjectsAndApp(StoreUtil.java:129)
    at com.intellij.openapi.application.impl.ApplicationImpl.saveAll(ApplicationImpl.java:1449)
    at com.intellij.ide.SaveAndSyncHandlerImpl.saveProjectsAndDocuments(SaveAndSyncHandlerImpl.java:126)
    at com.intellij.ide.SaveAndSyncHandlerImpl$2.a(SaveAndSyncHandlerImpl.java:94)
    at com.intellij.openapi.application.TransactionGuardImpl.a(TransactionGuardImpl.java:86)
    at com.intellij.openapi.application.TransactionGuardImpl.a(TransactionGuardImpl.java:109)
    at com.intellij.openapi.application.TransactionGuardImpl.submitTransaction(TransactionGuardImpl.java:118)
    at com.intellij.openapi.application.TransactionGuard.submitTransaction(TransactionGuard.java:122)
    at com.intellij.ide.SaveAndSyncHandlerImpl$2.onFrameDeactivated(SaveAndSyncHandlerImpl.java:92)
    at com.intellij.ide.FrameStateManagerImpl.b(FrameStateManagerImpl.java:84)
    at com.intellij.ide.FrameStateManagerImpl.access$500(FrameStateManagerImpl.java:32)
    at com.intellij.ide.FrameStateManagerImpl$2.a(FrameStateManagerImpl.java:70)
    at com.intellij.util.concurrency.QueueProcessor.runSafely(QueueProcessor.java:246)
    at com.intellij.util.Alarm$Request.runSafely(Alarm.java:417)
    at com.intellij.util.Alarm$Request.access$700(Alarm.java:344)
    at com.intellij.util.Alarm$Request$1.run(Alarm.java:384)
    at com.intellij.openapi.application.TransactionGuardImpl$2.run(TransactionGuardImpl.java:314)
    at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.a(LaterInvocator.java:416)
    at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.run(LaterInvocator.java:399)
    at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:762)
    at java.awt.EventQueue.access$500(EventQueue.java:98)
    at java.awt.EventQueue$3.run(EventQueue.java:715)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:732)
    at com.intellij.ide.IdeEventQueue.e(IdeEventQueue.java:821)
    at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:649)
    at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:365)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Caused by: java.io.IOException: error=2, No such file or directory
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
    at java.lang.ProcessImpl.start(ProcessImpl.java:134)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
    ... 62 more

 

 

0

Hi! remote interpreters are not supported for Jupyter Notebooks for now unfortunately. Corresponding feature request: PY-26328.

0
Avatar
Permanently deleted user

Thanks for the ref. That would be great to have that!

0

请先登录再写评论。