Debugging Flask under gunicorn in docker container

Answered

Hi,

My projects is starting up multiple containers with docker-compose the flask app is running under gunicorn, now is it possible to use pycharm debugger to set breakpoints and use rest of the IDE?

Currently the only way to debug the project is to run containers with docker-compose up command and use pdb.set_trace() to set a breakpoint.

 

1
15 comments

Hi! If your project has a complicated setup, you can try to use Remote Debug Server (https://www.jetbrains.com/help/pycharm/remote-debugging.html#python-debug-server).

0
Avatar
Permanently deleted user

Hi Elizabeth

Thank you for your reply! Unfortunately it did not work since pycharm is throwing error message which state the following: "Failed to find free socket port". This is happening because i am trying to connect remote debugger to docker container which is up on my local machine and uses 5009 port. Could you please comment on this?

0

Streak Flash Are you using Remote Interpreter or Remote Debug Server?

1
Avatar
Permanently deleted user

I am using local virtual environment and a  remote debug server. i've put the .egg file in my docker container and included required library in my python file. When i hit the debug button debugger starts up and waits for connection to my given port. but then i am unable to start the docker container since the port is locked by pychamr debugger

0

Hi Streak Flash! Is it possible to access host from inside your container? Could you please provide more details about network and port configuration you made? Doesn't selecting another port for Remote Debug Server work? I made a quick setup on mac (btw what is your OS?):

  • pydevd.settrace('docker.for.mac.localhost', port=1234, stdoutToServer=True, stderrToServer=True) inside a Python script
  • interpreter based on the python:latest image (no specific port or network configuration made)
  • run Remote Debug Server
  • run Python script in PyCharm
  • successful connection
0
Avatar
Permanently deleted user

Hi Pavel,

Sorry for late response!

I am also running MacOS, please see below a part of my Dockerfile. 


COPY pycharm-debug-py3k.egg /app
EXPOSE 80
WORKDIR /app
# Execute the app
ENTRYPOINT ["/usr/local/bin/newrelic-admin", "run-program"]
CMD ["/usr/local/bin/gunicorn", "app:application", "-b", "0.0.0.0:80", "--reload"]


And a slice from docker-compose.yml

services:
my-service:
build: .
image: my-service
ports:
- "5009:80"
volumes:
- .:/app
environment:
...
links:
- mongo
networks:
- default
- my_network

networks:
my_network:
external: true
0
Avatar
Permanently deleted user

I have figured this out!

 

I just had to set "docker-compose" as remote interpreter for my project and then create a python runnable where Script path="/usr/local/bin/gunicorn" and Script parameters="app:application -t 0 <and so on>"

 

1
Avatar
Permanently deleted user

Streak Flash,

Could you provide your complete config you ended up with? I am running into a similar issue and would like to see if I can reproduce what you did when you got it working.

Sincerely,

Robert

0
Avatar
Permanently deleted user

Robert - Here is complete flow to set up the debugger

1. Pycharm -> Preferences -> Build, Execution, Deployment -> Docker
* Press + (Add)
* Connect to Docker daemon with: Docker for Mac
* Navigate to Docker -> Tools
* Docker Machine executable: `/usr/local/bin/docker-machine`
* Docker Compose executable: `/usr/local/bin/docker-compose`
2. Navigate to Preferences -> Project -> Project Interpreter
3. Locate “gear” icon -> Add remote -> Docker Compose
1. Server: Choose your previously created Docker server
2. Configuration file: Locate your project’s `docker-compose.yml` file
3. Service: Set your main service here
4. If not set, select newly created Remote Python interpreter in “Project Interpreter” field.
5. Run -> Edit Configurations - Add -> Python
6. Setup appropriate fields as follows
* Script path: `/usr/local/bin/gunicorn`
* Parameters: `app:application -b 0.0.0.0:80 -t 0`
7. Press OK button
8. Now you should be able to use PyCharm debugger

2
Avatar
Permanently deleted user

Streak Flash--

Thank you for taking the time to get me that info. I will take a look and see if I can get that to to work on my box today!

Sincerely,

Robert

0

Hello~. Thanks so much to Streak Flash, but I'm having trouble debugging my FastApi Flask application.

I was able to successfully set everything up, but code execution is not pausing at my breakpoints.

The request I made:

After request, you can see that my setup is working in that it shows this request:

`ccdh-api | INFO: 172.19.0.1:59612 - "GET /models/ HTTP/1.1" 200 OK`

However, as you can see, I've set a breakpoint on line 121 inside of `get_models()`, which is what is called when I access http://localhost:7070/models. But you can see that it did not pause on line 121. Why?

Here's my debug configuration:

Edit: I added path mapping to debug config:

`/Users/joeflack4/projects/ccdh-terminology-service=/app`

Here's my interpreter:

Edit: I added path mapping to interpreter config:

`/Users/joeflack4/projects/ccdh-terminology-service=/app`

---

Can someone help me? Why is it not pausing execution when it looks like I've set everything up correctly?

0
Avatar
Permanently deleted user

@Joseph E. Flack IV the setup directly depends on how your Dockerfile and docker-compose.yml you have chosen are configured. Check application path mapping in your Dockerfile maybe its not /app and check the run command in the docker-compose.

0

Hey there Streak~. Wow. You're still active in this thread! Appreciated.

Here's what I have. I didn't think there was anything else relevant in this way that would change what I have said up.

## My mapping

I SSH'ed into my container and did `pwd` inside of the project directory. It is indeed located at `/app`.

## Docker-compose

```yml

# Production environment
version: "3"
services:
ccdh-api:
build:
dockerfile: ./docker/ccdh-api/Dockerfile
context: ..
container_name: ccdh-api
restart: on-failure
ports:
- "7070:8000"
depends_on:
ccdh-neo4j:
condition: service_healthy
ccdh-redis:
condition: service_healthy

ccdh-redis:
image: "redis:alpine"
container_name: ccdh-redis
ports:
- "6379:6379"
volumes:
- ./redis-data:/var/lib/redis
healthcheck:
test: [ "CMD", "redis-cli", "ping" ]
interval: 1s
timeout: 10s
retries: 30

ccdh-neo4j:
image: neo4j:4.1.3
container_name: ccdh-neo4j
restart: always
volumes:
- ./neo4j/logs:/logs
- ./neo4j/data:/data
- ./neo4j/plugins:/var/lib/neo4j/plugins
- ./neo4j/conf:/var/lib/neo4j/conf
- ./neo4j/import:/var/lib/neo4j/import
- ./neo4j/wrapper.sh:/opt/wrapper.sh
- ./neo4j/initialize_neo4j.sh:/opt/initialize_neo4j.sh
- ./neo4j/cyphers:/cyphers
ports:
- "7687:7687"
environment:
NEO4J_AUTH: $NEO4J_USERNAME/$NEO4J_PASSWORD
NEO4J_dbms_memory_heap_max__size: 8G
env_file:
- .env
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:7474"]
interval: 15s
timeout: 10s
retries: 10
entrypoint: /opt/wrapper.sh

volumes:
ccdh-neo4j:
ccdh-redis:

```

### Dockerfile and 'run' command (`ccdh-api` container)

'run' command (I think this is what you want to see)

`

CMD pipenv run uvicorn ccdh.api.app:app $ROOT_PATH --host 0.0.0.0 --port 8000

`

Dockerfile

```

FROM python:3.8

RUN pip install pipenv
COPY Pipfile* /tmp/
RUN cd /tmp && pipenv lock --requirements > requirements.txt
RUN pip install -r /tmp/requirements.txt
COPY . /app
COPY docker/.env /app/.env

WORKDIR /app
CMD pipenv run uvicorn ccdh.api.app:app $ROOT_PATH --host 0.0.0.0 --port 8000

```

0

I manged to make it work partially. I can run my application but I can't debug. I run in the debug mode and it won't stop at my breakpoints. This is my settings. Any clue on why breakpoints are not working?

0
Hello Mauricio, 
Please create an issue on YouTrack so we can properly investigate it. PyCharm logs (Help | Collect Logs and Diagnostic Data), dockerfile, and docker-compose.yml will help us to process it faster.
0

Please sign in to leave a comment.