How to customize gdb target connection timeout

Answered

I have been using GDB and Openocd to debug my embedded platform. GDB connects to GDB server run by Openocd, which is in turn invoked using a customized version of OpenOCD + STM32CubeMX plugin. Everything was working great until the latency of remote connection increased. The openocd startup script now takes longer to execute so the remote connection is apparently timing out. I can reproduce the issue without CLion on the command line and can solve it by setting a higher connection timeout by either sending gdb command "set remotetimeout <num>" before target connection, or by providing the commandline option "-l <timeout>" when starting gdb. How can I do either of these while starting gdb from CLion?

One option I tried was to create .gdbinit and add  "set remotetimeout" there. This works, but is not the ideal solution for me as I will have to instruct a large team of people to do this. I am looking some solution through the Openocd plugin, which I can deploy without much disruption. Can I pass in commandline options from the plugin while starting gdb? Or could I somehow customize the plugin to run a set of instructions before the remote connection like what .gdbinit does?

BTW, I also tried changing various timeouts I found in CLion registry too, but nothing worked. 

CLion Version: 2018.3.3

11 comments
Comment actions Permalink

Hello! Do I understand correctly that you use the "GDB Remote Debug" run/debug configuration? What timeouts have you tried to change in CLion's registry?

0
Comment actions Permalink

Thanks for the response!

I am using a customized version of "OpenOCD + STM32CubeMX" plugin, which lets me create an "Openocd Download & Run" configuration. The plugin in turn starts an Openocd process and a gdb process. Openocd starts a GDB server and the debug process tries to connect to it but times out. Does that answer your question?

Following is the error I am getting from GDB:

Remote replied unexpectedly to 'vMustReplyEmpty': PacketSize=3fff;qXfer:memory-map:read-;qXfer:features:read+;qXfer:threads:read+;QStartNoAckMode+;vContSupported+

 

I tried increasing the following timeouts in CLion registry (some of them were already much higher than the worst case latency of starting gdb server) :

actionSystem.commandProcessingTimeout
cidr.debugger.timeout.load
cidr.debugger.timeout
cidr.debugger.timeout.evaluate
application.deactivation.timeout
appcode.simulator.debugger.launch.timeout
clion.remote.tar.timeout

1
Comment actions Permalink

I've consulted with a developer. Unfortunately, there is no way to tune such parameters currently. Feel free to create a feature request in our tracker: https://youtrack.jetbrains.com/issues/CPP.

Also we have a request about an easier way to create, inspect and modify gdbinit/lldbinit: https://youtrack.jetbrains.com/issue/CPP-14723. Feel free to comment or upvote in order to get updates.

1
Comment actions Permalink

Thanks for looking into this! I will file a feature request to allow the plugins to pass in GDB commandline options on starting GDB. 

1
Comment actions Permalink

Any update on this? It would be a very helpful feature to be able to specify what command line options are sent to gdb/openocd. Other debug configurations support argument passing.

0
Comment actions Permalink

Hi Bthacher!

>Other debug configurations support argument passing.

Do you mean the "Program arguments" field? It's about passing arguments to your application, not to debugger.

>It would be a very helpful feature to be able to specify what command line options are sent to gdb/openocd.

You can create a project-specific .gdbinit file and specify there GDB configuration options - https://www.jetbrains.com/help/clion/configuring-debugger-options.html#gdbinit-lldbinit.

 

0
Comment actions Permalink

Turns out what I needed was custom startup commands for both gdb and openocd, which I was able to do through a custom .gdbinit file and specifying command line arguments in the "Embedded GDB Server" Run configuration.

I'm still having another weird issue though. My final goal is to get an embedded application built in a docker container and debugged on OSX, and so far I've gotten the executable building in docker without a problem, the issue comes when I try to debug it and the file paths in the debug sections of the ELF all exist as absolute paths in the docker container, but don't exist on the host OSX machine. I tried creating a .gdbinit file and running two separate set substitute-path commands (one for toolchain headers and the other for the source code in the project root directory), but whenever the second set substitute-path command runs, the gdb server doesn't start, and I get a 'Command timed out' message from the Debug tab at the bottom of CLion. I've added a print command at the end of the .gdbinit file so I know that both set substitute-path commands are running, but for some reason the gdb server isn't starting. I can confirm that this is happening as if I comment out the second set substitute-path command, almost immediately after the build finishes, a loading dialog comes up that says 'Starting GDB Server', and debugging works (besides incorrect path mapping). If I uncomment the line, the dialog doesn't come up until the 'Command timed out' message pops up, at which point gdb has already given up, so openocd sits open doing nothing. It seems like a super specific issue but any help is greatly appreciated. Thanks!

0
Comment actions Permalink

Update: I tried copying the source directory to a place outside of the CLion project, and path remapping seems to have worked. Of course this is basically useless because I can't put breakpoints in the project files, as they aren't the files that have been remapped. But it does narrow the issue down to path mapping directories in the project root causing issues.

0
Comment actions Permalink

Bthacher where is the executable (which is built in Docker) located on the local machine? This directory on the local machine should have the same relative path to the project root as in the Docker container.

Also I advise you to read https://www.jetbrains.com/help/clion/complicated-remote-scenarios.html, maybe it will help.

1
Comment actions Permalink

Previously, the project directory on the local machine had a different path than in the docker container, causing the path issue. I just changed the deployment path to match the project directory on the local machine, and the issue persists. Using readelf, I can confirm that the .debug_info section has references to files that exist on the local machine, and no references to the paths specific to the docker container. I don't know what would cause openocd not to start. Are there log files associated with CLion build tasks that I could look at? It feels like gdb is trying to load the symbol file (same file as the ELF) and for some reason it's having trouble, maybe because it's being used somewhere else. If the 'Command timed out' was more descriptive, I could see what was going on. If it's having trouble loading the symbol file, that would mean it's stuck with the ELF file open and preventing openocd from starting, because openocd is waiting for the ELF to be available. But debugging works from the command line without any issues like this.

0
Comment actions Permalink

Well this is embarrassing, it seems like there was a breakpoint from earlier in my troubleshooting, and for some reason it took a very long time to add it while openocd started up. When I removed it, CLion didn't tell gdb to insert the breakpoint, so gdb started up much faster than before, allowing openocd to start fully, and for gdb to connect to it. Still, this should not be an issue, as I would like to add breakpoints that are hit when the program begins. Now the issue I'm having is that if I set a breakpoint in main, it can't load the stack frame and times out. I enabled verbose logging for gdb and can't figure out what's causing the timeout.

0

Please sign in to leave a comment.