[PATCH] Fix search of debug files for remote debuggee

[Andrey Utkin]

When using remote debugging server, and when debuggee filename is
inferred via qXfer:exec-file:read request, or sysroot starts with
"target:", this "target:" prefix of filepaths is not treated correctly
during debug file search - it appears in the middle of the looked up
paths.

In the following example, unpatched GDB can't find separate debug files for
neither the executable, nor standard libraries:

$ gdb -ex 'set debug separate-debug-file 1' -ex 'set sysroot target:/' -ex 'set debug-file-directory /usr/lib/debug:/home/j/hide' -ex 'target remote :3333' -ex 'break main' -ex 'continue' -ex 'bt' -ex 'info sharedlibrary' -ex 'set confirm off' -ex 'quit'
GNU gdb (Gentoo 9999 vanilla) 8.2.50.20181109-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Remote debugging using :3333
Reading /home/j/test from remote target...
warning: File transfers from remote targets can be slow. Use "set sysroot" to access files locally instead.
Reading /home/j/test from remote target...
Reading symbols from target:/home/j/test...

Looking for separate debug info (debug link) for target:/home/j/test
Trying target:/home/j/test.debug
Reading /home/j/test.debug from remote target...
Trying target:/home/j/.debug/test.debug
Reading /home/j/.debug/test.debug from remote target...
Trying /usr/lib/debug/target:/home/j/test.debug
Trying /home/j/hide/target:/home/j/test.debug
(No debugging symbols found in target:/home/j/test)
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
Reading symbols from target:/lib64/ld-linux-x86-64.so.2...

Looking for separate debug info (debug link) for target:/lib64/ld-linux-x86-64.so.2
Trying target:/lib64/ld-2.27.so.debug
Reading /lib64/ld-2.27.so.debug from remote target...
Trying target:/lib64/.debug/ld-2.27.so.debug
Reading /lib64/.debug/ld-2.27.so.debug from remote target...
Trying /usr/lib/debug/target:/lib64/ld-2.27.so.debug
Trying /home/j/hide/target:/lib64/ld-2.27.so.debug
(No debugging symbols found in target:/lib64/ld-linux-x86-64.so.2)

Looking for separate debug info (build-id) for system-supplied DSO at 0x7ffff7ffa000
Trying /usr/lib/debug/.build-id/fd/03d584bc1a90ba28be457635a02662c9f9c1f2.debug
Trying /home/j/hide/.build-id/fd/03d584bc1a90ba28be457635a02662c9f9c1f2.debug
0x00007ffff7dd7000 in ?? () from target:/lib64/ld-linux-x86-64.so.2
Breakpoint 1 at 0x4005eb
Continuing.
Reading /lib64/libpthread.so.0 from remote target...
Reading /lib64/libc.so.6 from remote target...

Looking for separate debug info (debug link) for target:/lib64/libpthread.so.0
Trying target:/lib64/libpthread-2.27.so.debug
Reading /lib64/libpthread-2.27.so.debug from remote target...
Trying target:/lib64/.debug/libpthread-2.27.so.debug
Reading /lib64/.debug/libpthread-2.27.so.debug from remote target...
Trying /usr/lib/debug/target:/lib64/libpthread-2.27.so.debug
Trying /home/j/hide/target:/lib64/libpthread-2.27.so.debug

Looking for separate debug info (debug link) for target:/lib64/libc.so.6
Trying target:/lib64/libc-2.27.so.debug
Reading /lib64/libc-2.27.so.debug from remote target...
Trying target:/lib64/.debug/libc-2.27.so.debug
Reading /lib64/.debug/libc-2.27.so.debug from remote target...
Trying /usr/lib/debug/target:/lib64/libc-2.27.so.debug
Trying /home/j/hide/target:/lib64/libc-2.27.so.debug

Breakpoint 1, 0x00000000004005eb in main ()
#0 0x00000000004005eb in main ()
From To Syms Read Shared Object Library
0x00007ffff7dd6e80 0x00007ffff7df4650 Yes (*) target:/lib64/ld-linux-x86-64.so.2
0x00007ffff7bbbb70 0x00007ffff7bcbfee Yes (*) target:/lib64/libpthread.so.0
0x00007ffff780f200 0x00007ffff7962d7c Yes (*) target:/lib64/libc.so.6
(*): Shared library is missing debugging information.

With current fix, the paths used always have target: in the beginning
and this helps to find all debug files:

$ gdb -ex 'set debug separate-debug-file 1' -ex 'set sysroot target:/' -ex 'set debug-file-directory /usr/lib/debug:/home/j/hide' -ex 'target remote :3333' -ex 'break main' -ex 'continue' -ex 'bt' -ex 'info sharedlibrary' -ex 'set confirm off' -ex 'quit'
GNU gdb (Gentoo 9999 vanilla) 8.2.50.20181109-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Remote debugging using :3333
Reading /home/j/test from remote target...
warning: File transfers from remote targets can be slow. Use "set sysroot" to access files locally instead.
Reading /home/j/test from remote target...
Reading symbols from target:/home/j/test...

Looking for separate debug info (debug link) for target:/home/j/test
Trying target:/home/j/test.debug
Reading /home/j/test.debug from remote target...
Trying target:/home/j/.debug/test.debug
Reading /home/j/.debug/test.debug from remote target...
Trying target:/usr/lib/debug//home/j/test.debug
Reading /usr/lib/debug//home/j/test.debug from remote target...
Trying target:/home/j/hide//home/j/test.debug
Reading /home/j/hide//home/j/test.debug from remote target...
Reading /home/j/hide//home/j/test.debug from remote target...
Reading symbols from target:/home/j/hide//home/j/test.debug...
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
Reading symbols from target:/lib64/ld-linux-x86-64.so.2...

Looking for separate debug info (debug link) for target:/lib64/ld-linux-x86-64.so.2
Trying target:/lib64/ld-2.27.so.debug
Reading /lib64/ld-2.27.so.debug from remote target...
Trying target:/lib64/.debug/ld-2.27.so.debug
Reading /lib64/.debug/ld-2.27.so.debug from remote target...
Trying target:/usr/lib/debug//lib64/ld-2.27.so.debug
Reading /usr/lib/debug//lib64/ld-2.27.so.debug from remote target...
Reading /usr/lib/debug//lib64/ld-2.27.so.debug from remote target...
Reading symbols from target:/usr/lib/debug//lib64/ld-2.27.so.debug...

Looking for separate debug info (build-id) for system-supplied DSO at 0x7ffff7ffa000
Trying /usr/lib/debug/.build-id/fd/03d584bc1a90ba28be457635a02662c9f9c1f2.debug
Trying /home/j/hide/.build-id/fd/03d584bc1a90ba28be457635a02662c9f9c1f2.debug
0x00007ffff7dd7000 in _start () from target:/lib64/ld-linux-x86-64.so.2
Breakpoint 1 at 0x4005ef: file test/test16.c, line 13.
Continuing.
Reading /lib64/libpthread.so.0 from remote target...
Reading /lib64/libc.so.6 from remote target...

Looking for separate debug info (debug link) for target:/lib64/libpthread.so.0
Trying target:/lib64/libpthread-2.27.so.debug
Reading /lib64/libpthread-2.27.so.debug from remote target...
Trying target:/lib64/.debug/libpthread-2.27.so.debug
Reading /lib64/.debug/libpthread-2.27.so.debug from remote target...
Trying target:/usr/lib/debug//lib64/libpthread-2.27.so.debug
Reading /usr/lib/debug//lib64/libpthread-2.27.so.debug from remote target...
Reading /usr/lib/debug//lib64/libpthread-2.27.so.debug from remote target...

Looking for separate debug info (debug link) for target:/lib64/libc.so.6
Trying target:/lib64/libc-2.27.so.debug
Reading /lib64/libc-2.27.so.debug from remote target...
Trying target:/lib64/.debug/libc-2.27.so.debug
Reading /lib64/.debug/libc-2.27.so.debug from remote target...
Trying target:/usr/lib/debug//lib64/libc-2.27.so.debug
Reading /usr/lib/debug//lib64/libc-2.27.so.debug from remote target...
Reading /usr/lib/debug//lib64/libc-2.27.so.debug from remote target...

Breakpoint 1, main () at test/test16.c:13
13 for ( i=0; i<10; ++i)
#0 main () at test/test16.c:13
From To Syms Read Shared Object Library
0x00007ffff7dd6e80 0x00007ffff7df4650 Yes target:/lib64/ld-linux-x86-64.so.2
0x00007ffff7bbbb70 0x00007ffff7bcbfee Yes target:/lib64/libpthread.so.0
0x00007ffff780f200 0x00007ffff7962d7c Yes target:/lib64/libc.so.6

gdb/ChangeLog:

2018-11-09 Andrey Utkin <***@undo.io>

* symfile.c (find_separate_debug_file): Fix search of debug files for
remote debuggee.

Signed-off-by: Andrey Utkin <***@undo.io>
---
gdb/ChangeLog | 5 +++++
gdb/symfile.c | 14 +++++++++++---
2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/gdb/ChangeLog b/gdb/ChangeLog
index b6c612bd0b..a3f308e31b 100644
--- a/gdb/ChangeLog
+++ b/gdb/ChangeLog
@@ -1,3 +1,8 @@
[phone] Andrey Utkin <***@undo.io>
+
+ * symfile.c (find_separate_debug_file): Fix search of debug files for
+ remote debuggee.
+
2018-11-09 Hafiz Abid Qadeer <***@codesourcery.com>

* configure: Regenerate.
diff --git a/gdb/symfile.c b/gdb/symfile.c
index 8ab6a25de7..5eb9048d4d 100644
--- a/gdb/symfile.c
+++ b/gdb/symfile.c
@@ -1399,14 +1399,19 @@ find_separate_debug_file (const char *dir,
Keep backward compatibility so that DEBUG_FILE_DIRECTORY being "" will
cause "/..." lookups. */

+ bool remote = !strncmp(dir, "target:", 7);
+ const char *dir_notarget = remote ? dir + 7 : dir;
std::vector<gdb::unique_xmalloc_ptr<char>> debugdir_vec
= dirnames_to_char_ptr_vec (debug_file_directory);

for (const gdb::unique_xmalloc_ptr<char> &debugdir : debugdir_vec)
{
- debugfile = debugdir.get ();
+ debugfile = "";
+ if (remote)
+ debugfile += "target:";
+ debugfile += debugdir.get ();
debugfile += "/";
- debugfile += dir;
+ debugfile += dir_notarget;
debugfile += debuglink;

if (separate_debug_file_exists (debugfile, crc32, objfile))
@@ -1419,7 +1424,10 @@ find_separate_debug_file (const char *dir,
strlen (gdb_sysroot)) == 0
&& IS_DIR_SEPARATOR (canon_dir[strlen (gdb_sysroot)]))
{
- debugfile = debugdir.get ();
+ debugfile = "";
+ if (remote)
+ debugfile += "target:";
+ debugfile += debugdir.get ();
debugfile += (canon_dir + strlen (gdb_sysroot));
debugfile += "/";
debugfile += debuglink;

[John Baldwin]

Post by Andrey Utkin
When using remote debugging server, and when debuggee filename is
inferred via qXfer:exec-file:read request, or sysroot starts with
"target:", this "target:" prefix of filepaths is not treated correctly
during debug file search - it appears in the middle of the looked up
paths.

I only have a somewhat related comment. I have a similar-ish issue when
using sysroot and cross-debugging without a remote target. My use case is
that I'm on an x86 host but that I run QEMU instances of other architectures
(mips, arm, riscv, etc.) on my host. I tend to build base OS builds and
install them to some path (e.g. /ufs/riscv64/rootfs) and then use a tool
like makefs to generate a filesystem image to use with QEMU. I will then
scp a core dump out and debug it like so:

gdb /path/to/binary
...
(gdb) set sysroot /ufs/riscv64/rootfs
(gdb) core /path/to/core

Now there are debug symbols for the various base OS libraries and binaries
installed in the sysroot at /ufs/riscv64/rootfs/usr/lib/debug. For example,
FreeBSD's libc is installed as /lib/libc.so.7 under the sysroot
(/ufs/riscv64/lib/libc.so.7) and the debug info is stored at /usr/lib/debug
under the sysroot (/ufs/riscv64/usr/lib/debug/lib/libc.so.7.debug). However,
GDB places the sysroot after the debug prefix and ends up looking for
/usr/lib/debug/ufs/riscv64/lib/libc.so.7 on the host instead. I usually
work around this by creating a symlink from /usr/lib/debug/ufs/riscv64 to
/ufs/riscv64/usr/lib/debug, but it seems like a bug to me that the we don't
look for debug files "under" the sysroot.

I bring this up because I wonder if a more general solution of always putting
the sysroot before the debug prefix (e.g. "/usr/lib/debug") wouldn't fix
both your case and mine. It's not clear to me what the "correct" behavior
is for mixing sysroot with separate debug files. It may be that the current
behavior of <debug dir>/<sysroot>/<object file> is intentional rather than
the <sysroot>/<debug dir>/<object file> behavior that I would like (and that
I think your use case also wants).

[Andrey Utkin]

Post by John Baldwin
Now there are debug symbols for the various base OS libraries and binaries
installed in the sysroot at /ufs/riscv64/rootfs/usr/lib/debug. For example,
FreeBSD's libc is installed as /lib/libc.so.7 under the sysroot
(/ufs/riscv64/lib/libc.so.7) and the debug info is stored at /usr/lib/debug
under the sysroot (/ufs/riscv64/usr/lib/debug/lib/libc.so.7.debug). However,
GDB places the sysroot after the debug prefix and ends up looking for
/usr/lib/debug/ufs/riscv64/lib/libc.so.7 on the host instead. I usually
work around this by creating a symlink from /usr/lib/debug/ufs/riscv64 to
/ufs/riscv64/usr/lib/debug, but it seems like a bug to me that the we don't
look for debug files "under" the sysroot.
I bring this up because I wonder if a more general solution of always putting
the sysroot before the debug prefix (e.g. "/usr/lib/debug") wouldn't fix
both your case and mine. It's not clear to me what the "correct" behavior
is for mixing sysroot with separate debug files. It may be that the current
behavior of <debug dir>/<sysroot>/<object file> is intentional rather than
the <sysroot>/<debug dir>/<object file> behavior that I would like (and that
I think your use case also wants).

I see what you mean.

Your intention makes sense, but since GDB does not assume your case out
of the box, I guess

set debug-file-directory /ufs/riscv64/usr/lib/debug

should just work for you?

From code I see that GDB checks quite a few places for debug files,
maybe some day it would check also where you want.

[Andrey Utkin]

Reported as https://sourceware.org/bugzilla/show_bug.cgi?id=23893