Written by dap
on September 18, 2012

Debugging dynamic library dependencies on illumos

In this short follow-up to my post on illumos process tools, I’ll expand a bit on ldd and pldd, which print the dynamic linking dependencies of binaries and processes, respectively, and crle, which prints out the runtime linker configuration. These tools are available in most illumos distributions including SmartOS.

Understanding builds (and broken builds in particular) can be especially difficult. I hate running into issues like this one:

$ ffmpeg
ld.so.1: ffmpeg: fatal: libavdevice.so.53: open failed: No such file or directory
Killed

You can use ldd to see the dynamic library dependencies of a binary:

$ ldd $(which ffmpeg)
        libavdevice.so.53 =>     (file not found)
        libavfilter.so.2 =>      (file not found)
        libavformat.so.53 =>     (file not found)
        libavcodec.so.53 =>      (file not found)
        libswresample.so.0 =>    (file not found)
        libswscale.so.2 =>       (file not found)
        libavutil.so.51 =>       (file not found)
        libsocket.so.1 =>        /lib/libsocket.so.1
        libnsl.so.1 =>   /lib/libnsl.so.1
        libvpx.so.0 =>   /opt/local/lib/libvpx.so.0
        libm.so.2 =>     /lib/libm.so.2
        libbz2.so.0 =>   /opt/local/lib/libbz2.so.0
        libz.so.1 =>     /lib/libz.so.1
        libc.so.1 =>     /lib/libc.so.1
        libmp.so.2 =>    /lib/libmp.so.2
        libmd.so.1 =>    /lib/libmd.so.1
        libpthread.so.1 =>       /lib/libpthread.so.1
        librt.so.1 =>    /lib/librt.so.1
        libgcc_s.so.1 =>         /opt/local/lib/libgcc_s.so.1

In this case, the problem is that I installed ffmpeg into /usr/local, but the ffmpeg build appears not to have used the -R linker flag, which tells the runtime linker where to look dynamic libraries when the program is loaded. As a result, ffmpeg doesn’t know where to find its own libraries. If I set LD_LIBRARY_PATH, I can see that it will work:

$ LD_LIBRARY_PATH=/usr/local/lib ldd $(which ffmpeg)
        libavdevice.so.53 =>     /usr/local/lib/libavdevice.so.53
        libavfilter.so.2 =>      /usr/local/lib/libavfilter.so.2
        libavformat.so.53 =>     /usr/local/lib/libavformat.so.53
        libavcodec.so.53 =>      /usr/local/lib/libavcodec.so.53
        libswresample.so.0 =>    /usr/local/lib/libswresample.so.0
        libswscale.so.2 =>       /usr/local/lib/libswscale.so.2
        libavutil.so.51 =>       /usr/local/lib/libavutil.so.51
        ...

I resolved this by rebuilding ffmpeg explicitly with LDFLAGS += -R/usr/local.

ldd only examines binaries, and so can only print out dependencies built into the binary. Some programs use dlopen to open libraries whose name isn’t known until runtime. Node.js add-ons and Apache modules are two common examples. You can view these with pldd, which prints the dynamic libraries loaded in a running process. Here’s the output on a Node program with the node-dtrace-provider add-on:

$ pfexec pldd $(pgrep -x node)
32113:  /usr/local/bin/node /home/snpp/current/js/snpp.js -l 80 -d
/usr/local/bin/node
/lib/libz.so.1
/lib/librt.so.1
/lib/libssl.so.0.9.8
/lib/libcrypto.so.0.9.8
/lib/libdl.so.1
/lib/libsocket.so.1
/lib/libnsl.so.1
/lib/libkstat.so.1
/opt/local/lib/libstdc++.so.6.0.16
/lib/libm.so.2
/opt/local/lib/libgcc_s.so.1
/lib/libc.so.1
/usr/sfw/lib/libgcc_s.so.1
/home/dap/node_modules/dtrace-provider/build/Release/DTraceProviderBindings.node

If you want to see where the system looks for dynamic libraries, use crle, which prints or edits the runtime linker configuration:

$ crle
Configuration file [version 4]: /var/ld/ld.config
  Platform:     32-bit LSB 80386
  Default Library Path (ELF):   /lib:/usr/lib:/opt/local/lib
  Trusted Directories (ELF):    /lib/secure:/usr/lib/secure  (system default)
Command line:
  crle -c /var/ld/ld.config -l /lib:/usr/lib:/opt/local/lib

Of course, for more information on any of these tools, check out their man pages. They’re well-documented. If you find yourself debugging build problems, you’ll probably also want to know about nm, objdump, and elfdump, which are available on many systems and well documented elsewhere.

← → Top