missing sofiles / linker / asterisk / pjsip

Written by
Walter Doekes
Published on 2016-02-21

When compiling Asterisk with a PJProject debianized using the debian/ directory to Ubuntu/Trusty, I got the following compile error:

$ gcc -o chan_pjsip.so -pthread -shared
    -Wl,--version-script,chan_pjsip.exports,--warn-common \
    chan_pjsip.o pjsip/dialplan_functions.o -lpjsua2 -lstdc++ -lpjsua -lpjsip-ua \
    -lpjsip-simple -lpjsip -lpjmedia-codec -lpjmedia-videodev -lpjmedia-audiodev \
    -lpjmedia -lpjnath -lpjlib-util -lsrtp -lpj -lm -lrt -lpthread \
    -lSDL2 -lavformat -lavcodec -lswscale -lavutil -lv4l2 -lopencore-amrnb \
    -lopencore-amrwb
/usr/bin/ld: cannot find -lSDL2
/usr/bin/ld: cannot find -lavformat
/usr/bin/ld: cannot find -lavcodec
/usr/bin/ld: cannot find -lswscale
/usr/bin/ld: cannot find -lavutil
/usr/bin/ld: cannot find -lv4l2
/usr/bin/ld: cannot find -lopencore-amrnb
/usr/bin/ld: cannot find -lopencore-amrwb
collect2: error: ld returned 1 exit status

That’s odd. I have those libs installed. Why is there no versionless .so variant?

$ dpkg -L libsdl2-2.0-0  | grep /lib/
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libSDL2-2.0.so.0.2.0
/usr/lib/x86_64-linux-gnu/libSDL2-2.0.so.0

No .so symlink in that package?

We can make the build succeed by doing the following:

$ cd /usr/lib/x86_64-linux-gnu
$ sudo ln -s libSDL2-2.0.so.0.2.0 libSDL2.so

That fixed the first error, and now the other ones:

$ for x in avformat avcodec swscale v4l2 avutil opencore-amrnb opencore-amrwb
    do test -f lib$x.so ||
      sudo ln -s `readlink lib$x.so.* | sort | tail -n1` lib$x.so
done

At this point, you must re-run configure on Asterisk, because some configure autodetection tests may have failed on the same issue.

$ ./configure --enable-dev-mode
$ make
... finished successfully ...

A bug for this issue was filed here: debian bug #804460

But… that is only half of the story.

There shouldn’t be any .so files in the the regular library package. They are in the -dev package:

$ dpkg -L libavutil52 | grep /lib/.*so
/usr/lib/x86_64-linux-gnu/libavutil.so.52.3.0
/usr/lib/x86_64-linux-gnu/libavutil.so.52
$ dpkg -L libavutil-dev | grep /lib/.*so
/usr/lib/x86_64-linux-gnu/libavutil.so

Why is that?

The Debian packaging manual: 8.4 Development files has this to say:

If there are development files associated with a shared library, the source package needs to generate a binary development package named […] libraryname-dev. […]

The development package should contain a symlink for the associated shared library without a version number. For example, the libgdbm-dev package should include a symlink from /usr/lib/libgdbm.so to libgdbm.so.3.0.0. This symlink is needed by the linker (ld) when compiling packages, as it will only look for libgdbm.so when compiling dynamically.

As you know, a binary X is generally linked against a shared library libY.so.MAJOR. Like this:

$ ldd /bin/ls | grep libacl
  libacl.so.1 => /lib/x86_64-linux-gnu/libacl.so.1 (0x00007f23466cd000)

But, I don’t have version 1, I have version 1.1.0.

$ readlink /lib/x86_64-linux-gnu/libacl.so.1
libacl.so.1.1.0

And when ls was linked, it wasn’t against libacl.so.1, but against libacl.so — without version:

$ gcc ... -lacl ...

This works, not because ld knows anything about taking only the MAJOR version, but because that version was built into the so file:

$ objdump -p /lib/x86_64-linux-gnu/libacl.so.1.1.0 | grep SONAME
  SONAME               libacl.so.1

That explains the versions and why we do need the libacl.so.1 symlink in the non-dev package, but we don’t need libacl.so there. We only need that when developing, and then we get it from the the -dev package.

As for the bug at hand: I believe this is a misconfiguration in Asterisk or PJProject and should be fixed there. The pjproject libs may be linked to other libraries, but that doesn’t mean we (Asterisk) needs to link to those too.

In fact, the problems lies with Asterisk calling pkg-config --libs pjproject and pjproject listing its own used libraries in the public Libs section (instead of the private one).

$ grep ^Libs /usr/lib/x86_64-linux-gnu/pkgconfig/libpjproject.pc
Libs: -L${libdir} -lpjsua2 -lstdc++ -lpjsua -lpjsip-ua -lpjsip-simple -lpjsip
  -lpjmedia-codec -lpjmedia -lpjmedia-videodev -lpjmedia-audiodev -lpjmedia
  -lpjnath -lpjlib-util -lsrtp -lpj -lm -lrt -lpthread
  -L/usr/lib/x86_64-linux-gnu -lSDL2 -lavformat -lavcodec -lswscale -lavutil
  -lv4l2 -lopencore-amrnb -lopencore-amrwb

Unless I’m mistaken, only the -lpj* should be in there. The others shouldn’t, or be in the Libs.private section instead, as this libtiff example shows:

$ grep ^Libs /usr/lib/x86_64-linux-gnu/pkgconfig/libtiff-4.pc
Libs: -L${libdir} -ltiff
Libs.private: -llzma -ljbig -ljpeg -lz

Or, Asterisk shouldn’t use the pkg-config values, hauling in lots of unnecessary dependencies. Or both ;-)

Update 2016-08-16

Harm Geerts pointed me to this clear explanation about “Overlinking” when I ran into this issue again.

This time the issue reared its head during Asterisk configure time when it decided that the compilation of the HAVE_PJ_TRANSACTION_GRP_LOCK configure test failed, not because pjsip_tsx_create_uac2 did not exist, but because the “overlinked” dependencies — like -lSDL2 — did not exist on my build system.

When the test incorrectly assumed that HAVE_PJ_TRANSACTION_GRP_LOCK was false, the compilation failed in res/res_pjsip/pjsip_distributor.c where old-style pj_mutex_unlock(tsx->mutex) was called, instead of newer pj_grp_lock_release(tsx->grp_lock).

Update 2016-08-31

An updated pjproject.pc could look like this:

# Package Information for pkg-config

prefix=/usr
exec_prefix=${prefix}
libdir=/usr/lib/x86_64-linux-gnu
includedir=/usr/include

Name: libpjproject
Description: Multimedia communication library
URL: http://www.pjsip.org
Version: 2.4.5
Libs: -L${libdir} -L/usr/lib/x86_64-linux-gnu -lpjsua2 -lpjsua -lpjsip-ua -lpjsip-simple -lpjsip -lpjmedia-codec -lpjmedia -lpjmedia-videodev -lpjmedia-audiodev -lpjmedia -lpjnath -lpjlib-util -lpj
Libs.private: -lstdc++ -lsrtp -lm -lrt -lpthread -lSDL2 -lavformat -lavcodec -lswscale -lavutil -lv4l2 -lopencore-amrnb -lopencore-amrwb
Cflags: -I${includedir} -I/usr/include -DPJ_AUTOCONF=1   -DPJ_IS_BIG_ENDIAN=0 -DPJ_IS_LITTLE_ENDIAN=1

missing sofiles / linker / asterisk / pjsip