Make Installing Your Software Sane

I’m not of the Easybuild faith — I want to use a package manager rather than just a build system — but I commend a talk from its primary author from FOSDEM a few years ago, How To Make Package Managers Cry. It’s worth the recording — join the laughter! — even if you prefer not to spend time watching videos, especially as disclaimers on the slides give a misleading impression of pandering to people who don’t do irony.

I have extensive experience building technical software with varying degrees of difficulty, going back at least to achieving a GCC 1.something cross-compiler, and more recently building many operating system packages. From that experience everything in the talk rings so true. I think I’ve hit all the examples, either exactly, or instances of the same thing with different software. I was surprised the horror of Java stuff didn’t appear, which has been a particular source of grief for Fedora maintenance recently.I confess to having abandoned getting TAU into RPM shape.

(Also, damn much proprietary stuff which charges you lots of money for the installation and ongoing support pain it causes, not least with licence managers, but that’s out of scope.)

If you work on some free software system, please consider people building your software generally and specifically packaging it for GNU/Linux distributions which allow users just to use ‘package‑manager install’. Ideally do both RPM and dpkg packaging for it yourself, roughly consistent with the well-chosen distribution requirements,Like not bundling dependencies or expecting them to be downloaded during the build; they can be difficult to keep track of in detail, but you can leave that to distribution maintainers.

even if you don’t distribute built packages, but you might want to consider maintaining packages in one or more distribution. Packaging indicates how sane building it is and what to consider changing.

Ease of installation should surely be part of software engineering, but you wouldn’t guess it generally. Where it is, increased respect if you can make builds reproducible for anything that needs to be trustworthy — not something most preachers of Reproducibility seem to consider.

As Hoste suggested, putting some effort into that is probably good if you want your stuff to be used. When I became overall maintainer of the CCP4 crystallography collaborative computational project, as a professional duty I did myself out of the previous lucrative personal consultancy for pharmaceutical companies who needed the existing mess installed. I wrote a configureModelled after GCC’s, as Autoconf was quite new and not really suitable then.

script after I’d cleaned things up and started version control and proper versioned releases. I was the poorer, but the income for the project from companies paying fees to use it increased markedly.The licensing position was highly dubious, and I didn’t like it mostly being at best semi-free software, but former academics in particular were pleased to get their companies to support development on that basis, and there was general happiness.

By the way, a container image isn’t a good answer, for a bunch of reasons.

Ignoring the other sources of lachrymosity and sticking to build systems, my memory was that Hoste said this, but I’ve obviously imagined that, or was thinking of something else: Anyway, I’ll say life is best if you use GNU Autotools (autoconf, automake, possibly gnulib, and libtool). Yes, it’s old stuff, and a lot of what it was intended for isn’t relevant on current platforms, but it does the job, and people like me can understand it to write support for specific features and fix build problems, unlike with CMake most of the time.I’m not the only non-beginner who groans at the prospect of C-unmade software after seeing too much of it and inscrutable failures.

I don’t think it deserves the opprobrium it gets. Yes, it’s rather messy, but build configuration is a dirty business in general.

If you worry about portability to non-Unix systems, and think that means CMake, autotools worked for me on MS-DOS and MS Windows as well as multiple Unixes even over 20 years ago, and there’s development since. Yes, libtool is a wart these days, which should doubtless be replaced by something simple with automake. However, it usually gets the library part done, if with occasional trouble with rearranging link options, for instance. An impossible — according to Cygnus experts at the time — thing I did was to introduce autoconf to the GCC build system to support the G77 library and still cope with ‘Canadian cross’ (cross-building a cross-compiler for a third system).

If you use Autotools, once the dependencies are available, producing packages in various frameworks (RPM, dpkg, Guix, Spack…) is usually pretty trivial, perhaps basically abstracted away, as with Debian’s debhelper(7), and Guix’ and Spack’s frameworks. (Similar abstraction may make CMake-based stuff, for instance, just work too, but more often than with autotools I find it doesn’t and I struggle.)