Daniel J. Bernstein's softwares all in one

Well, if not all, quite a few of them.

What this is

Over the years, Daniel J. Bernstein has released several toolsets, as in the infobox. As of 2016 there are some problems with this.

• They aren't conveniently packaged for some of us. The nosh package replaces most of daemontools, ucspi-tcp, and ptyget. But some things are outwith its remit, and most is not all. One still needs publicfile's httpd and ftpd, as they are not within the remit of the nosh toolset. One might really need multilog, where cyclog simply doesn't satisfy the use case.

  People like Gerrit Pape and Paul Jarc have packaged these toolsets much as they originally came from Bernstein, dividing the binaries up as he divided the sources. With their packages: Pulling in publicfile's httpd and ftpd also pulls in publicfile's configure, that conflicts with other programs named configure (and that isn't necessary given that nosh has service bundles for publicfile's HTTP and FTP servers). One cannot just pull in the Pape daemontools package for multilog, as that pulls in conflicting tools named svstat, svscan, and so forth.

• There's a lot of partial duplication. Many of these toolsets use the same Bernstein libraries, such as his TAI library, internally; but the libraries were evolving as the packages were written. Newer packages have newer and better versions of the libraries than the older packages.

  This is exemplified by publicfile's TODO list, which contains a note to migrate things like timeoutread() to the iopause library. This was actually done for later toolsets; and this consolidation thus directly addresses a publicfile TODO item.

• It's all a bit haphazard. ptyget comes with redo scripts, but the other packages do not. daemontools comes with slashpackage scripts, but the other packages do not. Only ptyget and clockspeed have manual pages in the package. (Gerrit Pape has added manual pages for ucspi-tcp and daemontools; although, bizarrely, they are buried in a separate area as if they were specific to Debian Linux.) djbdns servers aren't really designed to inherit open sockets, ucspi style.

• There are some bugs that never got fixed. The Bernstein-published packages still try to declare errno themselves. Bernstein-published ptyget still relies upon an ioctl, TIOCREMOTE, that has long since been consigned to the dustbin (except by OpenBSD). There are a large number of implicit function declarations, void main()s, clashes in const-ness, and other things that C1999 and C2011 compilers gravely dislike.

This is a consolidation of all of these packages.

• It's not a improvement along the lines of nosh, s6, daemontools-encore, and so forth, and of ucspi-ssl and so forth.

  • The often-overlooked (but documented in the original Bernstein manuals and README) requirement to only use the Olson "right" timezones is still there.

  • tcpserver is still insecure by default, with secure mode (where we do not trust DNS and IDENT data that attackers have control over) being the non-default setting.

  • The problem where dnscache looks up stuff again that it has already just looked up, is still there.

  • ftpd still does not support IPv6. dnscache still does not support IPv6 addresses for content DNS servers.

  • dnscache still has problems with back-end transactions for looking up content DNS servers, and with duplicate back-end transactions.

• The build system is redo; like with ptyget but finished and extended to everything.

• All of the manual pages, Bernstein's originals and Pape's additions, are included. Most of them are now in Docbook XML, from which the build system compiles roff and HTML manual pages.

• The duplicated libraries have been consolidated. In some cases, in particular in some of the very oldest code, this has meant switching from (say) the "substdio" library to the "buffer" library. Some code, such as the old "sig" APIs for catching/ignoring signals, has been retained alongside its newer replacements from later iterations of the library, for the benefit of not updating older code that uses that API.

• There are some patches.

  • My djbdns patches are included, but they mainly add minor functionality (like additional querying tools) or fix outright errors (like just blithely continuing on regardless even when input is malformed).

  • My ucspi-tcp patches are included, but again they just fix outright errors (no error checking in argument parsing).

  • The content DNS and TAICLOCK servers have also been patched to enable them to inherit already-open sockets, using the systemd LISTEN_FDS mechanism. So has dnscache, which requires two already-open sockets, of the right kinds.

  • The easy-syntax SRV resource record support has been added (including for tinydns-get and dnsq).

  • Easy-syntax AAAA resource record support has been added (including for tinydns-get and dnsq).

  • rblsmtpd no longer defaults to a blacklist that has been defunct for over a decade.

  • gopherd has been added.

  • ftpd now copes with various poor and standards-violating practices in modern WWW browsers whose support for FTP is no longer as good as it was when publicfile was originally written.

  • A simple standalone host tool has been added, eliminating the need for the ISC implementation of that tool, and all of the ISC-specific additional packages that it pulls in.

  • All DNS servers now synthesize HINFO responses to ANY queries.

• The basic bugs are fixed. errno is declared by including the right standard C library header. Various missing function declarations that C1999 requires are added. Const correctness bugs are fixed in various places. Incorrect declarations of main() are fixed. Reliance upon implicit function declarations, non-conformant to C1999, is fixed by including quite a lot of missing headers. The obsolete TIOCREMOTE mechanism is disabled outwith OpenBSD.

• It is only the tools and their doco.

  • The various "install", "setup", "check" helpers that create and test directory hierarchies under /usr/local, longhand, in C code, are omitted. The source package is capable of building Debian, FreeBSD/TrueOS, and OpenBSD binary packages. Package management deals with installing files into hierarchies with ownerships and permissions; this is package management's bread and butter.

  • svscanboot is a thing of the past. Plumbing this in as one's service management mechanism, or under one's own system and service management mechanisms, is well beyond the scope here.

  • The helpers for creating djbdns and publicfile service bundles are omitted. The nosh toolset comes with service bundles for these; and the same goes for some other toolsets, all of which have different needs (not satisfied by the Bernstein helper programs) anyway.

No; qmail, fastforward, and ezmlm are not included. They're a project in their own rights. In fact, other people have already taken them on. I suggest looking at netqmail.

Obtaining it

You can obtain it in two ways.

• It is available in source form. This is distributed as a slashpackage-style package. That's a whole subject in itself; and the things that you need to know here are:

  • It is known to build, run, and work on FreeBSD 10, PC-BSD 10, Debian Linux version 9, and OpenBSD version 5.9. (For a short while, until I ported nosh to OpenBSD, my WWW site was the various Bernstein tools from this consolidated package built and running on OpenBSD.) It should similarly build, run, and work on any modern BSD and on any modern Linux flavour.

  • It requires redo to build, and builds with the gcc and clang compilers. The build system attempts to autodetect the available compiler.

  • To just build the package from source:

    package/compile

    This gives you:

    • binaries in command/, which you can just add to your PATH environment variable or symbolically link to;
    • manual pages in manual/;

    The build process updates files in these directories atomically. It doesn't create part-written executables or other files at their final names. So you can run things straight out of these directories whilst rebuilding.

  • To clean the build run

    package/clean

    or just rm -r build/

  • To clean the build, packaging, and built files run

    package/distclean

    or just rm -r build/ command/ manual/

  • To build the binary packages on FreeBSD, TrueOS, or OpenBSD run

    package/bsd/prepare && bsd/rules clean build binary

    On FreeBSD and TrueOS this requires pkg version 1.2 or later in order to avoid segmentation faults and other bugs. So ensure that pkg is up to that version in your ports tree.

  • To build the binary packages on Debian Linux run

    package/debian/prepare && dpkg-buildpackage -b -uc

  • Putting things into places outwith the self-contained directory is "exporting". To export to /usr/local run (as the superuser)

    package/export /usr/local

• Pre-built binary packages are available, that install the toolset under /usr/local.