Generating Makefiles
Creating makefiles for your project can be a labor of love or an exercise in futility, and sometimes it is both simultaneously. Much of it is consistent and repetitive between projects. In this section we'll find out what GNU makefiles should look like, and how to greatly simplify their creation. Many tools exist to ease the blow and allow you to get on with writing your code without all the hassle of makefile maintenance.
GNU Makefile Standards
The GNU project has an official set of guidelines for creating GNU software. These standards help lay out a single established way of doing things, which will help make your software easier to deal with and less susceptible to common coding and portability errors. In this section we will concentrate on the GNU guidelines for creating makefiles, in particular the standard directory locations and makefile targets. If you want to know more about the GNU standards, you can type info standards at the command line of most GNU systems, or visit the standards section of the GNU Web page, at http://www.gnu.org/prep/standards_toc.html. In addition to makefiles, the GNU standards cover program design, code formatting styles, licensing, documentation, and more.
The GNU standards document defines a long list of makefile variables that point to important directory locations. Most of them refer to install locations. For example, the $(bindir) variable typically points to /usr/local/bin, the directory in which most administrators will want to install program executables for their users. Other types of files end up in different directories- libraries, for example, in $(libdir), which defaults to /usr/local/lib. Installing them into a common directory makes it easier for the system to find them. Rather than having to add a new entry to the search path for every application you install, you have to add only one: /usr/local/bin, for example.
As we learned earlier, you can change the values of these prefix variables at configure time, by passing different parameters to the configure script. Often, if a software package is less than stable-or perhaps you haven't decided yet if you want to install it permanently on your system-you'll want to install it into a directory path other than /usr/local. Linux distributions typically install their binary packages into the /usr path. Some people like to install GNOME into the /opt/gnome prefix, to keep it separate from other things on their system. This is particularly useful if you want to set up two parallel de- velopment environments, for two versions of GNOME. You might have a stable version in /opt/gnome and a cutting-edge experimental version in /opt/gnome-unstable. If you use a configure script generated by autoconf, all you have to do is specify a new base prefix:
./configure --prefix=/opt/gnome
|
For a full list of the directories that configure supports, you can run configure --help. The names of these configure options correspond directly to the names of the GNU makefile variables. You can refer to them in your Makefile.in files by using the autoconf substitution syntax because autoconf implicitly calls AC_SUBST on all of them, as part of the AC_INIT macro. For example, you can use configure's value of $(includedir) in your own Makefile.in by referencing @includedir@:
includedir = @includedir@
|
As we'll see in the following sections, automake does a lot of this for you automatically, so for the most part you can just start using these variables, trusting autoconf and automake to populate them with the proper values. Table 3.1 lists the most common directory paths, including their default values, how they are constructed, their autoconf substitutions, and a brief description of each. This table uses the ${} notation to highlight the fact that these vari- ables can be used in both makefiles and shell scripts; the $( ) notation is appropriate only in makefiles.
Table 3.1 Standard GNU Makefile Path Variables
|
A few items in Table 3.1 bear some additional explanation. First let's look at the distinction between ${prefix} and ${exec_prefix}. Since all other paths are derived from ${prefix}, they will automatically use the same base path, even if it changes. ${prefix} and ${exec_prefix} are usually identical, except that the former points to architecture-independent (i.e., universal) components and the latter points to architecture-dependent (i.e., system-specific) components. You should change them only if you really know what you're doing. If an administrator wanted to export and share the same /usr/local directory across multiple machines with different architectures, she might set ${exec_prefix} to /usr/local/i386 or /usr/local/alpha, for different compilations of the same code base. This would leave the architecture-independent paths untouched while retargeting the architecture-specific files into subdirectories. Thus ${datadir} would point to /usr/local/share in both cases, while ${bindir} would point to /usr/local/i386/bin in the first case and to /usr/local/alpha/bin in the second case.
Normally you will always use ${includedir}-or an appropriate subdirectory-as an install destination for your C header files. As long as you stick to the defaults, the compiler should implicitly search this path to satisfy #include declarations; you should be able to reference files from this directory in your programs without explicitly listing the path on the compile line, with the -I option. GNU's gcc compiler will implicitly look in both /usr/include and /usr/local/include, so ${includedir}'s default value of /usr/local/include works fine.
The ${srcdir} variable is the odd one of the bunch. Unlike most of the other install-related variables, ${srcdir} points to someplace inside the currently building source tree. The value of this variable can theoretically differ, depending on which subdirectory of the source tree you're in, but usually it just points to the current directory. If you're using GNU make, you can run builds from outside the source tree; in this case ${srcdir} will be a relative path from the build directory to the source directory.
The GNU standards document also describes a comprehensive set of makefile targets. By complying with these standard targets, you ensure that people will know how to compile and install your application. Each target has a clear, simple name and well-defined behavior. For example, the standard defines four different types of "clean" targets, each of which deletes a certain category of files within the project. automake supports all of them, plus a few of its own.
Table 3.2 lists most of the common GNU targets, divided into four general categories according to their general purpose. Most of the targets should be self-explanatory.
Table 3.2 Standard GNU Makefile Targets
|
During normal development you will probably use only all, clean, and install, and perhaps uninstall. When you're ready to make a release of your software, you will probably run the distcheck target, which creates a compressed archive (.tar.gz) file of your software package and then tests it to make sure it builds properly. Technically the distcheck target isn't an official part of the GNU standard, but automake always generates it for you, and it is important enough to mention alongside the other targets. It will help protect you and your users from the nightmare of an incomplete release.
If you want to supply any optional test programs that administrators can run to ensure that the software works before installing it, you should put these under the check target, not under the all target. They should not build by default; administrators should have to explicitly run make check to build the test programs. automake provides some basic support for the check target.
Normally you'll want to install software with debug information compiled in, unless you are absolutely sure it contains no bugs. If you really don't want debug information, you can install with the install-strip target instead of the normal install target. If the software ends up crashing, however, it will be much harder to diagnose the problem. On most modern operating systems, the extra debug information barely affects performance, if at all, so the main drawback to installing binaries with debug information is the extra hard disk space it uses.
Using automake
The autoconf tool does a lot of work to create a configure script for you, but it doesn't do much with your makefiles, other than some simple variable substitution. It provides no common makefile structure, no conventions, nothing to help manage those beasts that can rapidly grow out of control in a complex project. You're still stuck with hand-writing your compile rules. If you want to support the full standard GNU makefile semantics, you will end up repeating a lot of it for each new project you start.
The automake tool addresses this problem by establishing a makefile template convention that allows you to express your project's build structure in a clear, simple manner. You save this structure in a Makefile.am file, which automake then expands into a Makefile.in file. The configure script can then convert the generated Makefile.in file into a Makefile file. In essence, automake puts an abstraction layer around the build system, freeing you from the drudgery of hand-writing your makefiles. automake also tracks the GNU standard very closely, so you no longer have to update your makefiles when that standard changes; automake will do that for you. You can also change the strictness with which automake adheres to the standards with command line parameters to automake.
Part of the strength of automake is the closeness with which it integrates itself with its sister tools. The automake distribution includes many extra m4 macros for use with the autoconf system. In addition to supplying numerous new tests to make sure the target system can handle the chores automake needs to do, these macros help cement the integration between automake and autoconf, and between each of these and libtool as well. You'll add these macros to your configure.in file alongside the autoconf macros, sometimes replacing the latter. For example, if you are using autoconf without automake, you should call the AC_CONFIG_HEADER macro; however, if you want to use automake, you should call AM_CONFIG_HEADER instead. Automake's macros begin with the "AM_" prefix to distinguish them from autoconf's macros.
The only mandatory automake macro is AM_INIT_AUTOMAKE(PACKAGE, VERSION). You use it like this in your configure.in file:
AM_INIT_AUTOMAKE(myapp, 0.0.1)
|
The first parameter is the PACKAGE name, which is often the name of the main executable. This name will also appear in the distribution file name-for example, myapp-0.0.1.tar.gz. The macro calls AC_DEFINE for both of these parameters, so you can use PACKAGE and VERSION directly in your code. We will make use of this feature later for initializing the GNOME libraries (Section 5.3) and setting up the GNOME documentation system (Chapter 12).
To make all this macro juggling a little easier on you, automake ships with aclocal, a utility to pull all the external nonautoconf macros into a single file, aclocal.m4 (see Figure 3.2). aclocal grabs whatever automake-specific macros you used in configure.in and adds them to aclocal.m4. It also looks in the local file acinclude.m4; if you've written any custom macros for your software package, you should put them in this file. The aclocal utility will make sure they're shuffled properly into place for autoconf. If your project previously used autoconf but not automake, and if you had some custom macros in an aclocal.m4 file, you should move them all to acinclude.m4. If you don't, aclocal will overwrite them with its autogenerated aclocal.m4 file.
The automake script also scans configure.in and uses what it finds to customize its production of Makefile.in files. A prominent example is automake's extra handling of the AC_SUBST macro. As we learned earlier, if you call AC_SUBST on a shell variable in configure.in, the configure script will set up a substitution for your Makefile.in files. For example, if you call AC_SUBST(MYVAR), configure will replace all occurrences of @MYVAR@ in Makefile.in with the value of MYVAR. However, it will not actually add the necessary assignment statements. If you are writing your own Makefile.in files, you will have to add a line like this to make use of that substitution:
MYVAR = @MYVAR@
|
If you're using autoconf only and you don't explicitly add it to Makefile.in, your generated Makefile files will not contain the MYVAR variable at all. On the other hand, if you're using automake, you get all that for free. automake creates your Makefile.in files for you, and as part of its scanning process it automatically adds these assignment lines. Anytime you call AC_SUBST in configure.in, you can be assured that that variable will exist in your makefiles, with no additional work on your part.
Automake Variables
Strictly speaking, all Makefile.am files are simply Makefile.in files that follow special formatting rules. You can use autoconf variable substitutions like @VERSION@, as we discussed in Section 3.1.3, inside your Makefile.am files. You can also embed raw makefile rules and targets when necessary. automake will copy the entire contents of each Makefile.am file into the corresponding Makefile.in file; the configure script will then translate that Makefile.in into a Makefile file, performing variable substitutions as it finds them.
Of course, if that's all automake did, it would be no more helpful than the cp command. automake adds quite a few makefile targets and rules of its own, based on the syntax of the variable names in Makefile.am. Some of these makefile targets exist to satisfy the GNU standards we discussed in Section 3.3.1. Others exist to make the build system more robust and convenient. For example, automake adds dependencies on most of the generated files in the build system, such as the configure script, the Makefile.in and Makefile files, and even config.h. If any of these files have been touched, a simple make command will trigger a rebuild of the affected files. If you tweaked a line in a Makefile.am file in a subdirectory, make will rerun automake for you before continuing.
Part of what allows the automake system to express so much about the make process in so few commands is the elegance of its naming scheme. When automake parses Makefile.am, it looks for a special type of variable name. When it finds the special variables, it creates the necessary makefile targets to handle the contents of that variable properly.
These special variables take the form of target_PRIMARY. target specifies where the contents are supposed to go; PRIMARY tells automake what to do with those contents. automake supports three general types of primaries, each of which determines what sort of contents the variable can contain: binary files, source files, and linker options.
The first type of primary declares binary targets, such as libraries and executables. A variable with the _PROGRAMS primary holds a list of one or more executables; the _LIBRARIES and _LTLIBRARIES primaries both refer to librar- ies. Use _LIBRARIES to create static-only libraries. The _LTLIBRARIES primary invokes the libtool script to create static and dynamic libraries in a platform-independent manner. We'll explore library creation in greater depth in Section 3.4.
The target name for these primaries should indicate the install location for the binary files listed in the variable's contents. The most common variations you'll use are bin_PROGRAMS, lib_LIBRARIES, and lib_LTLIBRARIES. The bin and lib target names correspond directly to the $(bindir) and $(libdir) variables. Thus with the bin_PROGRAMS variable you can specify a list of programs you want installed into whichever $(bindir) directory the administrator specifies at configure time. Here are a few examples:
bin_PROGRAMS = myfirstapp mysecondapp
lib_LIBRARIES = libdothis.a libdothat.a
lib_LTLIBRARIES = libmylibtoollibrary.la
|
You may not always want to install all of your binary files. Sometimes the build process works better if you create transient convenience libraries; if the source code for your target executable is spread across multiple subdirectories, you may have to build a static library for each subdirectory. However, you don't want to install these convenience libraries. They're part of the journey, not the destination. The noinst target exists for this very reason. Any files listed under noinst will be built but not installed. If libdothis.a and libdothat.a in the example here were convenience libraries, we would change the _LIBRARIES declaration to this:
noinst_LIBRARIES = libdothis.a libdothat.a
|
Sometimes you'll have binary files that you do want installed but may or may not want built each time, depending on how configure is invoked. The EXTRA target covers this situation. For example, if you wanted automake always to build myfirstapp, but to build mysecondapp only if configure added it to the @extra_app@ variable substitution, you could list the optional mysecondapp binary in an EXTRA_PROGRAMS variable, and myfirstapp and @extra_app@ in the regular bin_PROGRAMS variable:
EXTRA_PROGRAMS = mysecondapp
bin_PROGRAMS = myfirstapp @extra_app@
|
Then, if @extra_app@ were empty, the makefile would build only myfirstapp; on the other hand, if @extra_app@ contained mysecondapp, the makefile would build both. automake can't just guess at the contents of @extra_app@ because it needs a concrete target name to create the proper de- pendencies. The EXTRA automake target provides you with a place to declare it.
The second category of primaries refers to lists of source code files. The _SOURCES primary, one of the most commonly used primaries, lists source files that must be compiled into object files. These source files are then linked into an executable or library declared in a _PROGRAMS, _LIBRARIES, or _LTLIBRARIES primary. You don't have to include any of your header files in Makefile.am unless you want to install them on the target system. In this case you list the public header files inside a _HEADERS primary, using the same target convention we used for the _PROGRAMS primary. So, if we had a myapp.h and an otherstuff.h file that we wanted to install into $(includedir), we would declare the following variable:
include_HEADERS = myapp.h otherstuff.h
|
The target directory concept is very simple and flexible. automake does not do any unusual processing to the "include" phrase from "include_HEADERS" when converting it into a reference to $(includedir). It simply wraps it with "$(-dir)". In fact, you can define your own directory variables and use them with a primary, just like the include target. You may need to do this if you want to install header files into a subdirectory of $(includedir). To install the header files we've mentioned here into the $(includedir)/myapp directory, you could do something like this:
myappdir = $(includedir)/myapp
myapp_HEADERS = myapp.h otherstuff.h
|
So far, we know how to tell automake where to install our files, but how do we tell it which source files belong with which binary? How do we declare that, for example, main1.c and flubber.c should end up in the myfirstapp executable, and main2.c and dingbat.c are for mysecondapp? The _PROGRAMS primary sets up the target directory for the executable file. We don't need to install the .c files, so it makes no sense to use a directory target for the _SOURCES primary. Instead we use the name of the corresponding binary as the target for the _SOURCES primary (you can also do this with _HEADERS primaries):
myfirstapp_SOURCES = main1.c flubber.c
mysecondapp_SOURCES = main2.c dingbat.c
|
The library primaries work similarly, with a small caveat: Because the period is not a legal character for makefile variables, we cannot have a libdothis.a_SOURCES variable. We must canonicalize it by converting the questionable punctuation into underscores, producing libdothis_a_SOURCES instead. automake expects to see underscores, not hyphens or any other canonicalized character, so don't be creative here. There's only one way to canonicalize an automake variable. The libraries we've already mentioned would look like this:
libdothis_a_SOURCES = dothis.c
libdothat_a_SOURCES = dothat.c
libmylibtoollibrary_la_SOURCES = dotheotherthing.c
|
The _SOURCES and _HEADERS primaries provide a means for you to inform automake about the important compilable and distributable source code files in your project. automake supports some less frequently used primaries for other types of files. For miscellaneous architecture-independent data files, you can use the _DATA primary. This works well for HTML files, as well as special configuration files like the GNOME desktop files (see Section 5.6). By default, _DATA files are not included in a distribution of your project, so you will have to remind automake to include them by listing them in the EXTRA_DIST variable (see Section 3.3.5).
Other related primaries include _SCRIPTS for shell scripts, Perl scripts, and the like; _MANS for man pages; _TEXINFOS for Texinfo documentation; and a few others. Check the automake documentation for more information on how to use these primaries.
The final category of primaries holds command line options for the linker. The _LDFLAGS primary lets you define a list of miscellaneous linker flags for a specific target that don't belong with the other library primaries; it's a catchall. Any variables that use the _LDADD or _LIBADD primaries should contain only object files, libraries (-l), and library paths (-L) to be linked into a specific ex- ecutable (_LDADD) or library (_LIBADD). You cannot mix the contents of any of these three primaries. automake is very particular about what goes into these primaries and will bail out with an error if you misuse them.
Here's what we would do to link our libdothis.a and libdothat.a libraries into our myfirstapp executable:
myfirstapp_LDADD = -ldothis -ldothat
|
We'll go into the specifics of the linker primaries in Section 3.4.1.
Conditional Compiling
Sooner or later you'll want to add support for optional features to one of your projects. Many different approaches have developed over the years for handling this-some of them tidy, and some of them not so tidy. The autoconf and automake systems work together to provide a clean, simple, flexible solution. Through autoconf you can add command line parameters to the configure script that allow administrators to turn your extra features on or off. Automake allows you to express the conditions with familiar if/else/endif statements inside your Makefile.am files.
First we'll discuss setting up the command line parameters. We can add any number of --enable options to the configure script-one per invocation-with another m4 macro, AC_ARG_ENABLE. The macro takes the following form:
AC_ARG_ENABLE (FEATURE, HELP-STRING [, ACTION-IF-GIVEN [,
ACTION-IF-NOT-GIVEN]])
|
The FEATURE is the second half of the enable option. In the example that follows, we test for the feature of whether or not to generate a separate browsing application. We want the option to be --enable-browser, so our FEATURE will be simply browser. The HELP-STRING is the text we want to display for this option inside the configure --help listing. The ACTION parameters are shell script fragments that are executed depending on whether or not configure is invoked with our --enable option. Typically you will want to set a shell variable here; we will use the contents of that variable later, for our conditional tests. Our example sets the shell variable $enable_browser to whatever value is passed into the --enable-browser parameter, or to no if the parameter isn't used. The AC_ARG_ENABLE macro always sets $enableval to the passed-in value.
AC_ARG_ENABLE(browser,
[ --enable-browser build external browser [default=no]],
enable_browser="$enableval", enable_browser="no")
|
Next we must pass the results of this test into the make system. We could do this the hard way, by calling AC_SUBST and rigging up special support code in both configure.in and Makefile.am. Or we could do it the easy way and use the support that's already wired into the AM_CONDITIONAL macro. This macro takes two parameters: the name of a makefile variable, and a shell script command that returns a Boolean value. You'll use the first parameter to test for the condition in your Makefile.am file; AM_CONDITIONAL calls AC_SUBST on this parameter, so it will be available in your makefiles. autoconf will insert the second parameter into an if/else/fi statement in the configure shell script to set the value of the first parameter. The test for our theoretical browser might look something like this:
AM_CONDITIONAL(ENABLE_BROWSER, test "$enable_browser" = "yes")
|
In your Makefile.am you can include an if/else/endif block to test against the first parameter in AM_CONDITIONAL. You'll usually want to set one or more makefile variables inside the if block to have different values for each conditional branch. You can then refer to these variables in the rest of your Makefile.am file, instead of referring to their contents directly. Here's the imaginary Makefile.am file for our browser example:
if ENABLE_BROWSER
BROWSER_APP = mybrowser
else
BROWSER_APP =
endif
bin_PROGRAMS = mainapp $(BROWSER_APP)
mybrowser_SOURCES = browser_main.c
mainapp_SOURCES = main.c
|
Generally it is better to avoid putting the special automake variables, like bin_PROGRAMS, inside your if statements. automake seems to become confused when you do this in certain ways. Thus you should avoid constructs like this:
if ENABLE_BROWSER
bin_PROGRAMS = myapp mybrowser
else
bin_PROGRAMS = myapp
endif
|
These if/else/endif blocks are purely an automake construct; they don't even make it to the Makefile.in file. When automake sees an if block, it creates a separate set of variable declarations for each condition, inside the Makefile.in file. At configure time, each block of variables will either be left alone or commented out, depending on which configure options the administrator selects. For example, the Makefile file created by running ./configure --enable-browser would have the following entries:
BROWSER_APP = mybrowser
#BROWSER_APP =
|
Running configure without --enable-browser would create these entries instead:
#BROWSER_APP = mybrowser
BROWSER_APP =
|
You can also base your AM_CONDITIONAL tests on the results of other autoconf macros. This is easy with the AC_CHECK_* macros. All you have to do is call the true command for ACTION-IF-FOUND, and the false command for ACTION-IF-NOT-FOUND. Gnome-libs uses the following test to optionally extract documentation from the source files if the gtkdoc-mkdb utility exists:
AC_CHECK_PROG(GTKDOC, gtkdoc-mkdb, true, false)
AM_CONDITIONAL(HAVE_GTK_DOC, $GTKDOC)
|
These basic tools are very flexible if you know how to use them. You are free to use whatever conditional tests you need, and you can choose whatever variable names are most descriptive for the task at hand. If you do run into trouble, don't hesitate to crack open the generated Makefile and see what's going on inside. Try changing subtle parts of your Makefile.am file and see how those changes affect Makefile. Run the configure script with different op- tions. You can learn a lot with commands like
diff -u Makefile.before Makefile.after
|
Cutting a Distribution
Creating a source code distribution is very easy with an automake project. You practically get it for free. automake always creates a make dist target in your topmost makefile. This target grabs the distributable files in your project tree, and runs tar and gzip on them to create a file of the form PACKAGE-VERSION.tar.gz, using the values of PACKAGE and VERSION from the AM_INIT_AUTOMAKE macro.
In addition, automake creates the make distcheck target, a superset of make dist that will create the compressed archive, or tarball, extract it into a temporary directory, attempt to build it, and then report on whether or not the build was successful. This is a good way to make sure your distribution is in good shape, that you haven't forgotten to include any necessary files, and that your code compiles all the way through without errors. The distcheck target cleans up its mess, leaving only the tarball. Note that distcheck only verifies the build process; it does not try to actually run the application, so it won't pick up any runtime problems.
By default, automake distributes any files you list under the _SOURCES and _HEADERS primaries. A complete distribution requires many other standard files, many of which are created by configure, libtoolize, and other utili- ties. automake implicitly includes these in any distribution it creates. You can get a full list of these files by running automake --help.
To make life easier for users of your distributions, automake also includes the Makefile.in files from your source tree. This means that the automake and autoconf packages are not a requirement for anyone installing the distribution. The user needs only run the provided configure script, which in turn translates the Makefile.in files into Makefile files. To avoid grabbing any rogue Makefile.in files in your source tree that don't actually belong in the distribution, automake scans through the master list of generated files in AC_OUTPUT.
This takes care of all the files automake will automatically put into a distribution. In many projects this will cover all the files you need to distribute: the source code and the prepackaged build system. However, sometimes you will have additional files that are important to your application but aren't necessarily source code or build scripts. You might have some graphics or sound files, or end user documentation, or shell scripts. automake provides a catchall variable, EXTRA_DIST, where you can list all these extra files. Also, since EXTRA_DIST does not have to pass through the make process, it is not subject to limitations from the makefile subdirectory boundaries. For this reason you can include files in subdirectories; furthermore, if you add a directory name to EXTRA_DIST, automake recursively copies that entire directory into the distribution. This feature is great if you have many media files to distribute and don't want to have to create a Makefile.am file for every subdirectory.
Occasionally you will need to do some processing of the distribution tree just before make dist archives it. Perhaps you need to patch up some files, or remove an intermediate file that's listed in a _SOURCES variable but shouldn't actually be distributed. To handle this task, automake supplies a customized dist-hook target. You can declare this target inside your Makefile.am file, using the $(distdir) variable to refer to any files or paths inside the distri- bution directory. For example, to move a directory of graphics files from extra/media/graphics to browser/graphics in the distribution, you could add the following to Makefile.am:
dist-hook:
mv $(distdir)/extra/media/graphics $(distdir)/browser/graphics
|
A common courtesy in any complex software package is the test suite. According to the GNU standards, the make check target builds and optionally runs this test suite but does not install it. You should always write your test suites to run inside the source tree, prior to installation. If the administrator has to install your software before running the test suite, the whole purpose of the test suite is defeated.
To set up a check target, add the check_PROGRAMS variable to your Makefile.am file, and a corresponding _SOURCES primary. If you stop there, the make check target will build the test programs but will not run them. You can get make check to automatically run the tests and produce a report of which ones passed and which ones failed by adding your test programs to the TESTS variable in Makefile.am:
TESTS = myapp-test
check_PROGRAMS = myapp-test
myapp_test_SOURCES = myapp-test.c
|