Generated Header

Here we will show how to write a Tupfile to handle a fairly common use-case: an automatically generated header file.

Handling auto-generated header files in tup is actually fairly easy, though at first glance it may appear to be tricky. They are not completely automatic, however. You do still need to specify the generated header as an input to any compilation rules. Let's get a C program from the First Tupfile example to start with. Later we will generate a new header file from a separate shell script and see how that affects our Tupfile.

$ tup init tup_test3
$ cd tup_test3
Tupfile 
: foreach *.c |> gcc -Wall -c %f -o %o |> %B.o
: *.o |> gcc %f -o %o |> hello
hello.c 
#include <stdio.h>
#include "square.h"

int main(void)
{
	printf("Hi, everybody!\n");
	printf("Five squared is: %i\n", square(5));
	return 0;
}
square.c 
#include "square.h"

int square(int x)
{
	return x * x;
}
square.h 
int square(int x);

Now we can run our code and get the original output:

$ tup
[ tup ] Scanning filesystem...0.005s
[ tup ] No tup.config changes.
[ tup ] Parsing Tupfiles...
[    1/1    ] .
[ tup ] No files to delete.
[ tup ] Executing Commands...
[    1/3    ] gcc -Wall -c square.c -o square.o
[    2/3    ] gcc -Wall -c hello.c -o hello.o
[    3/3    ] gcc hello.o square.o -o hello
[ tup ] Updated.
$ ./hello
Hi, everybody!
Five squared is: 25

Suppose we want to add a new interface, only for reasons beyond our control, we have to generate the header file from a shell script rather than write it by hand. This is not a problem - we just have to tell tup how to create the header file.

gen_triangle.sh 
#! /bin/sh
echo '#define SIDES_OF_A_TRIANGLE 3'
Tupfile 
: |> sh gen_triangle.sh > %o |> triangle.h
: foreach *.c |> gcc -Wall -c %f -o %o |> %B.o
: *.o |> gcc %f -o %o |> hello

$ tup
[ tup ] Scanning filesystem...0.009s
[ tup ] No tup.config changes.
[ tup ] Parsing Tupfiles...
[    1/1    ] .
[ tup ] No files to delete.
[ tup ] Executing Commands...
[    1/1    ] sh gen_triangle.sh > triangle.h
[ tup ] Updated.
$ cat triangle.h
#define SIDES_OF_A_TRIANGLE 3

It looks like our new header file was generated properly. Let's try to include it in our hello.c file:

hello.c 
#include <stdio.h>
#include "square.h"
#include "triangle.h"

int main(void)
{
	printf("Hi, everybody!\n");
	printf("Five squared is: %i\n", square(5));
	printf("A triangle has %i sides\n", SIDES_OF_A_TRIANGLE);
	return 0;
}

$ tup
[ tup ] Scanning filesystem...0.007s
[ tup ] No tup.config changes.
[ tup ] No Tupfiles to parse.
[ tup ] No files to delete.
[ tup ] Executing Commands...
[    1/2    ] gcc -Wall -c hello.c -o hello.o
 *** tup errors ***
tup error: Missing input dependency - a file was read from, and was not
specified as an input link for the command. This is an issue because the file
was created from another command, and without the input link the commands may
execute out of order. You should add this file as an input, since it is
possible this could randomly break in the future.
 - [17] triangle.h
 *** Command ID=8 ran successfully, but tup failed to save the dependencies.

Oops, looks like we stumbled onto the missing input dependency error message again. Since triangle.h is an output of another command, we have to tell tup about it in the input section of the compilation rule. Right now the input section of our compilation rule (highlighted below) has 'foreach *.c':

: foreach *.c |> gcc -Wall -c %f -o %o |> %B.o

Note that we can't simply list triangle.h as another input here, because the foreach keyword will try to iterate over it and generate a rule to compile the header file directly. Instead, we want to make use of the order-only inputs section, which is separated from the rest of the inputs by a single '|' (pipe) character. The difference between regular inputs and order-only inputs is that regular inputs are used in foreach and flags like %f, while order-only inputs are not. All inputs also add dependency links from the file pointing to the commands.

Tupfile 
: |> sh gen_triangle.sh > %o |> triangle.h
: foreach *.c | triangle.h |> gcc -Wall -c %f -o %o |> %B.o
: *.o |> gcc %f -o %o |> hello

Now we can finish the update and take a look at the dependency graph:

$ tup
[ tup ] Scanning filesystem...0.008s
[ tup ] No tup.config changes.
[ tup ] Parsing Tupfiles...
[    1/1    ] .
[ tup ] No files to delete.
[ tup ] Executing Commands...
[    1/2    ] gcc -Wall -c hello.c -o hello.o
[    2/2    ] gcc hello.o square.o -o hello
[ tup ] Updated.
$ tup graph . | dot -Tpng > ~/ex_autogen_1.png

Even though we specified triangle.h as an input dependency for both compilation rules, you can see that changing the gen_triangle.sh script only re-compiles the one file that actually includes the header. In contrast, changing square.h re-compiles both files, since they both include that header. The compilation commands are highlighted below:

$ touch gen_triangle.sh
$ tup
[ tup ] Scanning filesystem...0.006s
[ tup ] No tup.config changes.
[ tup ] No Tupfiles to parse.
[ tup ] No files to delete.
[ tup ] Executing Commands...
[    1/3    ] sh gen_triangle.sh > triangle.h
[    2/3    ] gcc -Wall -c hello.c -o hello.o
[    3/3    ] gcc hello.o square.o -o hello
[ tup ] Updated.
$ touch square.h
$ tup
[ tup ] Scanning filesystem...0.006s
[ tup ] No tup.config changes.
[ tup ] No Tupfiles to parse.
[ tup ] No files to delete.
[ tup ] Executing Commands...
[    1/3    ] gcc -Wall -c square.c -o square.o
[    2/3    ] gcc -Wall -c hello.c -o hello.o
[    3/3    ] gcc hello.o square.o -o hello
[ tup ] Updated.

Although it is difficult to show in this example, the presence of the dotted-line dependency from triangle.h to the gcc -Wall -c square.c -o square.o command is very important. If we were to, in a single patch perhaps, change the gen_triangle.sh script, and change square.c to include the generated header, the dependency ensures that the header is generated before the C file is compiled. Until the C file actually includes the header, however, the dependency link is somewhat dormant - changing the header will not cause the C file to be re-compiled.

The result is that tup only re-compiles the minimum number of files necessary, even if inputs in the Tupfile are over-specified. Additionally, inputs in the Tupfile always guarantee the order that the commands are executed in, even if the commands do not actually read from the input files.