7.3.2.1 The C Compiler as Pipeline

As a final example of eval, we'll revisit our old friend occ, the C compiler from the previous three chapters. Recall that the compiler does its work by calling separate programs to do the actual compile from C to object code (the ccom program), optimization of object code (optimize), assembly of assembler code files (as), and final linking of object code files into an executable program (ld). These separate programs use temporary files to store their outputs.

Now we'll assume that these components (except the linker) pass information in a pipeline to the final object code output. In other words, each component takes standard input and produces standard output instead of taking filename arguments. We'll also change an earlier assumption: instead of compiling a C source file directly to object code, occ compiles C to assembler code, which the assembler then assembles to object code. This lets us suppose that occ works like this:

ccom < filename.c | as | optimize > filename.o

Or, if you prefer:

cat filename.c | ccom | as | optimize > filename.o

To get this in the proper framework for eval, let's assume that the variables srcname and objname contain the names of the source and object files, respectively. Then our pipeline becomes:

cat $srcname | ccom | as | optimize > $objname

As we've already seen, this is equivalent to:

eval cat \$srcname | ccom | as | optimize > \$objname

Knowing what we do about eval, we can transform this into:

eval cat \$srcname " | ccom" " | as" " | optimize" > \$objname

and from that into:

compile=" | ccom"
assemble=" | as"
optimize=" | optimize"

eval cat \$srcname \$compile \$assemble \$optimize > \$objname

Now, consider what happens if you don't want to invoke the optimizer - which is the default case anyway. (Recall that the -O option invokes the optimizer.) We can do this:

optimize=""
if -O given then
    optimize=" | optimize"
fi

In the default case, $optimize evaluates to the empty string, causing the final pipeline to "collapse" into:

eval cat $srcname | ccom | as > $objname

Similarly, if you pass occ a file of assembler code (filename.s), you can collapse the compile step: [12]

[12] Astute readers will notice that, according to this rationale, we would handle object-code input files (filename.o) with the pipeline eval cat $srcname > $objname, where the two names are the same. This will cause UNIX to destroy filename.o by truncating it to zero length. We won't worry about this here.

assemble="| as"
if $srcname ends in .s then
    compile=""
fi

That results in this pipeline:

eval cat \$srcname | as > \$objname

Now we're ready to show the full "pipeline" version of occ. It's similar to the previous version, except that for each input file, it constructs and runs a pipeline as above. It processes the -g (debug) option and the link step in the same way as before. Here is the code:

# initialize option-related variables
do_link=true
debug=""
link_libs="-l c"
exefile=""

# initialize pipeline components
compile=" | ccom"
assemble=" | as"
optimize=""

# process command-line options
while getopts ":cgl:o:O" opt; do
    case $opt in 
        c )    do_link=false ;;
        g )    debug="-g" ;;
        l )    link_libs="$link_libs -l $OPTARG" ;;
        o )    exefile="-o $OPTARG" ;;
        O )    optimize=" | optimize" ;;
        \? )   print 'usage: occ [-cgO] [-l lib] [-o file] files...'
               return 1 ;;
    esac
done
shift $(($OPTIND - 1))

# process the input files
for filename in "$@"; do
    case $filename in
	*.c )
	    objname=${filename%.c}.o ;;
	*.s )
	    objname=${filename%.s}.o
	    compile="" ;;
	*.o )
	    compile=""
	    assemble="" ;;
	*   )
        print "error: $filename is not a source or object file."
        return 1 ;;
    esac

# run a pipeline for each input file
eval cat \$filename \$compile \$assemble \$optimize > \$objname
    objfiles=$objfiles" "$objname
done

if [[ $do_link = true ]]; then
    ld $exefile $link_libs $objfiles
fi

We could go on forever with increasingly complex examples of eval, but we'll settle for concluding the chapter with a few exercises. The last two are really more like items on the menu of food for thought; the very last one is particularly difficult.

1. Here are a couple of ways to enhance occ, our C compiler:

   1. Real-world C compilers accept the option -S, which tells the compiler to suppress the assembly step and leave the output in files of assembler code whose names end in .s. Modify occ so that it recognizes this option.

   2. The language C++ is an evolutionary successor to C; it includes advanced features like operator overloading, function argument type checking, and class definitions. (Don't worry if you don't know what these are.) Some C++ compilers use C as an "assembly language", i.e., they compile C++ source files to C code and then pass them to a C compiler for further processing. Assume that C++ source files have names ending in .cc, and that /lib/cfront is the C++ compiler "front-end" that produces C code on its standard output. Modify occ so that it accepts C++ as well as C, assembler, and object code files.

2. The possibilities for customizing your prompt string are practically endless. Here are two enhancements to customization schemes that we've seen already:

   1. Enhance the current-directory-in-the-prompt scheme by limiting the prompt string's length to a number of characters that the user can define with an environment variable.

   2. On some UNIX systems, it's not possible to get a list of all users by looking at /etc/passwd. For example, networks of Suns use the Network Information Service (NIS, a.k.a. "Yellow Pages"), which stores a protected password file for the entire network on one server machine, instead of having separate /etc/passwd files on each machine.

      If such a machine is set up so that all login directories are under a common directory (e.g., /users), you can get a list of all users by simply ls-ing that directory. Modify the tildize function so that it uses this technique; pay particular attention to execution speed.

3. The function makecmd in the solution to Task 7-6 represents an oversimplification of the real make's functionality. make actually checks file dependencies recursively, meaning that a source on one line in a makefile can be a target on another line. For example, the book chapters in the example could themselves depend on some figures in separate files that were made with a graphics package.

   1. Write a function called readtargets that goes through the makefile and stores all of the targets in a variable or temp file.

   2. Instead of reading the makefile from standard input, read it into an array variable called lines. Use the variable curline as the "current line" index. Modify makecmd so that it reads lines from the array starting with the current line.

   3. makecmd merely checks to see if any of the sources are newer than the given target. It should really be a recursive routine that looks like this:

      function makecmd {
          target=$1
          get sources for $target
          for each source src; do
              if $src is also a target in this makefile then
                  makecmd $src
              fi
              if [[ $src -nt $target ]]; then
                  run commands to make target
                  return
              fi
          done
      }

      Implement this.

   4. Write the "driver" script that turns the makecmd function into a full make program. This should make the target given as argument, or if none is given, the first target listed in the makefile.

   5. The above makecmd still doesn't do one important thing that the real make does: allow for "symbolic" targets that aren't files. These give make much of the power that makes it applicable to such an incredible variety of situations. Symbolic targets always have a modification time of 0, so that make always runs the commands to make them. Modify makecmd so that it allows for symbolic targets. (Hint: the crux of this problem is to figure out how to get a file's modification time. This is quite difficult.)

4. Finally, here are some problems that really test your knowledge of eval and the shell's command-line processing rules. Solve these and you're a true Korn shell hacker!

   1. Advanced shell programmers sometimes use a little trick that includes eval: using the value of a variable as the name of another variable. In other words, you can give a shell script control over the names of variables to which it assigns values. How would you do this? (Hint: if $fred equals "dave", and $dave is "bob", then you might think that you could type print $$fred and get the response bob. This doesn't actually work, but it's on the right track.)

   2. You could use the above technique together with other eval tricks to implement new control structures for the shell. For example, see if you can write a script that emulates the behavior of a for loop in a conventional language like C or Pascal, i.e., a loop that iterates a fixed number of times, with a loop variable that steps from 1 to the number of iterations (or, for C fans, 0 to iterations-1). Call your script loop to avoid clashes with the keywords for and do.