The DocStrip program ^{∗}

Frank Mittelbach

            Mittelbach@mzdmza.zdv.uni-mainz.de

Denys Duchier

    duchier-denys@cs.yale.edu

Johannes Braams TeXniek Kooienswater 62 2715 AJ Zoetermeer

                                 JLBraams@cistron.nl

Marcin Woliński

               wolinski@bull.mimuw.edu.pl

Printed March 27, 1996

Abstract This document describes the implementation of the DocStrip program. The original version of this program was developed by Frank Mittelbach to accompany his doc.sty which enables literate programming in L^{A}TEX. Denys Duchier rewrote it to run either with TEX or with L^{A}TEX, and to allow full boolean expressions in conditional guards instead of just comma-separated lists. Johannes Braams re-united the two implementations, documented and debugged the code.

1 Preface to version 2.3

Basically this version is just faster than previous ones. It is easy to observe that while installing large packages DocStrip reads some files more than once. This is done when several output files are to be generated from one input. But TEX can write to up to 16 files on the same time. Version 2.3 takes advantage of this to speed things up.

A short example: 12 of L^{A}TEX 2ε files are generated from file classes.dtx. All of them can be generated while reading source once. New version of DocStrip uses one fourth of time needed by older versions to do this. However when installing whole L^{A}TEX 2ε most of the time is spent on generating file latex.ltx which is just one, so installation of L^{A}TEX takes about half the previous time.

The bad news is that DocStrip’s interface had to be changed somewhat to be able to specify which files can be generated concurrently.

∗This file has version number 2.3d, last revised 1995/11/17, documentation dated 1995/11/18.

2 Introduction

2.1 Why the DocStrip program?

When Frank Mittelbach created the doc package, he invented a way to combine TEX code and its documentation. From then on it was more or less possible to do literate programming in TEX.

This way of writing TEX programs obviously has great advantages, especially when the program becomes larger than a couple of macros. There is one drawback however, and that is that such programs may take longer than expected to run because TEX is an interpreter and has to decide for each line of the program file what it has to do with it. Therefore, TEX programs may be speeded up by removing all comments from them.

By removing the comments from a TEX program a new problem is introduced. We now have two versions of the program and both of them have to be maintained. Therefore it would be nice to have a possibility to remove the comments automatically, instead of doing it by hand. So we need a program to remove comments from TEX programs. This could be programmed in any high level language, but maybe not everybody has the right compiler to compile the program. Everybody who wants to remove comments from TEX programs has TEX. Therefore the DocStrip program is implemented entirely in TEX.

2.2 Functions of the DocStrip program

Having created the DocStrip program to remove comment lines from TEX programs^{1} it became feasible to do more than just strip comments. Wouldn’t it be nice to have a way to include parts of the code only when some condition is set true? Wouldn’t it be as nice to have the possibility to split the source of a TEX program into several smaller files and combine them later into one ‘executable’? Both these wishes have been implemented in the DocStrip program.

2.3 How to use the DocStrip program

A number of ways exist to use the DocStrip program:

1. The easiest way is to instruct TEX to read the file docstrip.tex and to see what happens. TEX will ask you a few questions about the file you would like to be processed. When you have answered these questions it does its job and strips the comments from your TEX code.

2. When you would like to be able to process more than one file with one command, or you would like to add a couple of lines to the beginning (or end) of the stripped file you can write a ‘batch file’ for DocStrip. The default name for such a file is docstrip.cmd. When a file by that name exists in your current directory when you instruct TEX to read docstrip.tex it will verify (by asking you) that you want to use that file and if so, will process the commands in it.

1Note that only comment lines, that is lines that start with a single % character, are removed; all other comments st in the code.

You do not need to use the default name, however. You can give a batch file any name you like such as, say, foo.bar and then run TEX by using a command such as:

    TeX \def\batchfile{foo.bar}\input docstrip

3. You may use yet another way that is only slightly different from the previous method. You can write a batch file in such a way that it can be directly processed by TEX. This allows you to set up a distribution where you can instruct the user to simply run

TEX ⟨batch file⟩

to generate the executable versions of your files from the distribution sources. All distributions from Mainz are set up in this way (also this one); look, for example, at the installation file for this distribution. To produce such a batch file include a statement in your ‘batch file’ that instructs TEX to read docstrip.tex. You should then also define \batchfile to contain the name of that ‘batch file’. The beginning of such a file would look like:

    \def\batchfile{install.me}
    \input docstrip
    ...

3 The user interface

3.1 The main program

\processbatchFile The ‘main program’ starts with trying to process a batch file, this is accomplished by calling the macro \processbatchFile. It counts the number of batch files it processes, so that when the number of files processed is still zero after the call to \processbatchFile appropriate action can be taken.

\interactive When no batch files have been processed the macro \interactive is called. It prompts the user for information. First the extensions of the input and output files is determined. Then a question about optional code is asked and finally the user can give a list of files that have to be processed.

\ReportTotals When the stats option is included in the DocStrip-program it keeps a record of the number of files and lines that are processed. Also the number of comments removed and passed as well as the number of code lines that were passed to the output are accounted. The macro \ReportTotals shows a summary of this information.

3.2 Batchfile commands

The commands described in this section are available to build a batch file.

3.2.1 Old interface

\showprogress When the option stats is included in DocStrip it can write message to the terminal

\keepsilent as each line of the input file(s) is processed. This message consists of a single character, indicating kind of that particular line. We use the following characters:

% Whenever an input line is a comment %-character is written to the terminal.

. Whenever a code line is encountered a .-character is written on the terminal.

/ When a number of empty lines appear in a row in the input file, at most one of them is retained. The DocStrip program signals the removal of an empty line with the /-character.

< When a ‘guard line’ is found in the input and it starts a block of optionally included code, this is signalled on the terminal by showing the <-character, together with the boolean expression of the guard.

> The end of a conditionally included block of code is indicated by showing the >-character.

This feature is turned on by default when the option stats is included, otherwise it is turned of. The feature can be toggled with the commands \showprogress

and \keepsilent.

\preamble It is possible to add a number of lines to the output of the DocStrip pro-

\endpreamble
\postamble
\endpostamble

gram. The information you want to add to the start of the output file should be listed between the \preamble and \endpreamble commands; the lines you want to add to the end of the output file should be listed between the \postamble and \endpostamble commands. Everything that DocStrip finds for both the pre- and postamble it writes to the output file, but preceded with two %-characters. If you include a ^^M character in one of these lines, everything that follows it on the same line is written to a new line in the output file. This ‘feature’ can be used to add a \typeout or \message to the the stripped file.

\generateFile The main reason for constructing a DocStrip command file is that one doesn’t want to type in the instructions for TEX each time a macro file is stripped of its comments. The macro \generateFile is used to tell TEX what to do. Its syntax is:

\generateFile{⟨output⟩}{⟨ask⟩}{[\from{⟨input⟩}{⟨optionlist⟩}]*}

The ⟨output⟩ and ⟨input⟩ are normal file specifications as are appropriate for your computer system. The ⟨optionlist⟩ is a comma separated list of ‘options’ that specify which optional code fragments in ⟨input⟩ should be included in ⟨output⟩. With ⟨ask⟩ you can instruct TEX to either silently overwrite a previously existing file (f) or to issue a warning and ask you if it should overwrite the existing file (t).

It is possible to specify multiple input files, each with its own ⟨optionlist⟩. This is indicated by the notation […]*.

\include The earlier version of the DocStrip program supported a different kind of com- \processFile mand to tell TEX what to do. This command is less powerful than \generateFile;

it can be used when ⟨output⟩ is created from one ⟨input⟩. The syntax is:

\include{⟨optionlist⟩}

\processFile{⟨name⟩}{⟨inext⟩}{⟨outext⟩}{⟨ask⟩}

This command is based on environments where filenames are constructed of two parts, the name and the extension, separated with a dot. The syntax of this command assumes that the ⟨input⟩ and ⟨output⟩ share the same name and only differ in their extension. This command is retained to be backwards compatible with the older version of DocStrip, but its use is not encouraged.

\input The batch file commands can be put into several batch files which are then \batchinput executed from a master batch file. This is, for example, useful if a distribution

consists of several distinct parts. You can then write individual batch files for every part and in addition a master file that simply calls the batch files for the parts. For this, call the individual batch files from the master file with the command \batchinput{⟨file⟩}. You can nest batch files up to a nesting level of ten which is more than sufficient. Don’t use \input for this purpose, this command should be used only for calling the DocStrip program as explained above and is ignored when used for any other purpose.

\ifToplevel When batch files are nested you may want to suppress certain commands in the lower-level batch files such as terminal messages. For this purpose you can use the \ifToplevel command which executes its argument only if the current batch file is the outermost one. Make sure that you put the opening brace of the argument into the same line as the command itself, otherwise the DocStrip program will get confused.

3.2.2 New interface

These are new commands added in version 2.3. They can be mixed freely with all old commands.

\generate Here is the new syntax for specifing what files are to be generated:

\file
\from

\generate{[\file{⟨output⟩}{[\from{⟨input⟩}{⟨optionlist⟩}]*}]*}

The meaning of ⟨output⟩, ⟨input⟩ and ⟨optionslist⟩ is just as for \generateFile. But there can be many \file specifications in one \generate clause. This means that all these ⟨output⟩ files should be generated while reading each of ⟨input⟩ files once. Input files are read in order of first appearance in this clause. E.g.

     \generate{\file{p1.sty}{\from{s1.dtx}{foo,bar}}
               \file{p2.sty}{\from{s2.dtx}{baz}
                             \from{s3.dtx}{baz}}
               \file{p3.sty}{\from{s1.dtx}{zip}
                             \from{s2.dtx}{zip}}

              }

will cause DocStrip to read files s1.dtx, s2.dtx, s3.dtx (in that order) and pro-

duce files p1.sty, p2.sty, p3.sty.

Restriction to at most 16 output streams open in a while does not mean that you can produce at most 16 files with one \generate. In the example above only 2 streams are needed, since while s1.dtx is processed only p1.sty and p3.sty are being generated; while reading s2.dtx only p2.sty and p3.sty; and while reading s3.dtx file p2.sty . However example below needs 3 streams:

     \generate{\file{p1.sty}{\from{s1.dtx}{foo,bar}}
               \file{p2.sty}{\from{s2.dtx}{baz}
                             \from{s3.dtx}{baz}}
               \file{p3.sty}{\from{s1.dtx}{zip}
                             \from{s3.dtx}{zip}}
              }

Although while reading s2.dtx file p3.sty is not written it must remain open since some parts of s3.dtx will go to it later.

Sometimes it is not possible to create a file by reading all sources once. Consider the following example:

     \generate{\file{p1.sty}{\from{s1.dtx}{head}
                             \from{s2.dtx}{foo}
                             \from{s1.dtx}{tail}}
               \file{s1.drv}{\from{s1.dtx}{driver}}
              }

To generate p1.sty file s1.dtx must be read twice: first time with option head, then file s2.dtx is read and then s1.dtx again this time with option tail. Doc-Strip handles this case correctly: if inside one \file declaration there are multiple \fromes with the same input file this file is read multiple times.

If order of \froms specified in one of your \file specifications does not match the order of input files established by previous \files DocStrip will raise an error and abort. Then you may either read one of next sections or give up and put that file in separate \generate (but then sources will be read again just for that file).

For impatient. Try following algorithm: Find file that is generated from largest number of sources, start writing \generate clause with this file and its sources in proper order. Take other files that are to be generated and add them checking if they don’t contradict order of sources for the first one. If this doesn’t work read next sections.

For mathematicians. Relation “file A must be read before file B” is a partial order on the set of all your source files. Each \from clause adds a chain to this order. What you have to do is to perform a topological sort i.e. to extend partial order to linear one. When you have done it just list your source files in \generate in such a way that order of their first appearance in the clause matches linear order. If this cannot be achieved read next paragraph. (Maybe future versions of DocStrip will perform this sort automatically, so all these troubles will disappear.)

For that who must know that all. There is a diverse case when it’s not possible to achieve proper order of reading source files. Suppose you have to generate two files, first from s1.dtx and s3.dtx (in that order) and second from s2.dtx and s3.dtx. Whatever way you specify this the files will be read in either as s1 s3 s2 or s2 s3 s1. The key to solution is magical macro \needed that marks a file as needed to be input but not directing any output from it to current \file. In our example proper specification is:

     \generate{\file{p1.sty}{\from{s1.dtx}{foo}
                             \needed{s2.dtx}
                             \from{s3.dtx}{bar}}
               \file{p2.sty}{\from{s2.dtx}{zip}
                             \from{s3.dtx}{zap}}
              }

Argument to \generate is executed inside a group, so all local declarations are undone when \generate concludes.

\askforoverwritetrue As you can see in the syntax above there is no place for ⟨ask⟩ specifier. Since in \askforoverwritefalse all installation scripts I’ve seen all settings for that were the same (t or f depending

on an author) I thought that it will be more convenient to specify if user should be asked for overwriting files just in one place. These switches are local and can be issued in any place in the file even inside \generate clause (between \files however).

\declarepreamble Sometimes it is desirable to have different preambles for different files

\declarepostamble
\usepreamble
\usepostamble

of a larger package (e.g. because some of them are customisable configuration files and they should be marked as such). In such case one can say \declarepreamble\somename, then type in his/her preamble, end it with \endpreamble, and later on \usepreamble\somename to switch to this preamble. The same mechanism works for postambles, \use... declarations are local and can appear inside \generate.

Commands \preamble and \postamble define and activate pre(post)ambles

named \defaultpreamble and \defaultpostamble.

4 Conditional inclusion of code

When you use the DocStrip program to strip comments out of TEX macro files you have the possibility to make more than one stripped macro file from one documented file. This is achieved by the support for optional code. The optional code is marked in the documented file with a ‘guard’.

A guard is a boolean expression that is enclosed in < and >. It also has to follow the % at the beginning of the line. For example:

    ...
    %<bool>\TeX code
    ...

In this example the line of code will be included in ⟨output⟩ if the option bool is present in the ⟨optionlist⟩ of the \generateFile command.

The syntax for the boolean expressions is:

⟨Expression⟩ ::= ⟨Secondary⟩ [{|, ,} ⟨Secondary⟩]*
⟨Secondary⟩ ::= ⟨Primary⟩ [& ⟨Primary⟩]*
⟨Primary⟩  ::= ⟨Terminal⟩ | !⟨Primary⟩ | (⟨Expression⟩)

The | stands for disjunction, the & stands for conjunction and the ! stands for negation. The ⟨Terminal⟩ is any sequence of letters and evaluates to ⟨true⟩ iff^{2} it occurs in the list of options that have to be included.

Two kinds of optional code are supported: one can either have optional code that ‘fits’ on one line of text, like the example above, or one can have blocks of optional code.

To distinguish both kinds of optional code the ‘guard modifier’ has been introduced. The ‘guard modifier’ is one character that immediately follows the < of the guard. It can be either * for the beginning of a block of code, or / for the end of a block of code^{3}. The beginning and ending guards for a block of code have to be on a line by themselves.

When a block of code is not included, any guards that occur within that block are not evaluated.

5 Producing the documentation

We provide a short driver file that can be extracted by the DocStrip program using the the conditional ‘driver’. To allow the use of docstrip.dtx as a program at IniTEX time (e.g., to strip off its own comments) we need to add a bit of primitive code. With this extra checking it is still possible to process this file with L^{A}TEX 2ε to typeset the documentation.

1 ⟨∗driver⟩

If \documentclass is undefined, e.g., if IniTEX or plain TEX is used for formatting, we bypass the driver file.

We use some trickery to avoid issuing \end{document} when the \ifx construction is unfinished. If condition below is true a \fi is constructed on the fly, the \ifx is completed, and the real \fi will never be seen as it comes after \end{document}. On the other hand if condition is false TEX skips over \csname fi\endcsname having no idea that this could stand for \fi, driver is skipped and only then the condition completed.

2iff stands for ‘if and only if’

3To be compatible with the earlier version of DocStrip also + and - are supported as ‘guard modifiers’. However, there is an incompatibility with the earlier version since a line with a +- modified guard is not included inside a block with a guard that evaluates to false, in contrast to the previous behaviour.

Additional guard gobble prevents DocStrip from extracting these tricks to real driver file.

2 ⟨∗gobble⟩

 3 \ifx\documentclass\undefined
 4 \else \csname fi\endcsname

5 ⟨/gobble⟩

Otherwise we process the following lines which will result in formatting the documentation.

 6 \documentclass{ltxdoc}
 7  \EnableCrossrefs
 8 %    \DisableCrossrefs    % use \DisableCrossrefs if the
 9  % index is ready
10 \RecordChanges
11 %   \OnlyDescription
12 \typeout{Expect some Under- and overfull boxes}
13 \begin{document}
14 \DocInput{docstrip.dtx}
15 \end{document}

16 ⟨∗gobble⟩

17 \fi

18 ⟨/gobble⟩ 19 ⟨/driver⟩

6 The implementation

6.1 Declarations and initializations

In order to be able to include the @-sign in control sequences its category code is changed to ⟨letter⟩. The ‘program’ guard here allows most of the code to be excluded when extracting the driver file.

20 ⟨∗program⟩

21 \catcode`\@=11

When we want to write multiple lines to the terminal with one statement, we need a character that tells TEX to break the lines. We use ^^J for this purpose.

22 \newlinechar=`\^^J

6.1.1 Switches

\ifGenerate The program will check if a file of the same name as the file it would be creating already exists. The switch \ifGenerate is used to indicate if the stripped file has to be generated.

23 \newif\ifGenerate

\ifContinue The switch \ifContinue is used in various places in the program to indicate if a \loop has to end.

24 \newif\ifContinue

\ifForlist The program contains an implementation of a for-loop, based on plain TEX’s \loop macros. The implementation needs a switch to terminate the loop.

25 \newif\ifForlist

\ifDefault The switch \ifDefault is used to indicate whether the default batch file has to be used.

26 \newif\ifDefault

\ifMoreFiles The switch \ifMoreFiles is used to decide if the user wants more files to be processed. It is used only in interactive mode; initially it evaluates to ⟨true⟩.

27 \newif\ifMoreFiles \MoreFilestrue

\ifaskforoverwrite The switch \askforoverwrite is used to decide if the user should be asked when a file is to be overwritten.

28 \newif\ifaskforoverwrite \askforoverwritetrue

6.1.2 Count registers

\blockLevel Optionally included blocks of code can be nested. The counter \blockLevel will be used to keep track of the level of nesting. Its initial value is zero.

29 \newcount\blockLevel \blockLevel\z@

\emptyLines The count register \emptyLines is used to count the number of consecutive empty input lines. Only the first will be copied to the output file.

30 \newcount\emptyLines \emptyLines \z@

\processedLines \commentsRemoved \commentsPassed \codeLinesPassed To be able to provide the user with some statistics about the stripping process four counters are allocated if the statistics have been included when this program was DocStripped. The number of lines processed is stored in the counter \processedLines. The number of lines containing comments that are not written on the output file is stored in the counter \commentsRemoved; the number of comments copied to the output file is stored in the counter \commentsPassed. The number of lines containing macro code that are copied to the output file is stored in the counter \codeLinesPassed.

31 ⟨∗stats⟩

32 \newcount\processedLines   \processedLines  \z@
33 \newcount\commentsRemoved  \commentsRemoved \z@
34 \newcount\commentsPassed   \commentsPassed  \z@
35 \newcount\codeLinesPassed  \codeLinesPassed \z@

\NumberOfFiles When more than one file is processed, the number of files is stored in the count register \NumberOfFiles when statistics are included.

36 \newcount\NumberOfFiles \NumberOfFiles\z@

\TotalprocessedLines \TotalcommentsRemoved \TotalcommentsPassed \TotalcodeLinesPassed When more than one file is processed and when statistics have been included we

provide the user also with information about the total amount of lines processed. For this purpose four more count registers are allocated here.

37 \newcount\TotalprocessedLines   \TotalprocessedLines  \z@
38 \newcount\TotalcommentsRemoved  \TotalcommentsRemoved \z@
39 \newcount\TotalcommentsPassed   \TotalcommentsPassed  \z@
40 \newcount\TotalcodeLinesPassed  \TotalcodeLinesPassed \z@

41 ⟨/stats⟩

6.1.3 I/O streams

\inFile For reading the file with documented TEX-code, an input stream \inFile is allocated.

42 \newread\inFile

\ttyin \ttyout Communication with the user goes through (nonexistent) stream 16.

43 \chardef\ttyin16
44 \chardef\ttyout16

\currbatchFile For batch file processing we need a number of input streams so that a batch file

can open further batch files using \batchinput.

We begin by allocating ten anonymous read streams.

45 \newread\@tempa

\lowestbatchFile We save the lowest number allocated in \lowestbatchFile, of course this code highly depends on the implementation of the plain allocation routines.

46 \let\lowestbatchFile\@tempa
47 \newread\@tempa \newread\@tempa \newread\@tempa \newread\@tempa
48 \newread\@tempa \newread\@tempa \newread\@tempa \newread\@tempa
49 \newread\@tempa

\highestbatchFile Same for the highest number.

50 \let\highestbatchFile\@tempa

Now we allocate a count register which is supposed to hold the the number of the batch file stream we are currently reading from. Initially this will contain the number for the first read stream allocated for this reason.

51 \newcount\currbatchFile
52 \currbatchFile=\highestbatchFile
53 \advance\currbatchFile by -9

\ifToplevel Execute the argument if current batch file is the outermost one. Otherwise suppress it.

54 \def\ifToplevel{\relax\ifnum\currbatchFile=\lowestbatchFile
55 \expandafter\iden \else \expandafter\@gobble\fi}

\batchinput When the file docstrip.tex is read because of an \input statement in a batch file we have to prevent an endless loop (well, limited by TEX’s stack). Therefore we save the original primitive \input and define a new macro with an argument delimited by ␣ (i.e. a space) that just gobbles the argument. Since the end-of-line character is converted by TEX to a space. This means that \input is not available as a command within batch files. In fact it would be dangerous to use it anyway, since it would mean mixing line for line reading with unlimited file reading.

\@@input We therefore keep a copy of the original under the name \@@input for internal use. If DocStrip runs under L^{A}TEX this command is already defined, so we make a quick test.

56 \ifx\undefined\@@input \let\@@input\input\fi

To allow the nesting of batch files the \batchinput command is provided it takes one argument, the name of the batch file to switch to.

57 \def\batchinput#1{%

We start a new group and locally redefine \batchFile to hold the new batch file name and advance \currbatchFile by one to get a new read stream. If the batch files are too deeply nested there may be no further stream available so we better check for this. The currently supported number is 10 which is generous enough even for large installation procedures.

58 \begingroup
59  \def\batchfile{#1}%
60  \advance\currbatchFile\@ne
61  \ifnum\currbatchFile>\highestbatchFile
62  \errhelp{Only up to ten levels of nesting for
63  \noexpand\batchinput commands are supported}%
64  \errmessage{Batchfiles too deeply nested}%
65  \fi

After this test we can simply call \processbatchFile which will open the new batch file and does a line for line reading until it is exhausted. Note that if the batch file is not available, or misspelled this routine will produce a warning and returns.

66  \processbatchFile

The values of \batchfile and \currbatchFile will be restored at this closing \endgroup, so that further processing continues in the calling batch file.

67 \endgroup

68 }

\skip@input And here is the promised redefinition of \input:

69 \def\skip@input#1 {}
70 \let\input\skip@input

6.1.4 Empty macros and macros that expand to a string

\guardStack Because blocks of code that will conditionally be included in the output can be nested, a stack is maintained to keep track of these blocks. The main reason for this is that we want to be able to check if the blocks are properly nested. The stack itself is stored in \guardStack.

71 \def\guardStack{}

\blockHead The macro \blockHead is used for storing and retrieving the boolean expression that starts a block.

72 \def\blockHead{}

{\yes \atop \y} When the user is asked a question that he has to answer with either ⟨yes⟩ or ⟨no⟩, his response has to be evaluated. For this reason the macros \yes and \y are defined.

73 \def\yes{yes}
74 \def\y{y}

\n We also define \n for use in DocStrip command files.

75 \def\n{n}

\Defaultbatchile When the DocStrip program has to process a batch file it can look for a batch file with a default name. This name is stored in \DefaultbatchFile.

76 \def\DefaultbatchFile{docstrip.cmd}

\perCent \DoubleperCent To be able to display percent-signs on the terminal, a % with category code 12 is stored in \perCent. The macro \DoubleperCent is used to write two percent-signs when writing the preamble of the output file.

77 {\catcode`\%=12
78 \gdef\perCent{%}
79 \gdef\DoubleperCent{%% }

80 }

In order to allow formfeeds in the input we define a one-character control sequence

^^L.
81 \def^^L{ }

The only result of using \Name is slowing down execution since its typical use (e.g. \Name\def{foo bar}...) has exactly the same number of tokens as its expansion. However I think that it’s easier to read. The meaning of \Name as a black box is: “construct a name from second parameter and then pass it to your first parameter as a parameter”.

\@stripstring is used to get tokens building name of a macro without leading backslash.

82 \def\Name#1#2{\expandafter#1\csname#2\endcsname}
83 \def\@stripstring{\expandafter\@gobble\string}

6.1.5 Miscellaneous variables

\sourceFileName The macro \sourceFileName is used to store the name of the current input file.

\batchfile The macro \batchfile is used to store the name of the batch file.

\inLine The macro \inLine is used to store the lines, read from the input file, before further processing.

\batchLine The macro \batchLine is used to store the lines, read from the batch file, before further processing.

\answer When some interaction with the user is needed the macro \answer is used to store his response.

\tmp Sometimes something has to be temporarily stored in a control sequence. For these purposes the control sequence \tmp is used.

6.2 Support macros

6.2.1 The stack mechanism

It is possible to have ‘nested guards’. This means that within a block of optionally included code a subgroup is only included when an additional option is specified. To keep track of the nesting of the guards the currently ‘open’ guard can be pushed on the stack \guardStack and later popped off the stack again. The macros that implement this stack mechanism are loosely based on code that is developed in the context of the L^{A}TEX3 project.

To be able to implement a stack mechanism we need a couple of support macros.

\eltStart \eltEnd The macros \eltStart and \eltEnd are used to delimit a stack element. They are both empty.

84 \def\eltStart{}
85 \def\eltEnd{}

\qStop The macro \qStop is a so-called ‘quark’, a macro that expands to itself^{4}.

86 \def\qStop{\qStop}

\pop The macro \pop⟨stack⟩⟨cs⟩ ‘pops’ the top element from the stack. It assigns the value of the top element to ⟨cs⟩ and removes it from ⟨stack⟩. When ⟨stack⟩ is empty a warning is issued and ⟨cs⟩ is assigned an empty value.

87 \def\pop#1#2{%
88 \ifx#1\empty
89  \Msg{Warning: Found end guard without matching begin}%
90  \let#2\empty
91 \else

4The concept of ‘quarks’ is developed for the L^{A}TEX3 project.

To be able to ‘peel’ off the first guard we use an extra macro \popX that receives both the expanded and the unexpanded stack in its arguments. The expanded stack is delimited with the quark \qStop.

92  \def\tmp{\expandafter\popX #1\qStop #1#2}%
93  \expandafter\tmp\fi}

\popX When the stack is expanded the elements are surrounded with \eltStart and \eltEnd. The first element of the stack is assigned to #4.

94 \def\popX\eltStart #1\eltEnd #2\qStop #3#4{\def#3{#2}\def#4{#1}}

\push Guards can be pushed on the stack using the macro \push⟨stack⟩⟨guard⟩. Again we need a secondary macro (\pushX) that has both the expanded and the unexpanded stack as arguments.

95 \def\push#1#2{\expandafter\pushX #1\qStop #1{\eltStart #2\eltEnd}}

\pushX The macro \pushX picks up the complete expansion of the stack as its first argument and places the guard in #3 on the ‘top’.

96 \def\pushX #1\qStop #2#3{\def #2{#3#1}}

6.2.2 Programming structures

\forlist When the program is used in interactive mode the user can supply a list of files that have to be processed. In order to process this list a for-loop is needed. This implementation of such a programming construct is based on the use of the \loop{⟨body⟩}\repeat macro that is defined in plain TEX. The syntax for this loop is:

\for⟨control sequence⟩ := ⟨list⟩ \do ⟨body⟩

\od

The ⟨list⟩ should be a comma separated list.

The first actions that have to be taken are to set the switch \ifForlist to ⟨true⟩ and to store the loop condition in the macro \ListCondition. This is done using an \edef to allow for a control sequence that contains a ⟨list⟩.

97 \def\forlist#1:=#2\do#3\od{%
98  \edef\ListCondition{#2}%
99  \Forlisttrue

Then we start the loop. We store the first element from the \ListCondition in the macro that was supplied as the first argument to \forlist. This element is then removed from the \ListCondition.

100  \loop
101  \edef#1{\expandafter\FirstElt\ListCondition,\empty.}%
102  \edef\ListCondition{\expandafter\OtherElts\ListCondition,\empty.}%

When the first element from the ⟨list⟩ is empty, we are done processing, so we switch \ifForlist to ⟨false⟩. When it is not empty we execute the third argument that should contain TEX commands to execute.

103  \ifx#1\empty \Forlistfalse \else#3\fi

Finally we test the switch \ifForlist to decide whether the loop has to be continued.

104  \ifForlist
105  \repeat}

\FirstElt The macro \FirstElt is used to get the first element from a comma-separated list.

106 \def\FirstElt#1,#2.{#1}

\OtherElts The macro \OtherElts is used to get all elements but the first element from a comma-separated list.

107 \def\OtherElts#1,#2.{#2}

\whileswitch When the program is used in interactive mode the user might want to process several files with different options or extensions. This goal could be reached by running the program several times, but it is more user-friendly to ask if he would like to process more files when we are done processing his last request. To accomplish this we need the implementation of a while-loop. Again plain TEX’s \loop{⟨body⟩}\repeat is used to implement this programming structure.

The syntax for this loop is:

\whileswitch⟨switch⟩ \fi ⟨list⟩ {⟨body⟩}

The first argument to this macro has to be a switch, defined using \newif; the second argument contains the statements to execute while the switch evaluates to ⟨true⟩.

108 \def\whileswitch#1\fi#2{#1\loop#2#1\repeat\fi}

6.2.3 Output streams allocator

For each of sixteen output streams available we have a macro named \s@0 through \s@15 saying if the stream is assigned to a file (1) or not (0). Initially all streams are not assigned.

We also declare 16 counters which will be needed by the conditional code inclusion algorithm.

109 \ifx\@tempcnta\undefined \newcount\@tempcnta \fi
110 \@tempcnta=0
111 \loop
112 \Name\chardef{s@\number\@tempcnta}=0
113 \csname newcount\expandafter\endcsname%
114 \csname off@\number\@tempcnta\endcsname
115 \advance\@tempcnta1
116 \ifnum\@tempcnta<16\repeat

We will use The TEXbook style list to search through streams.

117 \let\s@do\relax
118 \edef\@outputstreams{%

119 \s@do\Name\noexpand{s@0}\s@do\Name\noexpand{s@1}
120 \s@do\Name\noexpand{s@2}\s@do\Name\noexpand{s@3}
121 \s@do\Name\noexpand{s@4}\s@do\Name\noexpand{s@5}
122 \s@do\Name\noexpand{s@6}\s@do\Name\noexpand{s@7}
123 \s@do\Name\noexpand{s@8}\s@do\Name\noexpand{s@9}
124 \s@do\Name\noexpand{s@10}\s@do\Name\noexpand{s@11}
125 \s@do\Name\noexpand{s@12}\s@do\Name\noexpand{s@13}
126 \s@do\Name\noexpand{s@14}\s@do\Name\noexpand{s@15}
127 \noexpand\@nostreamerror

128 }

\@nostreamerror \@streamfound When \@outputstreams is executed \s@do is defined to do something on condition

of some test. If condition always fails macro \@nostreamerror on the end of the list causes an error. When condition succeeds \@streamfound is called, which gobbles rest of the list including the ending \@nostreamerror. It also gobbles \fi ending the condition, so the \fi is reinserted.

129 \def\@nostreamerror{\errmessage{No more output streams!}}
130 \def\@streamfound#1\@nostreamerror{\fi}

\@stripstr is auxiliary macro eating characters \s@ (backslash,s,@). It is defined in somewhat strange way since \s@ must have all category code 12 (other). This macro is used to extract stream numbers from stream names.

131 \bgroup\edef\x{\egroup
132 \def\noexpand\@stripstr\string\s@{}}

133 \x

\StreamOpen \StreamPut \StreamClose Here is stream opening operator. Its parameter should be a macro named the same as the external file being opened. E.g. to write to file foo.tex use

 \StreamOpen\foo, then \StreamPut\foo and \StreamClose\foo.
134 \def\StreamOpen#1{%
135 \chardef#1=16
136 \def\s@do##1{\ifnum##1=0
137  \chardef#1=\expandafter\@stripstr\string##1 %
138  \global\chardef##1=1 %
139  \immediate\openout#1=\WriteToDir\@stripstring#1 %
140  \@streamfound
141  \fi}
142 \@outputstreams

143 }

144 \def\StreamClose#1{
145 \immediate\closeout#1%
146 \def\s@do##1{\ifnum#1=\expandafter\@stripstr\string##1 %
147  \global\chardef##1=0 %
148  \@streamfound
149  \fi}
150 \@outputstreams
151 \chardef#1=16

152 }

153 \def\StreamPut{\immediate\write}

6.2.4 Input and Output

\maybeMsg \showprogress \keepsilent When this program is used it can optionally show its progress on the terminal. In that case it will write a special character to the terminal (and the transcript file) for each input line. This option is on by default when statistics are included in docstrip.tex. It is off when statistics are excluded. The commands \showprogress and \keepsilent can be used to choose otherwise.

154 \def\showprogress{\let\maybeMsg\message}
155 \def\keepsilent{\let\maybeMsg\@gobble}

156 ⟨∗stats⟩

157 \showprogress

158 ⟨/stats⟩

159 ⟨−stats⟩\keepsilent

\Msg For displaying messages on the terminal the macro \Msg is defined to write immediately to \ttyout.

160 \def\Msg{\immediate\write\ttyout}

\Ask The macro \Ask{⟨cs⟩}{⟨string⟩} is a slightly modified copy of the L^{A}TEX macro \typein. It is used to ask the user a question. The ⟨string⟩ will be displayed on his terminal and the response will be stored in the ⟨cs⟩. The trailing space left over from the carriage return is stripped off by the macro \strip. If the user just types a carriage return, the result will be an empty macro.

161 \def\iden#1{#1}
162 \def\strip#1#2 \@gobble{\def #1{#2}}
163 \def\@defpar{\par}
164 \def\Ask#1#2{%
165  \message{#2}\read\ttyin to #1\ifx#1\@defpar\def#1{}\else
166  \iden{\expandafter\strip
167  \expandafter#1#1\@gobble\@gobble} \@gobble\fi}

6.2.5 Miscellaneous

\@gobble A macro that has an argument and puts it in the bitbucket.

168 \def\@gobble#1{}

\Endinput When a doc file contains a \endinput on a line by itself this normally means that anything following in this file should be ignored. Therefore we need a macro containing \endinput as its replacement text to check this against \inLine (the current line from the current input file). Of course the backslash has to have the correct \catcode. One way of doing this is feeding \\ to the \string operation and afterwards removing one of the \ characters.

169 \edef\Endinput{\expandafter\@gobble\string\\endinput}

\makeOther During the process of reading a file with TEX code the category code of all special characters has to be changed to ⟨other⟩. The macro \makeOther serves this purpose.

170 \def\makeOther#1{\catcode`#1=12\relax}

\end For now we want the DocStrip program to be compatible with both plain TEX and L^{A}TEX. L^{A}TEX version 2.09 hides plain TEX’s \end command and calls it \@@end. We unhide it here.

171 \ifx\undefined\@@end\else\let\end\@@end\fi

\@addto A macro extending macro’s definition. The trick with \csname is necessary to get around \newtoks being outer in plain TEX and L^{A}TEX version 2.09.

172 \ifx\@temptokena\undefined \csname newtoks\endcsname\@temptokena\fi

173

174 \def\@addto#1#2{%
175 \@temptokena=\expandafter{#1}
176 \edef#1{\the\@temptokena#2}}

\@ifpresent This macro checks if its first argument is present on a list passed as the second argument. Depending on the result it executes either its third or fourth argument.

177 \def\@ifpresent#1#2#3#4{%
178 \def\tmp##1#1##2\qStop{\ifx!##2!}%
179 \expandafter\tmp#2#1\qStop #4\else #3\fi

180 }

\tospaces This macro converts its argument delimited with \secapsot to appropriate num-

ber of spaces. We need this for smart displaying messages on the screen.

\@spaces are used when we need many spaces in a row.

181 \def\tospaces#1{%
182 \ifx#1\secapsot\secapsot\fi\space\tospaces}
183 \def\secapsot\fi\space\tospaces{\fi}
184 \def\@spaces{\space\space\space\space\space}

\uptospace This macro extracts from its argument delimited with \qStop part up to first occurence of space.

185 \def\uptospace#1 #2\qStop{#1}

The names of these macros are as descriptive as their definitions.

186 \def\@firstoftwo#1#2{#1}
187 \def\@secondoftwo#1#2{#2}
188 \def\@firstofthree#1#2#3{#1}
189 \def\@secondofthree#1#2#3{#2}
190 \def\@thirdofthree#1#2#3{#3}

6.3 The evaluation of boolean expressions

For clarity we repeat here the syntax for the boolean expressions in somewhat changed but equivalent way:

⟨Expression⟩ ::= ⟨Secondary⟩ | ⟨Secondary⟩ {|, ,} ⟨Expression⟩
⟨Secondary⟩ ::= ⟨Primary⟩ | ⟨Primary⟩ & ⟨Secondary⟩
⟨Primary⟩  ::= ⟨Terminal⟩ | !⟨Primary⟩ | (⟨Expression⟩)

The | stands for disjunction, the & stands for conjunction and the ! stands for negation. The ⟨Terminal⟩ is any sequence of letters and evaluates to ⟨true⟩ iff it occurs in the list of options that have to be included.

Since we can generate multiple output files from one input, same guard expressions can be computed several times with different options. For that reason we first “compile” the expression to the form of one parameter macro \Expr expanding to nested \ifs that when given current list of options produces 1 or 0 as a result. The idea is to say \if1\Expr{⟨current set of options⟩}...\fi for all output files.

Here is a table recursively defining translations for right sides of the grammar. τ(X) denotes translation of X.

            τ(⟨Terminal⟩) = \t@<Terminal>,#1,<Terminal>,\qStop

τ(!⟨Primary⟩) = \if1 τ(⟨Primary⟩) 0\else1\fi
τ((⟨Expression⟩)) = τ(⟨Expression⟩)
τ(⟨Primary⟩&⟨Secondary⟩) = \if0 τ(⟨Primary⟩) 0\else τ(⟨Secondary⟩) \fi
τ(⟨Secondary⟩|⟨Expression⟩) = \if1 τ(⟨Secondary⟩) 1\else τ(⟨Expression⟩) \fi

\t@<Terminal> denotes macro with name constructed from t@ with appended tokens of terminal. E.g. for terminal foo the translation would be

    \t@foo,#1,foo,\qStop

This will end up in definition of \Expr, so #1 here will be substituted by current list of options when \Expr is called. Macro \t@foo would be defined as

    \def\t@foo#1,foo,#2\qStop{\ifx>#2>0\else1\fi}

When called as above this will expand to 1 if foo is present on current list of options and to 0 otherwise.

Macros below work in “almost expand-only” mode i.e. expression is analyzed only by expansion but we have to define some macros on the way (e.g. \Expr and

\t@foo).

The first parameter of each of these macros is “continuation” (in the sense similar to the language Scheme). Continuation is a function of at least one argument (parameter) being the value of previous steps of computation. For example macro \Primary constructs translation of ⟨Primary⟩ expression. When it decides that expression is completed it calls its continuation (its first argument) passing to it whole constructed translation. Continuation may expect more arguments if it wants to see what comes next on the input.

We will perform recursive descent parse, but definitions will be presented in bottom-up order.

\Terminal ⟨Terminal⟩s are recognized by macro \Terminal. The proper way of calling it is \Terminal{⟨current continuation⟩}{}. Parameters are: continuation, ⟨Terminal⟩ so far and next character from the input. Macro checks if #3 is one of terminalending characters and then takes appropriate actions. Since there are 7 ending chars and probably one \csname costs less than 7 nested \ifs we construct a name and check if it is defined.

We must expand \ifx completely before taking next actions so the following trickery happens:

191 \def\Terminal#1#2#3{%
192 \expandafter\ifx\csname eT@#3\endcsname\relax
193  \expandafter\@firstoftwo\else\expandafter\@secondoftwo\fi

If condition is true #3 belongs to current ⟨Terminal⟩ so we append it to ⟨Terminal⟩- so-far and call \Terminal again.

194 {\Terminal{#1}{#2#3}}%

When condition is false it’s time to end the ⟨Terminal⟩ so we call macro \TerminalX. Next character is reinserted to the input.

In both cases continuation is passed unchanged.

195 {\TerminalX{#1}{#2}#3}%

196 }

\eT@ Here we define macros marking characters that cannot appear inside terminal. The value is not important as long as it is different from \relax.

197 \Name\let{eT@>}=1
198 \Name\let{eT@&}=1 \Name\let{eT@!}=1
199 \Name\let{eT@|}=1 \Name\let{eT@,}=1
200 \Name\let{eT@(}=1 \Name\let{eT@)}=1

\TerminalX This macro should end scanning of ⟨Terminal⟩. Parameters are continuation and

gathered tokens of ⟨Terminal⟩.

Macro starts by issuing an error message if ⟨Terminal⟩ is empty.

201 \def\TerminalX#1#2{%
202 \ifx>#2> \errmessage{Error in expression: empty terminal}\fi

Then a macro is constructed for checking presence of ⟨Terminal⟩ in options list.

203 \Name\def{t@#2}##1,#2,##2\qStop{\ifx>##2>0\else1\fi}%

And then current continuation is called with translation of ⟨Terminal⟩ according to formula

          τ(⟨Terminal⟩) = \t@<Terminal>,#1,<Terminal>,\qStop
204 #1{\Name\noexpand{t@#2},##1,#2,\noexpand\qStop}%

205 }

\Primary Parameters are continuation and next character from the input.

According to the syntax ⟨Primari⟩es can have three forms. This makes us use even more dirty tricks than usual. Note the \space after a series of \ifxs. This series produces an one digit number of case to be executed. The number is given to \ifcase and \space stops TEX scanning for a ⟨number⟩. Use of \ifcase gives possibility to jump out of conditions with just one \expandafter.

206 \def\Primary#1#2{%
207 \ifcase \ifx!#20\else\ifx(#21\else2\fi\fi\space
208  \expandafter\@firstofthree\or
209  \expandafter\@secondofthree\or
210  \expandafter\@thirdofthree\fi

First case is for ! i.e. negated ⟨Primary⟩. In this case we call \Primary recursively but we create new continuation: macro \NPrimary that will negate result passed by \Primary and pass it to current continuation (#1).

211 {\Primary{\NPrimary{#1}}}%

When next character is ( we call \Expression giving it as continuation macro \PExpression which will just pass the result up but ensuring first that a ) comes next.

212 {\Expression{\PExpression{#1}}}%

Otherwise we start a ⟨Terminal⟩. #2 is not passed as ⟨Terminal⟩-so-far but reinserted to input since we didn’t check if it can appear in a ⟨Terminal⟩.

213 {\Terminal{#1}{}#2}%

214 }

\NPrimary Parameters are continuation and previously computed ⟨Primary⟩. This macro negates result of previous computations according to the rule

τ(!⟨Primary⟩) = \if1 τ(⟨Primary⟩) 0\else1\fi

215 \def\NPrimary#1#2{%
216 #1{\noexpand\if1#20\noexpand\else1\noexpand\fi}%

217 }

\PExpression Parameters: continuation, ⟨Expression⟩, next character from input. We are checking if character is ) and then pass unchanged result to our continuation.

218 \def\PExpression#1#2#3{%
219 \ifx)#3\else
220  \errmessage{Error in expression: expected right parenthesis}\fi
221 #1{#2}}

\Secondary Each ⟨Secondary⟩ expression starts with ⟨Primary⟩. Next checks will be performed

by \SecondaryX.
222 \def\Secondary#1{%
223 \Primary{\SecondaryX{#1}}}

\SecondaryX Parameters: continuation, translation of ⟨Primary⟩, next character. We start by checking if next character is &.

224 \bgroup\catcode`\&=12
225 \gdef\SecondaryX#1#2#3{%
226 \ifx&#3%
227  \expandafter\@firstoftwo\else
228  \expandafter\@secondoftwo\fi

If it is we should parse next ⟨Secondary⟩ and then combine it with results so far. Note that \SecondaryXX will have 3 parameters.

229 {\Secondary{\SecondaryXX{#1}{#2}}}%

Otherwise ⟨Secondary⟩ turned out to be just ⟨Primary⟩. We call continuation passing to it translation of that ⟨Primary⟩ not forgetting to reinsert #3 to the input as it does not belong here.

230 {#1{#2}#3}%

231 }

232 \egroup

\SecondaryXX Parameters: continuation, translation of ⟨Primary⟩, translation of ⟨Secondary⟩. We construct translation of whole construction according to the rule:

τ(⟨Primary⟩&⟨Secondary⟩) = \if0 τ(⟨Primary⟩) 0\else τ(⟨Secondary⟩) \fi

and pass it to our continuation.

233 \def\SecondaryXX#1#2#3{%
234 #1{\noexpand\if0#20\noexpand\else#3\noexpand\fi}}

\Expression Every ⟨Expression⟩ starts with ⟨Secondary⟩. We construct new continuation and

pass it to \Secondary.
235 \def\Expression#1{%
236 \Secondary{\ExpressionX{#1}}}

\ExpressionX Parameters: continuation, translation of ⟨Secondary⟩, next character. We perform check if character is | or ,.

237 \def\ExpressionX#1#2#3{%
238 \if0\ifx|#31\else\ifx,#31\fi\fi0
239  \expandafter\@firstoftwo\else
240  \expandafter\@secondoftwo\fi

If it is not ⟨Expression⟩ is just a ⟨Secondary⟩. We pass its translation to continuation and reinsert #3.

241 {#1{#2}#3}%

If we are dealing with complex ⟨Expression⟩ we should parse another \Expression now.

242 {\Expression{\ExpressionXX{#1}{#2}}}%

243 }

\ExpressionXX Parameters: continuation, translation of ⟨Secondary⟩, translation of ⟨Expression⟩. We finish up translating of ⟨Expression⟩ according to the formula:

τ(⟨Secondary⟩|⟨Expression⟩) = \if1 τ(⟨Secondary⟩) 1\else τ(⟨Expression⟩) \fi

244 \def\ExpressionXX#1#2#3{%
245 #1{\noexpand\if1#21\noexpand\else#3\noexpand\fi}}

\StopParse Here is initial continuation for whole parse process. It will be used by \Evaluate. Note that we assume that expression has > on its end. This macro eventually defines \Expr. Parameters: translation of whole ⟨Expression⟩ and next character from input.

246 \def\StopParse#1#2{%
247 \ifx>#2 \else\errmessage{Error in expression: spurious #2}\fi
248 \edef\Expr##1{#1}}

\Evaluate This macro is used to start parsing. We call \Expression with continuation defined above. On end of expression we append a >.

249 \def\Evaluate#1{%
250 \Expression\StopParse#1>}

6.4 Processing the input lines

\normalLine The macro \normalLine writes its argument (which has to be delimited with \endLine) on all active output files i.e. those with off-counters equal to zero. If statistics are included, the counter \codeLinesPassed is incremented by 1.

251 \def\normalLine#1\endLine{%

252 ⟨∗stats⟩

253 \advance\codeLinesPassed\@ne

254 ⟨/stats⟩

255 \maybeMsg{.}%
256 \def\inLine{#1}%
257 \let\do\putline@do
258 \activefiles

259 }

\putline@do This is a value for \do when copying line to output files.

260 \def\putline@do#1#2#3{%
261 \StreamPut#1{\inLine}}

\removeComment The macro \removeComment throws its argument (which has to be delimited with \endLine) away. When statistics are included in the program the removed comment is counted.

262 %

263 \def\removeComment#1\endLine{%

264 ⟨∗stats⟩

265  \advance\commentsRemoved\@ne

266 ⟨/stats⟩

267  \maybeMsg{\perCent}}

\putMetaComment If a line starts with two consecutive percent signs, it is considered to be a Meta-Comment. Such a comment line is passed on to the output file unmodified.

268 \def\putMetaComment#1\endLine{%

If statistics are included the line is counted.

269 ⟨∗stats⟩

270 \advance\commentsPassed\@ne

271 ⟨/stats⟩

The macro \putMetaComment has one argument, delimited with \endLine. But because of earlier processing the argument is discarded and \inLine, which contains the complete line as it was read from \inFile, is written on all output files.

272 \let\do\putline@do
273 \outputfiles

274 }

\processLine Each line that is read from the input stream has to be processed to see if it has to be written on the output stream. This task is performed by calling the macro \processLine. In order to do that, it needs to check whether the line starts with a ‘%’. Therefore the macro is globally defined within a group. Within this group the category code of ‘%’ is changed to 12 (other). Because a comment character is needed, the category code of ‘*’ is changed to 14 (comment character).

The macro increments counter \processedLines by 1 if statistics are included. We cannot include this line with %<*stats> since the category of %, so the whole definition is repeated embedded in guards.

The next token from the input stream is passed in #1. If it is a ‘%’ further processing has to be done by \processLineX; otherwise this is normal (not commented out) line.

In either case the character read is reinserted to the input as it may have to be written out.

275 ⟨∗!stats⟩

276 \begingroup
277 \catcode`\%=12 \catcode`\*=14
278 \gdef\processLine#1{*
279 \ifx%#1
280  \expandafter\processLineX
281 \else
282  \expandafter\normalLine
283 \fi
284 #1}
285 \endgroup

286 ⟨/!stats⟩ 287 ⟨∗stats⟩

288 \begingroup
289 \catcode`\%=12 \catcode`\*=14
290 \gdef\processLine#1{*
291 \advance\processedLines\@ne
292 \ifx%#1
293  \expandafter\processLineX
294 \else
295  \expandafter\normalLine
296 \fi
297 #1}
298 \endgroup

299 ⟨/stats⟩

\processLineX This macro is also defined within a group, because it also has to check if the next

token in the input stream is a ‘%’ character.

If the second token in the current line happens to be a ‘%’, a ⟨MetaComment⟩ has been found. This has to be copied in its entirety to the output. Another possible second character is ‘<’, which introduces a guard expression. The processing of such an expression is started by calling \checkOption.

When the token was neither a ‘%’ nor a ‘<’, the line contains a normal comment that has to be removed.

We express conditions in such a way that all actions appear on first nesting level of \ifs. In such conditions just one expandafter pushes us outside whole construction. A thing to watch here is \relax. It stops search for numeric constant. If it wasn’t here TEX would expand the first case of \ifcase before knowing the value.

300 \begingroup
301 \catcode`\%=12 \catcode`\*=14
302 \gdef\processLineX%#1{*
303 \ifcase\ifx%#10\else
304  \ifx<#11\else 2\fi\fi\relax
305  \expandafter\putMetaComment\or
306  \expandafter\checkOption\or
307  \expandafter\removeComment\fi
308 #1}
309 \endgroup

6.5 The handling of options

\checkOption When the macros that process a line have found that the line starts with ‘%<’, a guard line has been encountered. The first character of a guard can be an asterisk (*), a slash (/) a plus (+), a minus (-) or any other character that can be found in an option name. This means that we have to peek at the next token and decide what kind of guard we have.

We reinsert #1 as it may be needed by \doOption.

310 \def\checkOption<#1{%
311 \ifcase
312  \ifx*#10\else \ifx/#11\else
313  \ifx+#12\else \ifx-#13\else 4\fi\fi\fi\fi\relax
314 \expandafter\starOption\or
315 \expandafter\slashOption\or
316 \expandafter\plusOption\or
317 \expandafter\minusOption\or
318 \expandafter\doOption\fi
319 #1}

\doOption When no guard modifier is found by \checkOptions, the macro \doOption is called. It evaluates a boolean expression. The result of this evaluation is stored in \Expr. The guard only affects the current line, so \do is defined in such a way that depending on the result of the test \if1\Expr{⟨options⟩}, the current line is either copied to the output stream or removed. Then the test is computed for all active output files.

320 \def\doOption#1>#2\endLine{%
321 \maybeMsg{<#1 . >}%
322 \Evaluate{#1}%
323 \def\do##1##2##3{%
324  \if1\Expr{##2}\StreamPut##1{#2}\fi

325 }%

326 \activefiles

327 }

\plusOption When a ‘+’ is found as a guard modifier, \plusOption is called. This macro is very similar to \doOption, the only difference being that displayed message now contains ‘+’.

328 \def\plusOption+#1>#2\endLine{%
329 \maybeMsg{<+#1 . >}%
330 \Evaluate{#1}%
331 \def\do##1##2##3{%
332  \if1\Expr{##2}\StreamPut##1{#2}\fi

333 }%

334 \activefiles

335 }

\minusOption When a ‘-’ is found as a guard modifier, \minusOption is called. This macro is very similar to \plusOption, the difference is that condition is negated.

336 \def\minusOption-#1>#2\endLine{%
337 \maybeMsg{<-#1 . >}%
338 \Evaluate{#1}%
339 \def\do##1##2##3{%
340  \if1\Expr{##2}\else \StreamPut##1{#2}\fi

341 }%

342 \activefiles

343 }

\starOption When a ‘*’ is found as a guard modifier, \starOption is called. In this case a block of code will be included in the output on the condition that the guard expression evaluates to ⟨true⟩.

The current line is gobbled as #2, because it only contains the guard and possibly a comment.

344 \def\starOption*#1>#2\endLine{%

First we optionally write a message to the terminal to indicate that a new option starts here.

345 \maybeMsg{<*#1}%

Then we push the current contents of \blockHead on the stack of blocks, \guardStack and increment the counter \blockLevel to indicate that we are now one level of nesting deeper.

346 \expandafter\push\expandafter\guardStack\expandafter{\blockHead}%
347 \advance\blockLevel\@ne

The guard for this block of code is now stored in \blockHead.

348 \def\blockHead{#1}%

Now we evaluate guard expression for all output files updating off-counters. Then we create new list of active output files. Only files that were active in the outer block can remain active now.

349 \Evaluate{#1}%
350 \let\do\checkguard@do
351 \outputfiles
352 \let\do\findactive@do
353 \edef\activefiles{\activefiles}

354 }

\checkguard@do This form of \do updates off-counts according to the value of guard expression.

355 \def\checkguard@do#1#2#3{%

If this block of code occurs inside another block of code that is not included in the output, we increment the off counter. In that case the guard expression will not be evaluated, because a block inside another block that is excluded from the output will also be excluded, regardless of the evaluation of its guard.

356 \ifnum#3>0
357  \advance#3\@ne

When the off count has value 0, we have to evaluate the guard expression. If the result is ⟨false⟩ we increase the off-counter.

358 \else
359  \if1\Expr{#2}\else
360  \advance#3\@ne\fi
361 \fi}

\findactive@do This form of \do picks elements of output files list which have off-counters equal to zero.

362 \def\findactive@do#1#2#3{%
363 \ifnum#3=0
364  \noexpand\do#1{#2}#3\fi}

\slashOption The macro \slashOption is the counter part to \starOption. It indicates the end of a block of conditionally included code. We store the argument in the temporary control sequence \tmp.

365 \def\slashOption/#1>#2\endLine{%
366 \def\tmp{#1}%

When the counter \blockLevel has a value less than 1, this ‘end-of-block’ line has no corresponding ‘start-of-block’. Therefore we signal an error and ignore this end of block.

367 \ifnum\blockLevel<\@ne
368  \errmessage{Spurious end block </\tmp> ignored}%

Next we compare the contents of \tmp with the contents of \blockHead. The latter macro contains the last guard for a block of code that was encountered. If the contents match, we pop the previous guard from the stack.

369 \else
370  \ifx\tmp\blockHead
371  \pop\guardStack\blockHead

When the contents of the two macros don’t match something is amiss. We signal

Is this the desired be- this to the user, but accept the ‘end-of-block’. haviour??

372  \else
373  \errmessage{Found </\tmp> instead of </\blockHead>}%
374  \fi

When the end of a block of optionally included code is encountered we optionally signal this on the terminal and decrement the counter \blockLevel.

375  \maybeMsg{>}%
376  \advance\blockLevel\m@ne

The last thing that has to be done is to decrement off-counters and make new list of active files. Now whole list of output files has to be searched since some inactive files could have been reactivated.

377 \let\do\closeguard@do
378 \outputfiles
379 \let\do\findactive@do
380 \edef\activefiles{\outputfiles}
381 \fi

382 }

\closeguard@do This macro decrements non-zero off-counters.

383 \def\closeguard@do#1#2#3{%
384 \ifnum#3>0
385  \advance#3\m@ne
386 \fi}

6.6 Batchfile commands

\generate This macro executes its argument in a group. \inputfiles is a list of files to be read, \filestogenerate list of names of output files (needed for the message below). \files contained in #1 define \inputfiles in such a way that all that has to be done when the parameter is executed is to call this macro.

387 \def\generate#1{\begingroup
388 \let\inputfiles\empty \let\filestogenerate\empty
389 \let\file\@file

390 #1

391 \Msg{^^JGenerating file(s) \filestogenerate}%
392 \inputfiles
393 \endgroup}

\file The first argument is the file to produce, the second argument contains the list of

input files. Each entry should have the format \from{⟨filename.ext⟩}{⟨options⟩}.

The switch \ifGenerate is initially set to ⟨true⟩.

394 \def\file#1#2{\errmessage{Command `\string\file' only allowed in
395  argument to `\string\generate'}}
396 \def\@file#1{%
397 \Generatetrue

Next we decide if we have to be careful about overwriting existing files. If the user specified \askforoverwritetrue we will ask him if he wants to overwrite an existing file. Otherwise we simply go ahead.

398 \ifaskforoverwrite

We try to open a file with the name of the output file for reading. If this succeeds the file exists and we ask the user if he wants to overwrite the file.

399  \immediate\openin\inFile\WriteToDir#1\relax
400  \ifeof\inFile\else
401  \Ask\answer{File \WriteToDir#1 already exists
402  \ifx\empty\WriteToDir somewhere \fi
403  on the system.^^J%
404  Overwrite it%
405  \ifx\empty\WriteToDir\space if necessary\fi
406  ? [y/n]}%

We set the switch \ifGenerate according to his answer. We allow for both “y” and “yes”.

407  \ifx\y  \answer \else
408  \ifx\yes\answer \else
409  \Generatefalse\fi\fi\fi

Don’t forget to close the file just opened as we want to write to it.

410  \immediate\closein\inFile
411 \fi

If file is to be generated we pass control to macro \@fileX. Note that file name is turned into control sequence and \else branch is skipped before calling \@fileX.

412 \ifGenerate
413 \@addto\filestogenerate{\WriteToDir#1 }%
414 \Name\@fileX{#1\expandafter}%
415 \else

In case we were not allowed to overwrite an existing file we inform the user that we are not generating his file and we gobble \from specifications.

416  \Msg{Not generating file #1^^J}%
417 \expandafter\@gobble
418 \fi

419 }

\WriteToDir The macro \WriteToDir is either empty or holds the prefix necessary to read a file from the current directory. Under UNIX this is ./ but a lot of other systems adopted this concept.

420 \ifx\@currdir\@undefined
421 \def\WriteToDir{}
422 \else
423 \let\WriteToDir\@currdir
424 \fi

\@fileX We put name of current output file in \curout and initialize \curinfiles (the list of source files for this output file) to empty—these will be needed by \from. Then we start defining preamble for the current file.

425 \def\@fileX#1#2{%
426 \chardef#1=16
427 \def\curout{#1}\let\curinfiles\empty
428 \edef\curpre{\Name\noexpand{pre@\@stripstring#1}}
429 \edef\curinnames{\Name\noexpand{in@\@stripstring#1}}
430 \expandafter\def\curpre{\DoubleperCent ^^J%
431  \DoubleperCent The original source files were:^^J%
432  \DoubleperCent ^^J}%
433 \expandafter\def\curinnames{}

Next we execute second parameter. \froms will add reference lines to the preamble.

434 \let\from\@from \let\needed\@needed
435 #2%
436 \let\from\err@from \let\needed\err@needed

We check order of input files.

437 \checkorder

Each \from clause defines \curin to be its first parameter. So now \curin holds name of last input file for current output file. This means that current output file should be closed after processing \curin. We add #1 to proper ‘close list’.

438 \Name\@addto{e@\curin}{\noexpand\closeoutput{#1}}%

We append name of currently active preamble to preamble for current file and mark current postamble as postamble for current file.

439 \expandafter\@addto\curpre{\expandafter\noexpand\currentpreamble}
440 \Name\let{post@\@stripstring#1\expandafter}\currentpostamble

441 }

\checkorder This macro checks if the order of files in \curinfiles agrees with that of \inputfiles. The coding is somewhat clumsy.

442 \def\checkorder{%
443 \expandafter\expandafter\expandafter
444 \checkorderX\expandafter\curinfiles
445 \expandafter\qStop\inputfiles\qStop

446 }

447 \def\checkorderX(#1)#2\qStop#3\qStop{%

448 \def\tmp##1\readsource(#1)##2\qStop{%
449  \ifx!##2! \order@error
450  \else\ifx!#2!\else
451  \checkorderXX##2%
452  \fi\fi}%
453 \def\checkorderXX##1\readsource(#1)\fi\fi{\fi\fi
454  \checkorderX#2\qStop##1\qStop}%
455 \tmp#3\readsource(#1)\qStop

456 }

457 \def\order@error#1\fi\fi{\fi
458 \errmessage{DOCSTRIP error: Incompatible order of input
459  files specified for file
460  `\iden{\expandafter\uptospace\curin} \qStop'.^^J
461  Read DOCSTRIP documentation for explanation.^^J
462  This is a serious problem, I'm exiting}\end

463 }

\needed \@needed This macro uniquizes name of an input file passed as a parameter and marks it as needed to be input. It is used internally by \from, but can also be issued in argument to \file to influence the order in which files are read.

464 \def\needed#1{\errmessage{\string\needed\space can only be used in
465  argument to \string\file}}
466 \let\err@needed\needed
467 \def\@needed#1{%
468 \@ifpresent{(#1)}\curinfiles

If #1 is present on list of input files for current output file we add a space on end of its name and try again. The idea is to construct a name that will look different for TEX but will lead to the same file when seen by operating system.

469  {\@needed{#1 }}%

When it is not we check if #1 is present in the list of files to be processed. If not we add it and initialize list of output files for that input and list of output files that should be closed when this file closes. We also add constructed name to \curinfiles and define \curin to be able to access constructed name from

 \@from.
470  {\@ifpresent{\readsource(#1)}\inputfiles
471  {}{\@addto\inputfiles{\noexpand\readsource(#1)}%
472  \Name\let{o@#1}\empty\Name\let{e@#1}\empty}%
473  \@addto\curinfiles{(#1)}%
474  \def\curin{#1}}%

475 }

\from \from starts by adding a line to preamble for output file.

476 \def\from#1#2{\errmessage{Command `\string\from' only allowed in
477  argument to `\string\file'}}
478 \let\err@from\from
479 \def\@from#1#2{%
480 \expandafter\@addto\curpre{\DoubleperCent #1 \if>#2>\else
481  \space (with options: `#2')\fi^^J}%

482 \expandafter\@addto\curinnames{ #1}%

Then we mark the file as needed input file.

483 \needed{#1}%

And add \do⟨current output⟩{⟨options⟩} to the list of output files for current input file.

484 \Name\@addto{o@\curin}{\noexpand\do\curout{#2}}

485 }

\readsource This macro is called for each input file that is to be processed.

First we have to establish local environment for current input file by assigning to \outputfiles list of output files for this (#1) input file.

486 \def\readsource(#1){%
487 \expandafter\let\expandafter\outputfiles\csname o@#1\endcsname

A message is displayed on the terminal telling the user what we are about to do. For each output file we display one line saying with what options it is generated. First line contains also name of input file.

488 \Msg{} \def\@msg{Processing file \uptospace#1 \qStop}
489 \def\change@msg{%
490  \edef\@msg{\@spaces\@spaces\@spaces\space
491  \expandafter\tospaces\uptospace#1 \qStop\secapsot}
492  \let\change@msg\relax}
493 \def\do##1##2{%
494  \Msg{\@msg
495  \ifx>##2>\else\space(##2)\fi
496  \space -> \@stripstring##1}
497  \change@msg}%
498 \outputfiles

Then we try to open the input file. If this doesn’t succeed, we tell the user so and nothing else happens.

499 \immediate\openin\inFile\uptospace#1 \qStop\relax
500 \ifeof\inFile
501  \errmessage{Cannot find file \uptospace#1 \qStop}%
502 \else

If statistics are included we nullify line counters

503 ⟨∗stats⟩

504  \processedLines\z@
505  \commentsRemoved\z@
506  \commentsPassed\z@
507  \codeLinesPassed\z@

508 ⟨/stats⟩

When the input file was successfully opened, we ensure that all needed output files are open. Then we redefine \outputfiles again to construct off-counters names. Now \dos will have 3 parameters! All output files become active.

509  \let\do\ensureopen@do
510  \outputfiles

511  \let\do\makeoutlist@do
512  \edef\outputfiles{\outputfiles}%
513  \let\activefiles\outputfiles

Now we change the category code of a lot of characters to ⟨other⟩ and make sure that no extra spaces appear in the lines read by setting the \endlinechar to −1.

514  \makeOther\ \makeOther\\\makeOther\$%
515  \makeOther\#\makeOther\^\makeOther\^^K%
516  \makeOther\_\makeOther\^^A\makeOther\%%
517  \makeOther\~\makeOther\{\makeOther\}\makeOther\&%
518  \endlinechar-1\relax

Then we start a loop to process the lines in the file one by one.

519  \loop
520  \read\inFile to\inLine

The first thing we check is whether the current line contains an \endinput. To allow also real \endinput commands in the source file, \endinput is only recognized when it occurs directly at the beginning of a line.

521  \ifx\inLine\Endinput

In this case we output a message to inform the programmer (in case this was a mistake) and end the loop immediately by setting Continue to ⟨false⟩. Note that \endinput is not placed into the output file. This is important in cases where the output file is generated from several doc files.

522  \Msg{File #1 ended by \string\endinput.}%
523  \Continuefalse
524  \else

When the end of the file is found we have to interrupt the loop.

525  \ifeof\inFile
526  \Continuefalse

If the file did not end we check if the input line is empty. If it is, the counter \emptyLines is incremented.

527  \else
528  \Continuetrue
529  \ifx\inLine\empty
530  \advance\emptyLines\@ne
531  \else
532  \emptyLines\z@
533  \fi

When the number of empty lines seen so far exceeds 1, we skip them. If it doesn’t, the expansion of \inLine is fed to \processLine with \endLine appended to indicate the end of the line.

534  \ifnum \emptyLines<2
535  \expandafter\processLine\inLine\endLine
536  \else
537  \maybeMsg{/}%
538  \fi
539  \fi

540  \fi

When the processing of the line is finished, we check if there is more to do, in which case we repeat the loop.

541  \ifContinue
542  \repeat

The input file is closed.

543  \immediate\closein\inFile

We close output files for which this was the last input file.

544 \csname e@#1\endcsname

If the user was interested in statistics, we inform him of the number of lines processed, the number of comments that were either removed or passed and the number of codelines that were written to the output file. Also the totals are updated.

545 ⟨∗stats⟩

546  \Msg{Lines \space processed: \the\processedLines^^J%
547  Comments removed: \the\commentsRemoved^^J%
548  Comments \space passed: \the\commentsPassed^^J%
549  Codelines passed: \the\codeLinesPassed^^J}%
550  \global\advance\TotalprocessedLines  by \processedLines
551  \global\advance\TotalcommentsRemoved by \commentsRemoved
552  \global\advance\TotalcommentsPassed  by \commentsPassed
553  \global\advance\TotalcodeLinesPassed by \codeLinesPassed
554  \global\advance\NumberOfFiles by \@ne

555 ⟨/stats⟩

556  \fi}

\ensureopen@do This macro opens output streams and zeroes off-counters associated with them.

557 \def\ensureopen@do#1#2{%
558 \ifnum#1=16
559  \StreamOpen#1%
560  \WritePreamble#1%
561  \fi
562 \csname off@\number#1\endcsname=0

563 }

\makeoutlist@do This macro performs simple task of constructing names for off-counters.

564 \def\makeoutlist@do#1#2{%
565 \noexpand\do#1{#2}\csname off@\number#1\endcsname}

\closeoutput This macro closes output stream when it is no longer needed.

566 \def\closeoutput#1{%
567 \WritePostamble#1%
568 \StreamClose#1%

569 }

6.6.1 Preamble and postamble

\declarepreamble When a batch file is used the user can specify a preamble of his own that will be written to each file that is created. This can be useful to include an extra copyright notice in the stripped version of a file. Also a warning that both versions of a file should always be distributed together could be written to a stripped file by including it in such a preamble.

Every line that is written to \outFile that belongs to the preamble is preceded by two percent characters. This will prevent DocStrip from stripping these lines off the file.

The preamble should be started with the macro \declarepreamble; it is ended by \endpreamble. All processing is done within a group in order to be able to locally redefine some macros.

570 \def\declarepreamble#1{\begingroup

If the preamble contains an empty line this will end up in the macro \batchLine as \par. To be able to recognize a \par we define \tmp to contain \par. The macro \Endpreamble is defined to contain \endpreamble, which ends the processing of preamble lines. We also make \(in/out)FileName unexpandable.

571  \def\tmp{\par}%
572  \def\Endpreamble{\endpreamble}%
573  \let\inFileName\relax \let\outFileName\relax

The preamble lines will be read inside a \loop. When a line is found that contains \endpreamble the processing has to stop. This is controlled by the switch \ifContinue. We initialize it to ⟨true⟩.

574  \Continuetrue

The first argument to \declarepreamble will be defined to contain all preamble lines supplied by the user.

575  \global\let#1\empty

Each line in the preamble has to be added to #1, therefore we start a \loop to \read the lines. The control sequence \batchLine is used to store each line.

576  \loop
577  \read\currbatchFile to \batchLine

We check to see if the end of the preamble has been found. In that case the \loop has to come to an end, so we set the switch \ifContinue to ⟨false⟩.

578  \ifx\batchLine\Endpreamble \Continuefalse
579  \else

If an extra line has been read we add it to #1. If the \batchLine was empty, we write just the two percent characters; otherwise the complete line is written, preceded by the two percent characters.

580  {\globaldefs1
581  \@addto#1{%
582  \ifx#1\empty\else^^J\fi
583  \DoubleperCent
584  \ifx\batchLine\tmp\else\batchLine\fi}}%
585  \fi

The setting of the switch \ifContinue will decide if the \loop continues. If we end the \loop, we also have to close the group.

586  \ifContinue\repeat
587  \endgroup}

\declarepostamble Just as a preamble can be specified in a batch file, the same can be done for a

postamble.

The definition of \declarepostamble is very much like the definition above of \declarepreamble; the processing takes place inside a group and the switch \ifContinue is used to control the \loop.

588 \def\declarepostamble#1{\begingroup
589  \Continuetrue

The end of the postamble is indicated by \endpostamble and the postamble can contain empty lines so we have to be able to check if \batchLine contains a \par.

590  \def\Endpostamble{\endpostamble}%
591  \def\tmp{\par}%
592  \let\inFileName\relax \let\outFileName\relax

We initially define #1 as an empty macro. Commands will be added to it as we go along.

593  \global\let#1\empty

We start a loop to read lines from the batch file until \endpostamble is encountered.

594  \loop
595  \read\currbatchFile to \batchLine
596  \ifx\batchLine\Endpostamble \Continuefalse

The line just read will be written to output file prepended with two percent characters.

597  \else
598  {\globaldefs1
599  \@addto#1{%
600  \ifx#1\empty\else^^J\fi
601  \DoubleperCent
602  \ifx\batchLine\tmp\else\batchLine\fi}}%
603  \fi

The setting of the switch \ifContinue decides if we have to stop the \loop.

604  \ifContinue\repeat
605  \endgroup}

\usepreamble \usepostamble Macros for selecting [pre/post]amble to be used.

606 \def\usepreamble#1{\def\currentpreamble{#1}}
607 \def\usepostamble#1{\def\currentpostamble{#1}}

\preamble \postamble For backward compatibility we provide these macros defining default preamble and postamble.

608 \def\preamble{\usepreamble\defaultpreamble

609 \declarepreamble\defaultpreamble}
610 \def\postamble{\usepostamble\defaultpostamble
611 \declarepostamble\defaultpostamble}

\defaultpreamble \defaultpostamble Default values to use if nothing different is provided.

612 \def\defaultpreamble{%
613  \DoubleperCent ^^J%
614  \DoubleperCent IMPORTANT NOTICE:^^J%
615  \DoubleperCent ^^J%
616  \DoubleperCent For the copyright see the source file.^^J%
617  \DoubleperCent ^^J%
618  \DoubleperCent You are *not* allowed to modify this file.^^J%
619  \DoubleperCent ^^J%
620  \DoubleperCent You are *not* allowed to distribute this file.^^J%
621  \DoubleperCent For distribution of the original source see^^J%
622  \DoubleperCent the terms for copying and modification
623  in the file \inFileName.^^J%
624  \DoubleperCent}
625 \def\defaultpostamble{\string\endinput}
626 \usepreamble\defaultpreamble
627 \usepostamble\defaultpostamble

\WritePreamble First a couple of lines, stating what file it is and how it was created. If \AddGenerationDate is given in the batch file then the creation date and version of docstrip are also added.

628 \def\WritePreamble#1{%
629  \edef\outFileName{\@stripstring#1}%
630  \edef\inFileName{\csname in@\outFileName\endcsname}%
631  \StreamPut#1{%
632  \DoubleperCent ^^J%
633  \DoubleperCent This is file `\outFileName',^^J%
634  \DoubleperCent  generated with the docstrip utility.%

635 }%

Then the reference lines that tell from what source file(s) the stripped file was created and user supplied preamble.

636  \StreamPut#1{\csname pre@\@stripstring#1\endcsname}}

\AddGenerationDate Older versions of DocStrip added the date that any file was generated and the version number of DocStrip. This confused some people as they mistook this for the version/date of the file that was being written. So now this information is not normally written, but a batch file may call this to get an old style header.

637 \def\AddGenerationDate{%
638 \def\WritePreamble##1{%
639  \edef\outFileName{\@stripstring##1}%
640  \edef\inFileName{\csname in@\outFileName\endcsname}%
641  \StreamPut##1{%
642  \DoubleperCent ^^J%
643  \DoubleperCent This is file `\outFileName', generated %

644  on <\the\year/\the\month/\the\day> ^^J%
645  \DoubleperCent with the docstrip utility (\fileversion).%

646 }%

647  \StreamPut##1{\csname pre@\@stripstring##1\endcsname}}}

\WritePostamble Postamble attributed to #1 is written out. The last line written identifies the file again.

648 \def\WritePostamble#1{%
649  \edef\outFileName{\@stripstring#1}%
650  \edef\inFileName{\csname in@\outFileName\endcsname}%
651  \StreamPut#1{\csname post@\@stripstring#1\endcsname}%
652  \StreamPut#1{\DoubleperCent ^^J%
653  \DoubleperCent End of file `\outFileName'.}}

6.6.2 Compatibility with older versions

\generateFile Main macro of previous versions of DocStrip.

654 \def\generateFile#1#2#3{{
655 \ifx t#2\askforoverwritetrue
656 \else\askforoverwritefalse\fi
657 \generate{\file{#1}{#3}}%

658 }}

To support command files that were written for the first version of DocStrip the commands \include and \processFile are defined here. The use of this interface is not recommended as it may be removed in a future release of DocStrip.

\include To provide the DocStrip program with a list of options that should be included in the output the command \include{⟨Options⟩} can be used. This macro is meant to be used in conjunction with the \processFile command.

659 \def\include#1{\def\Options{#1}}

\processFile The macro \processFile{⟨filename⟩}{⟨inext⟩}{⟨outext⟩}{⟨t|f ⟩} can be used when a single input file is used to produce a single output file. The macro is also used in the interactive mode of the DocStrip program.

The arguments ⟨inext⟩ and ⟨outext⟩ denote the extensions of the input and output files respectively. The fourth argument can be used to specify if an existing file should be overwritten without asking. If ⟨t⟩ is specified the program will ask for permission before overwriting an existing file.

This macro is defined using the more generic macro \generateFile.

660 \def\processFile#1#2#3#4{%
661 \generateFile{#1.#3}{#4}{\from{#1.#2}{\Options}}}
\processfile \generatefile Early versions of DocStrip defined \processfile and \generatefile instead of

the commands as they are defined in this version. To remain upwards compatible, we still provide these commands, but issue a warning when they are used.

662 \def\processfile{\Msg{%
663  ^^Jplease use \string\processFile\space instead of

664  \string\processfile!^^J}%
665  \processFile}
666 \def\generatefile{\Msg{%
667  ^^Jplease use \string\generateFile\space instead of
668  \string\generatefile!^^J}%
669  \generateFile}

6.7 Interaction with the user

\processbatchFile When DocStrip is run it always tries to use a batch file. For this purpose it calls the macro \processbatchFile. The first thing this macro does, is check whether the user has defined the control sequence \batchfile. If he did, it should contain the name of the file to process. If he didn’t a default name is tried. Whether or not the default batch file is used is remembered by setting the switch \ifDefault to ⟨true⟩ or ⟨false⟩.

670 \def\processbatchFile{%
671  \ifx\undefined\batchfile
672  \let\batchfile\DefaultbatchFile
673  \Defaulttrue
674  \else
675  \Defaultfalse
676  \fi

Now we try to open the batch file for reading.

677  \openin\currbatchFile \batchfile\relax
678  \ifeof\currbatchFile

If we didn’t succeed in opening the file, we assume that it does not exist. If we tried the default filename, we silently continue; the DocStrip program will switch to interactive mode in this case.

679  \ifDefault
680  \else

If we failed to open the user-supplied file, something is wrong and we warn him about it. This will also result in a switch to interactive mode.

681  \Msg{**************************************************^^J%
682  * Could not find your \string\batchfile=\batchfile^^J%
683  * Try to continue without it^^J%
684  **************************************************}%
685  \fi
686  \else

When we were successful in opening a file, we again have to check whether it was the default file. In that case we tell the user we found that file and ask him if he wants to use it.

687  \ifDefault
688  \Msg{**************************************************^^J%
689  * Batchfile \DefaultbatchFile\space found Use it? (y/n)?}%
690  \Ask\answer{%
691  **************************************************}%

692  \else

When it was the user-supplied file we can safely assume he wants to use it so we

set \answer to y.
693  \let\answer\y
694  \fi

At this point we have successfully opened a batch file for reading. If the macro \answer contains a y we can proceed. We do that by setting the switch \ifContinue to true. When the batch file contains an \endinput we shouldn’t continue, so we make it switch \ifContinue to ⟨false⟩.

695  \ifx\answer\y
696  \Continuetrue
697  \let\endinput\Continuefalse

The contents of the batch file are read in a loop, line by line. When the end of the file is found processing should stop.

698  \loop
699  \ifeof\currbatchFile
700  \Continuefalse
701  \else

A line is read into the macro \batchLine, which is subsequently executed.

702  \read\currbatchFile to \batchLine
703  \batchLine
704  \fi

When the execution of \batchLine is finished we check the switch \ifContinue to see if have to continue reading.

705  \ifContinue \repeat
706  \fi
707  \fi}

\ReportTotals The macro \ReportTotals can be used to report total statistics for all files processed. This code is only included in the program if the option stats is included.

708 ⟨∗stats⟩

709 \def\ReportTotals{%
710 \ifnum\NumberOfFiles>\@ne
711  \Msg{Overall statistics:^^J%
712  Files \space processed: \the\NumberOfFiles^^J%
713  Lines \space processed: \the\TotalprocessedLines^^J%
714  Comments removed: \the\TotalcommentsRemoved^^J%
715  Comments \space passed: \the\TotalcommentsPassed^^J%
716  Codelines passed: \the\TotalcodeLinesPassed}%
717 \fi}

718 ⟨/stats⟩

\SetFileNames The macro \SetFileNames is used when the program runs in interactive mode and the user was asked to supply extensions and a list of filenames.

719 \def\SetFileNames{%
720  \edef\sourceFileName{\MainFileName.\infileext}%
721  \edef\destFileName{\MainFileName.\outfileext}}

\CheckFileNames In interactive mode, the user gets asked for the extensions for the input and output files. Also the name or names of the input files (without extension) is asked for. Then the names of the input and output files are constructed from this information by \SetFileNames. This assumes that the name of the input file is the same as the name of the output file. But we should not write to the same file we’re reading from so the extensions should differ.

The macro \CheckFileNames makes sure that the output goes to a different file to the one where the input comes from.

722 \def\CheckFileNames{%
723  \ifx\sourceFileName\destFileName

If input and output files are the same we signal an error and stop processing.

724  \Msg{^^J%
725  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!^^J%
726  ! It is not possible to read from and write to the same file !^^J%
727  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!^^J}%
728  \Continuefalse
729  \else

If they are not the same we check if the input file exists by trying to open it for reading.

730  \Continuetrue
731  \immediate\openin\inFile \sourceFileName\relax
732  \ifeof\inFile

If an end of file was found, the file couldn’t be opened, so we signal an error and stop processing.

733  \Msg{^^J%
734  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!^^J%
735  ! Your input file `\sourceFileName' was not found !^^J%
736  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!^^J}%
737  \Continuefalse
738  \else

The last check we have to make is if the output file already exists. Therefore we try to open it for reading. As a precaution we first close the input stream.

739  \immediate\closein\inFile
740  \immediate\openin\inFile\WriteToDir \destFileName\relax
741  \ifeof\inFile

If this fails, it didn’t exist and all is well.

742  \Continuetrue
743  \else

If opening of the output file for reading succeeded we have to ask the user if he wants to overwrite it. We assume he doesn’t want to overwrite it, so the switch \ifContinue is initially set to ⟨false⟩. Only if he answers the question positively with ‘y’ or ‘yes’ we set the switch back to ⟨true⟩.

744  \Continuefalse
745  \Ask\answer{File \WriteToDir\destFileName\space already
746  exists

747  \ifx\empty\WriteToDir somewhere \fi
748  on the system.^^J%
749  Overwrite it%
750  \ifx\empty\WriteToDir\space if necessary\fi
751  ? [y/n]}%
752  \ifx\y  \answer \Continuetrue \else
753  \ifx\yes\answer \Continuetrue \else
754  \fi\fi
755  \fi

All checks have been performed now, so we can close any file that was opened just for this purpose.

756  \fi
757  \fi
758  \immediate\closein\inFile}

\interactive The macro \interactive implements the interactive mode of the DocStrip program. The macro is implemented using the ⟨while⟩ construction. While the switch \ifMoreFiles remains true, we continue processing.

759 \def\interactive{%
760 \whileswitch\ifMoreFiles\fi%

To keep macro redefinitions local we start a group and ask the user some questions about what he wants us to do.

761 {\begingroup
762  \AskQuestions

The names of the files that have to be processed are stored as a comma-separated list in the macro \filelist by \AskQuestions. We use a ⟨for⟩ loop to process the files one by one.

763  \forlist\MainFileName:=\filelist
764  \do

First the names of the input and output files are constructed and a check is made if all filename information is correct.

765  \SetFileNames
766  \CheckFileNames
767  \ifContinue

If everything was well, produce output file.

768  \generateFile{\destFileName}{f}%
769  {\from{\sourceFileName}{\Options}}
770  \fi%

This process is repeated until \filelist is exhausted.

771  \od
772  \endgroup

Maybe the user wants more files to be processed, possibly with another set of options, so we give him the opportunity.

773  \Ask\answer{More files to process (y/n)?}%
774  \ifx\y  \answer\MoreFilestrue \else
775  \ifx\yes\answer\MoreFilestrue \else

If he didn’t want to process any more files, the switch \ifMoreFiles is set to ⟨false⟩ in order to interrupt the ⟨while⟩ loop.

776  \MoreFilesfalse\fi\fi

777 }}

\AskQuestions The macro \AskQuestions is called by \interactive to get some information

from the user concerning the files that need to be processed.

778 \def\AskQuestions{%
779  \Msg{^^J%
780  ****************************************************}%

We want to know the extension of the input files,

781  \Ask\infileext{%
782  * First type the extension of your input file(s): \space  *}%
783  \Msg{****************************************************^^J^^J%
784  ****************************************************}%

the extension of the output files,

785  \Ask\outfileext{%
786  * Now type the extension of your output file(s) \space: *}%
787  \Msg{****************************************************^^J^^J%
788  ****************************************************}%

if options are to be included and

789  \Ask\Options{%
790  * Now type the name(s) of option(s) to include \space\space: *}%
791  \Msg{****************************************************^^J^^J%
792  ****************************************************^^J%
793  * Finally give the list of input file(s) without \space\space*}%

the name of the input file or a list of names, separated by commas.

794  \Ask\filelist{%
795  * extension seperated by commas if necessary %
796  \space\space\space\space: *}%
797  \Msg{****************************************************^^J}}%

6.8 The main program

When TEX processes the DocStrip program it displays a message about the version of the program and its function on the terminal.

798 \Msg{Utility: `docstrip' \fileversion\space <\filedate>^^J%
799  English documentation \space\space\space <\docdate>}%
800 \Msg{^^J%
801  **********************************************************^^J%
802  * This program converts documented macro-files into fast *^^J%
803  * loadable files by stripping off (nearly) all comments! *^^J%
804  **********************************************************^^J}%

First we try to process a batch file.

805 \processbatchFile

When the number of files processed is still zero, no batch file was specified and the default batch file was not found, so we try interactive mode.

806 \ifnum\NumberOfFiles=\z@
807  \interactive
808 \fi

When we’re done and statistics are included we provide a statistics report about the complete run. Then we \end the TEX-run.

809 ⟨∗stats⟩

810 \ReportTotals

811 ⟨/stats⟩

812 \end

813 ⟨/program⟩