MAKE: redux patch

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

MAKE: redux patch

Marc Espie-2
So my development branch is getting a bit too far
ahead compared to what's committed.

Here's a big diff to test.  Comments as to what's going on
and the changes this contains:

- buffer changes: add support for "permanent static buffers"
that don't get reinit'd all the time (that's a mostly trivial
change)

- parser change: SPECIAL_TARGET and SPECIAL_SOURCE are not
really needed (figured out in netbsd)
- (related): create the special nodes once in a simpler way

- a bit of reorg: take the code for expand_children out
of suff.c proper, as this file is always fairly unwieldy.
This only requires Link_Parent, and find_best_path, which
are suff-independent.

- engine change: stop mixing parallel and compat so much.
In particular, don't run Make_Update when in compat mode (which
would happen because handle_all_signals goes into the main loop)

- (related): abstract the engine running into operations in
'enginechoice.c' so that it's perfectly clear what part is compatMake
and what part is not.   Have it report out_of_date and errors in
a simple way.  Don't mix up .BEGIN/.END handling.

- (related): job.c determines whether to use the expensive heuristics
or not depending on how many jobs we run.

- jobrunner cleanup: we have no need for maxjobs, it's enough to move
stuff around from available into running or error... do the move to
error later so that everything is in the same location.

- with the above changes: the special case of running_one_job is no
longer needed at all.

- feature coming from bmake: treat .BEGIN/.END more as normal targets.
More specifically, just invoke the engine on them if they exist.
This makes it possible to have
.BEGIN: somethingelse
which does make a lot of sense, actually.

I would really need this to go in so that I can keep pushing forward.

I think I might take out the "old" parallel engine (which is somewhat
broken, and frankly, I don't get how it can work) because modifying
compat to actually queue stuff and build it  is a distinct possibility
now.  e.g., having an actual parallel engine that works.


diff --git a/Makefile b/Makefile
index 0cd84fc..90747de 100644
--- a/Makefile
+++ b/Makefile
@@ -16,8 +16,8 @@ CFLAGS+=${CDEFS}
 HOSTCFLAGS+=${CDEFS}
 
 SRCS= arch.c buf.c cmd_exec.c compat.c cond.c dir.c direxpand.c dump.c \
- engine.c \
- error.c for.c init.c job.c lowparse.c main.c make.c memory.c parse.c \
+ engine.c enginechoice.c error.c expandchildren.c \
+ for.c init.c job.c lowparse.c main.c make.c memory.c parse.c \
  parsevar.c str.c stats.c suff.c targ.c targequiv.c timestamp.c \
  var.c varmodifiers.c varname.c
 
diff --git a/arch.c b/arch.c
index 85e8e7e..02c18b5 100644
--- a/arch.c
+++ b/arch.c
@@ -195,11 +195,10 @@ bool
 Arch_ParseArchive(const char **line, Lst nodes, SymTable *ctxt)
 {
  bool result;
- BUFFER expand;
+ static BUFFER expand;
 
- Buf_Init(&expand, MAKE_BSIZE);
+ Buf_Reinit(&expand, MAKE_BSIZE);
  result = parse_archive(&expand, line, nodes, ctxt);
- Buf_Destroy(&expand);
  return result;
 }
 
diff --git a/buf.c b/buf.c
index 74cbfe8..931bd7e 100644
--- a/buf.c
+++ b/buf.c
@@ -153,6 +153,15 @@ Buf_printf(Buffer bp, const char *fmt, ...)
  bp->inPtr += n;
 }
 
+void
+Buf_Reinit(Buffer bp, size_t size)
+{
+ if (bp->buffer == NULL)
+ Buf_Init(bp, size);
+ else
+ Buf_Reset(bp);
+}
+
 void
 Buf_Init(Buffer bp, size_t size)
 {
diff --git a/buf.h b/buf.h
index 1b56b27..20ea56a 100644
--- a/buf.h
+++ b/buf.h
@@ -106,6 +106,9 @@ extern void Buf_AddChars(Buffer, size_t, const char *);
  * Initializes a buffer, to hold approximately init chars.
  * Set init to 0 if you have no idea.  */
 extern void Buf_Init(Buffer, size_t);
+/* Buf_Reinit(buf, init);
+ * Initializes/reset a static buffer */
+extern void Buf_Reinit(Buffer, size_t);
 /* Buf_Destroy(buf);
  * Nukes a buffer and all its resources. */
 #define Buf_Destroy(bp) ((void)free((bp)->buffer))
diff --git a/compat.c b/compat.c
index 069e52f..fd78d78 100644
--- a/compat.c
+++ b/compat.c
@@ -193,14 +193,12 @@ CompatMake(void *gnp, /* The node to make */
  /* copy over what we just did */
  gn->built_status = sib->built_status;
 
- if (gn->built_status != ERROR) {
- /* If the node was built successfully, mark it so,
+ if (gn->built_status == REBUILT) {
+ /* If the node was built successfully,
  * update its modification time and timestamp all
  * its parents.
  * This is to keep its state from affecting that of
  * its parent.  */
- gn->built_status = REBUILT;
- sib->built_status = REBUILT;
  /* This is what Make does and it's actually a good
  * thing, as it allows rules like
  *
@@ -266,12 +264,20 @@ CompatMake(void *gnp, /* The node to make */
  }
 }
 
-bool
-Compat_Run(Lst targs) /* List of target nodes to re-create */
+void
+Compat_Init()
+{
+}
+
+void
+Compat_Update(GNode *gn)
+{
+}
+
+void
+Compat_Run(Lst targs, bool *has_errors, bool *out_of_date)
 {
  GNode  *gn = NULL; /* Current root target */
- int  errors;   /* Number of targets not built due to errors */
- bool out_of_date = false;
 
  /* For each entry in the list of targets to create, call CompatMake on
  * it to create the thing. CompatMake will leave the 'built_status'
@@ -283,7 +289,6 @@ Compat_Run(Lst targs) /* List of target nodes to re-create */
  *    ABORTED    gn was not built because one of its
  *                          dependencies could not be built due
  *          to errors.  */
- errors = 0;
  while ((gn = Lst_DeQueue(targs)) != NULL) {
  CompatMake(gn, NULL);
 
@@ -292,15 +297,10 @@ Compat_Run(Lst targs) /* List of target nodes to re-create */
  else if (gn->built_status == ABORTED) {
  printf("`%s' not remade because of errors.\n",
     gn->name);
- out_of_date = true;
- errors++;
+ *out_of_date = true;
+ *has_errors = true;
  } else {
- out_of_date = true;
+ *out_of_date = true;
  }
  }
-
- /* If the user has defined a .END target, run its commands.  */
- if (errors == 0 && !queryFlag)
- run_gnode(end_node);
- return out_of_date;
 }
diff --git a/compat.h b/compat.h
index 2420abd..0aa2de9 100644
--- a/compat.h
+++ b/compat.h
@@ -35,9 +35,11 @@
  *    - friendly variable substitution.
  */
 
-/* out_of_date = Compat_Run(to_create);
+/* Compat_Run(to_create, &has_errors, &out_of_date);
  * Run the actual make engine, to create targets that need to,
- * return true if any target is out of date. */
-extern bool Compat_Run(Lst);
+ * return info about what we did. */
+extern void Compat_Run(Lst, bool *, bool *);
+extern void Compat_Init(void);
+extern void Compat_Update(GNode *);
 
 #endif
diff --git a/dump.c b/dump.c
index 0f79c33..b3820eb 100644
--- a/dump.c
+++ b/dump.c
@@ -104,7 +104,7 @@ TargPrintNode(GNode *gn, bool full)
 {
  if (OP_NOP(gn->type))
  return;
- switch((gn->special & SPECIAL_MASK)) {
+ switch(gn->special) {
  case SPECIAL_SUFFIXES:
  case SPECIAL_PHONY:
  case SPECIAL_ORDER:
diff --git a/engine.c b/engine.c
index 7c786eb..7a9e912 100644
--- a/engine.c
+++ b/engine.c
@@ -603,8 +603,6 @@ run_command(const char *cmd, bool errCheck)
  _exit(1);
 }
 
-static Job myjob;
-
 void
 job_attach_node(Job *job, GNode *node)
 {
@@ -617,7 +615,7 @@ job_attach_node(Job *job, GNode *node)
 }
 
 void
-job_handle_status(Job *job, int status)
+handle_job_status(Job *job, int status)
 {
  bool silent;
  int dying;
@@ -666,19 +664,21 @@ job_handle_status(Job *job, int status)
  printf(" in target '%s'", job->node->name);
  if (job->flags & JOB_ERRCHECK) {
  job->node->built_status = ERROR;
- /* compute expensive status if we really want it */
- if ((job->flags & JOB_SILENT) && job == &myjob)
- determine_expensive_job(job);
  if (!keepgoing) {
  if (!silent)
  printf("\n");
- job->next = errorJobs;
- errorJobs = job;
+ job->flags |= JOB_KEEPERROR;
  /* XXX don't free the command */
  return;
  }
  printf(", line %lu of %s", job->location->lineno,
     job->location->fname);
+ /* Parallel make already determined whether
+ * JOB_IS_EXPENSIVE, perform the computation for
+ * sequential make to figure out whether to display the
+ * command or not.  */
+ if ((job->flags & JOB_SILENT) && sequential)
+ determine_expensive_job(job);
  if ((job->flags & (JOB_SILENT | JOB_IS_EXPENSIVE))
     == JOB_SILENT)
  printf(": %s", job->cmd);
@@ -699,17 +699,12 @@ job_handle_status(Job *job, int status)
 int
 run_gnode(GNode *gn)
 {
+ Job *j;
  if (!gn || (gn->type & OP_DUMMY))
  return NOSUCHNODE;
 
- job_attach_node(&myjob, gn);
- while (myjob.exit_type == JOB_EXIT_OKAY) {
- bool finished = job_run_next(&myjob);
- if (finished)
- break;
- handle_one_job(&myjob);
- }
-
+ Job_Make(gn);
+ loop_handle_running_jobs();
  return gn->built_status;
 }
 
@@ -792,7 +787,7 @@ do_run_command(Job *job, const char *pre)
  * and there's nothing left to do.
  */
  if (random_delay)
- if (!(runningJobs == NULL && no_jobs_left()))
+ if (!(runningJobs == NULL && nothing_left_to_build()))
  usleep(arc4random_uniform(random_delay));
  run_command(cmd, errCheck);
  /*NOTREACHED*/
diff --git a/engine.h b/engine.h
index cb96139..ce7ebd4 100644
--- a/engine.h
+++ b/engine.h
@@ -103,19 +103,20 @@ struct Job_ {
  struct Job_ *next; /* singly linked list */
  pid_t pid; /* Current command process id */
  Location *location;
- int exit_type; /* last child exit or signal */
-#define JOB_EXIT_OKAY 0
-#define JOB_EXIT_BAD 1
-#define JOB_SIGNALED 2
  int code; /* exit status or signal code */
+ unsigned short exit_type; /* last child exit or signal */
+#define JOB_EXIT_OKAY 0
+#define JOB_EXIT_BAD 1
+#define JOB_SIGNALED 2
+ unsigned short flags;
+#define JOB_SILENT 0x001 /* Command was silent */
+#define JOB_IS_EXPENSIVE 0x002
+#define JOB_LOST 0x004 /* sent signal to non-existing pid ? */
+#define JOB_ERRCHECK 0x008 /* command wants errcheck */
+#define JOB_KEEPERROR 0x010 /* should place job on error list */
  LstNode next_cmd; /* Next command to run */
  char *cmd; /* Last command run */
  GNode *node;     /* Target of this job */
- unsigned short flags;
-#define JOB_SILENT 0x001 /* Command was silent */
-#define JOB_IS_EXPENSIVE 0x002
-#define JOB_LOST 0x004 /* sent signal to non-existing pid ? */
-#define JOB_ERRCHECK 0x008 /* command wants errcheck */
 };
 
 /* Continuation-style running commands for the parallel engine */
@@ -131,10 +132,10 @@ extern void job_attach_node(Job *, GNode *);
  */
 extern bool job_run_next(Job *);
 
-/* job_handle_status(job, waitstatus):
+/* handle_job_status(job, waitstatus):
  * process a wait return value corresponding to a job, display
  * messages and set job status accordingly.
  */
-extern void job_handle_status(Job *, int);
+extern void handle_job_status(Job *, int);
 
 #endif
diff --git a/enginechoice.c b/enginechoice.c
new file mode 100644
index 0000000..32ba046
--- /dev/null
+++ b/enginechoice.c
@@ -0,0 +1,32 @@
+#include "config.h"
+#include "defines.h"
+#include "compat.h"
+#include "make.h"
+
+struct engine {
+ void (*run_list)(Lst, bool *, bool *);
+ void (*node_updated)(GNode *);
+ void (*init)(void);
+}
+ compat_engine = { Compat_Run, Compat_Update, Compat_Init },
+ parallel_engine = { Make_Run, Make_Update, Make_Init },
+ *engine;
+
+void
+choose_engine(bool compat)
+{
+ engine = compat ? &compat_engine: &parallel_engine;
+ engine->init();
+}
+
+void
+engine_run_list(Lst l, bool *has_errors, bool *out_of_date)
+{
+ engine->run_list(l, has_errors, out_of_date);
+}
+
+void
+engine_node_updated(GNode *gn)
+{
+ engine->node_updated(gn);
+}
diff --git a/enginechoice.h b/enginechoice.h
new file mode 100644
index 0000000..6441231
--- /dev/null
+++ b/enginechoice.h
@@ -0,0 +1,8 @@
+#ifndef ENGINECHOICE_H
+#define ENGINECHOICE_H
+
+extern void engine_run_list(Lst, bool *, bool *);
+extern void engine_node_updated(GNode *);
+extern void choose_engine(bool);
+
+#endif
diff --git a/expandchildren.c b/expandchildren.c
new file mode 100644
index 0000000..28a1004
--- /dev/null
+++ b/expandchildren.c
@@ -0,0 +1,246 @@
+/* $OpenBSD$ */
+/* $NetBSD: suff.c,v 1.13 1996/11/06 17:59:25 christos Exp $ */
+
+/*
+ * Copyright (c) 1988, 1989, 1990, 1993
+ * The Regents of the University of California.  All rights reserved.
+ * Copyright (c) 1989 by Berkeley Softworks
+ * All rights reserved.
+ *
+ * This code is derived from software contributed to Berkeley by
+ * Adam de Boor.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+/*-
+ * expandchildren.c --
+ * Dealing with final children expansion before building stuff
+ */
+
+#include <ctype.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include "config.h"
+#include "defines.h"
+#include "direxpand.h"
+#include "engine.h"
+#include "arch.h"
+#include "expandchildren.h"
+#include "var.h"
+#include "targ.h"
+#include "lst.h"
+#include "gnode.h"
+#include "suff.h"
+
+static void ExpandChildren(LstNode, GNode *);
+static void ExpandVarChildren(LstNode, GNode *, GNode *);
+static void ExpandWildChildren(LstNode, GNode *, GNode *);
+
+void
+LinkParent(GNode *cgn, GNode *pgn)
+{
+ Lst_AtEnd(&cgn->parents, pgn);
+ if (!has_been_built(cgn))
+ pgn->children_left++;
+ else if ( ! (cgn->type & (OP_EXEC|OP_USE))) {
+ if (cgn->built_status == REBUILT)
+ pgn->child_rebuilt = true;
+ (void)Make_TimeStamp(pgn, cgn);
+ }
+}
+
+static void
+ExpandVarChildren(LstNode after, GNode *cgn, GNode *pgn)
+{
+ GNode *gn; /* New source 8) */
+ char *cp; /* Expanded value */
+ LIST members;
+
+
+ if (DEBUG(SUFF))
+ printf("Expanding \"%s\"...", cgn->name);
+
+ cp = Var_Subst(cgn->name, &pgn->localvars, true);
+ if (cp == NULL) {
+ printf("Problem substituting in %s", cgn->name);
+ printf("\n");
+ return;
+ }
+
+ Lst_Init(&members);
+
+ if (cgn->type & OP_ARCHV) {
+ /*
+ * Node was an archive(member) target, so we want to call
+ * on the Arch module to find the nodes for us, expanding
+ * variables in the parent's context.
+ */
+ const char *sacrifice = (const char *)cp;
+
+ (void)Arch_ParseArchive(&sacrifice, &members, &pgn->localvars);
+ } else {
+ /* Break the result into a vector of strings whose nodes
+ * we can find, then add those nodes to the members list.
+ * Unfortunately, we can't use brk_string because it
+ * doesn't understand about variable specifications with
+ * spaces in them...  */
+ const char *start, *cp2;
+
+ for (start = cp; *start == ' ' || *start == '\t'; start++)
+ continue;
+ for (cp2 = start; *cp2 != '\0';) {
+ if (ISSPACE(*cp2)) {
+ /* White-space -- terminate element, find the
+ * node, add it, skip any further spaces.  */
+ gn = Targ_FindNodei(start, cp2, TARG_CREATE);
+ cp2++;
+ Lst_AtEnd(&members, gn);
+ while (ISSPACE(*cp2))
+ cp2++;
+ /* Adjust cp2 for increment at start of loop,
+ * but set start to first non-space.  */
+ start = cp2;
+ } else if (*cp2 == '$')
+ /* Start of a variable spec -- contact variable
+ * module to find the end so we can skip over
+ * it.  */
+ Var_ParseSkip(&cp2, &pgn->localvars);
+ else if (*cp2 == '\\' && cp2[1] != '\0')
+ /* Escaped something -- skip over it.  */
+ cp2+=2;
+ else
+ cp2++;
+    }
+
+    if (cp2 != start) {
+    /* Stuff left over -- add it to the list too.  */
+    gn = Targ_FindNodei(start, cp2, TARG_CREATE);
+    Lst_AtEnd(&members, gn);
+    }
+ }
+ /* Add all elements of the members list to the parent node.  */
+ while ((gn = Lst_DeQueue(&members)) != NULL) {
+ if (DEBUG(SUFF))
+ printf("%s...", gn->name);
+ if (Lst_Member(&pgn->children, gn) == NULL) {
+ Lst_Append(&pgn->children, after, gn);
+ after = Lst_Adv(after);
+ LinkParent(gn, pgn);
+ }
+ }
+ /* Free the result.  */
+ free(cp);
+ if (DEBUG(SUFF))
+ printf("\n");
+}
+
+static void
+ExpandWildChildren(LstNode after, GNode *cgn, GNode *pgn)
+{
+ char *cp; /* Expanded value */
+
+ LIST exp; /* List of expansions */
+ Lst path; /* Search path along which to expand */
+
+ if (DEBUG(SUFF))
+ printf("Wildcard expanding \"%s\"...", cgn->name);
+
+ /* Find a path along which to expand the word: if
+ * the word has a known suffix, use the path for that suffix,
+ * otherwise use the default path. */
+ path = find_best_path(cgn->name);
+
+ /* Expand the word along the chosen path. */
+ Lst_Init(&exp);
+ Dir_Expand(cgn->name, path, &exp);
+
+ /* Fetch next expansion off the list and find its GNode.  */
+ while ((cp = Lst_DeQueue(&exp)) != NULL) {
+ GNode *gn; /* New source 8) */
+ if (DEBUG(SUFF))
+ printf("%s...", cp);
+ gn = Targ_FindNode(cp, TARG_CREATE);
+
+ /* If gn isn't already a child of the parent, make it so and
+ * up the parent's count of children to build.  */
+ if (Lst_Member(&pgn->children, gn) == NULL) {
+ Lst_Append(&pgn->children, after, gn);
+ after = Lst_Adv(after);
+ LinkParent(gn, pgn);
+ }
+ }
+
+ if (DEBUG(SUFF))
+ printf("\n");
+}
+
+/*-
+ *-----------------------------------------------------------------------
+ * ExpandChildren --
+ * Expand the names of any children of a given node that contain
+ * variable invocations or file wildcards into actual targets.
+ *
+ * Side Effects:
+ * The expanded node is removed from the parent's list of children,
+ * and the parent's children to build counter is decremented,
+ *      but other nodes may be added.
+ *-----------------------------------------------------------------------
+ */
+static void
+ExpandChildren(LstNode ln, /* LstNode of child, so we can replace it */
+    GNode *pgn)
+{
+ GNode *cgn = Lst_Datum(ln);
+
+ /* First do variable expansion -- this takes precedence over wildcard
+ * expansion. If the result contains wildcards, they'll be gotten to
+ * later since the resulting words are tacked on to the end of the
+ * children list.  */
+ if (strchr(cgn->name, '$') != NULL)
+ ExpandVarChildren(ln, cgn, pgn);
+ else if (Dir_HasWildcards(cgn->name))
+ ExpandWildChildren(ln, cgn, pgn);
+ else
+    /* Third case: nothing to expand.  */
+ return;
+
+ /* Since the source was expanded, remove it from the list of children to
+ * keep it from being processed.  */
+ pgn->children_left--;
+ Lst_Remove(&pgn->children, ln);
+}
+
+void
+expand_children_from(GNode *parent, LstNode from)
+{
+ LstNode np, ln;
+
+ for (ln = from; ln != NULL; ln = np) {
+ np = Lst_Adv(ln);
+ ExpandChildren(ln, parent);
+ }
+}
diff --git a/expandchildren.h b/expandchildren.h
new file mode 100644
index 0000000..2c5212e
--- /dev/null
+++ b/expandchildren.h
@@ -0,0 +1,16 @@
+#ifndef EXPANDCHILDREN_H
+#define EXPANDCHILDREN_H
+/* $OpenBSD: suff.h,v 1.10 2012/12/06 14:30:35 espie Exp $ */
+
+extern void LinkParent(GNode *, GNode *);
+
+/* partial expansion of children. */
+extern void expand_children_from(GNode *, LstNode);
+/* expand_all_children(gn):
+ * figure out all variable/wildcards expansions in gn.
+ * TODO pretty sure this is independent from the main suff module.
+ */
+#define expand_all_children(gn) \
+    expand_children_from(gn, Lst_First(&(gn)->children))
+
+#endif
diff --git a/extern.h b/extern.h
index 89797d6..e461ecb 100644
--- a/extern.h
+++ b/extern.h
@@ -40,7 +40,6 @@
  * from: @(#)nonints.h 8.3 (Berkeley) 3/19/94
  */
 
-extern bool compatMake; /* True if we are make compatible */
 extern bool ignoreErrors; /* True if should ignore all errors */
 extern bool beSilent; /* True if should print no commands */
 extern bool noExecute; /* True if should execute nothing */
diff --git a/gnode.h b/gnode.h
index d014d87..283fead 100644
--- a/gnode.h
+++ b/gnode.h
@@ -65,32 +65,34 @@
  *   to create this target.
  */
 
+/* constants for specials
+ * Most of these values are only handled by parse.c.
+ * In many cases, there is a corresponding OP_* flag
+ */
 #define SPECIAL_NONE 0U
-#define SPECIAL_PATH 21U
-#define SPECIAL_MASK 63U
-#define SPECIAL_TARGET 64U
-#define SPECIAL_SOURCE 128U
-#define SPECIAL_TARGETSOURCE (SPECIAL_TARGET|SPECIAL_SOURCE)
+#define SPECIAL_PATH 62U /* handled by parse.c and suff.c */
 
-#define SPECIAL_EXEC 4U
+#define SPECIAL_EXEC 4U
 #define SPECIAL_IGNORE 5U
-#define SPECIAL_NOTHING 6U
-#define SPECIAL_INVISIBLE 8U
+#define SPECIAL_NOTHING 6U /* this is used for things we
+ * recognize for compatibility but
+ * don't do anything with... */
+#define SPECIAL_INVISIBLE 8U
 #define SPECIAL_JOIN 9U
 #define SPECIAL_MADE 11U
 #define SPECIAL_MAIN 12U
 #define SPECIAL_MAKE 13U
 #define SPECIAL_MFLAGS 14U
-#define SPECIAL_NOTMAIN 15U
-#define SPECIAL_NOTPARALLEL 16U
-#define SPECIAL_OPTIONAL 18U
+#define SPECIAL_NOTMAIN 15U
+#define SPECIAL_NOTPARALLEL 16U
+#define SPECIAL_OPTIONAL 18U
 #define SPECIAL_ORDER 19U
 #define SPECIAL_PARALLEL 20U
 #define SPECIAL_PHONY 22U
 #define SPECIAL_PRECIOUS 23U
 #define SPECIAL_SILENT 25U
 #define SPECIAL_SUFFIXES 27U
-#define SPECIAL_USE 28U
+#define SPECIAL_USE 28U
 #define SPECIAL_WAIT 29U
 #define SPECIAL_NOPATH 30U
 #define SPECIAL_ERROR 31U
diff --git a/job.c b/job.c
index 4374e71..0a2180f 100644
--- a/job.c
+++ b/job.c
@@ -70,19 +70,11 @@
  *
  * Job_Init Called to initialize this module.
  *
- * Job_Begin execute commands attached to the .BEGIN target
- * if any.
- *
  * can_start_job Return true if we can start job
  *
  * Job_Empty Return true if the job table is completely
  * empty.
  *
- * Job_Finish Perform any final processing which needs doing.
- * This includes the execution of any commands
- * which have been/were attached to the .END
- * target.
- *
  * Job_AbortAll Abort all current jobs. It doesn't
  * handle output or do anything for the jobs,
  * just kills them.
@@ -113,23 +105,24 @@
 #include "lst.h"
 #include "gnode.h"
 #include "memory.h"
-#include "make.h"
 #include "buf.h"
+#include "enginechoice.h"
 
 static int aborting = 0;    /* why is the make aborting? */
 #define ABORT_ERROR 1    /* Because of an error */
 #define ABORT_INTERRUPT 2    /* Because it was interrupted */
 #define ABORT_WAIT 3    /* Waiting for jobs to finish */
 
-static int maxJobs; /* The most children we can run at once */
-static int nJobs; /* Number of jobs already allocated */
 static bool no_new_jobs; /* Mark recursive shit so we shouldn't start
  * something else at the same time
  */
+bool sequential;
 Job *runningJobs; /* Jobs currently running a process */
 Job *errorJobs; /* Jobs in error at end */
+Job *availableJobs; /* Pool of available jobs */
 static Job *heldJobs; /* Jobs not running yet because of expensive */
 static pid_t mypid; /* Used for printing debugging messages */
+static Job *extra_job; /* Needed for .INTERRUPT */
 
 static volatile sig_atomic_t got_fatal;
 
@@ -141,15 +134,11 @@ static sigset_t sigset, emptyset;
 static void handle_fatal_signal(int);
 static void handle_siginfo(void);
 static void postprocess_job(Job *);
-static Job *prepare_job(GNode *);
 static void determine_job_next_step(Job *);
-static void remove_job(Job *);
 static void may_continue_job(Job *);
-static void continue_job(Job *);
 static Job *reap_finished_job(pid_t);
 static bool reap_jobs(void);
 
-static void loop_handle_running_jobs(void);
 static bool expensive_job(Job *);
 static bool expensive_command(const char *);
 static void setup_signal(int);
@@ -541,8 +530,14 @@ postprocess_job(Job *job)
  * non-zero status that we shouldn't ignore, we call
  * Make_Update to update the parents. */
  job->node->built_status = REBUILT;
- Make_Update(job->node);
- free(job);
+ engine_node_updated(job->node);
+ }
+ if (job->flags & JOB_KEEPERROR) {
+ job->next = errorJobs;
+ errorJobs = job;
+ } else {
+ job->next = availableJobs;
+ availableJobs = job;
  }
 
  if (errorJobs != NULL && aborting != ABORT_INTERRUPT)
@@ -567,10 +562,10 @@ postprocess_job(Job *job)
  * is set, so jobs that would fork new processes are accumulated in the
  * heldJobs list instead.
  *
- * This heuristics is also used on error exit: we display silent commands
- * that failed, unless those ARE expensive commands: expensive commands
- * are likely to not be failing by themselves, but to be the result of
- * a cascade of failures in descendant makes.
+ * XXX This heuristics is also used on error exit: we display silent commands
+ * that failed, unless those ARE expensive commands: expensive commands are
+ * likely to not be failing by themselves, but to be the result of a cascade of
+ * failures in descendant makes.
  */
 void
 determine_expensive_job(Job *job)
@@ -646,35 +641,6 @@ expensive_command(const char *s)
  return false;
 }
 
-static Job *
-prepare_job(GNode *gn)
-{
- /* a new job is prepared unless its commands are bogus (we don't
- * have anything for it), or if we're in touch mode.
- *
- * Note that even in noexec mode, some commands may still run
- * thanks to the +cmd construct.
- */
- if (node_find_valid_commands(gn)) {
- if (touchFlag) {
- Job_Touch(gn);
- return NULL;
- } else {
- Job *job;      
-
- job = emalloc(sizeof(Job));
- if (job == NULL)
- Punt("can't create job: out of memory");
-
- job_attach_node(job, gn);
- return job;
- }
- } else {
- node_failure(gn);
- return NULL;
- }
-}
-
 static void
 may_continue_job(Job *job)
 {
@@ -684,18 +650,29 @@ may_continue_job(Job *job)
     (long)mypid, job->node->name);
  job->next = heldJobs;
  heldJobs = job;
- } else
- continue_job(job);
+ } else {
+ bool finished = job_run_next(job);
+ if (finished)
+ postprocess_job(job);
+ else if (!sequential)
+ determine_expensive_job(job);
+ }
 }
 
 static void
-continue_job(Job *job)
+may_continue_heldback_jobs()
 {
- bool finished = job_run_next(job);
- if (finished)
- remove_job(job);
- else
- determine_expensive_job(job);
+ while (!no_new_jobs) {
+ if (heldJobs != NULL) {
+ Job *job = heldJobs;
+ heldJobs = heldJobs->next;
+ if (DEBUG(EXPENSIVE))
+ fprintf(stderr, "[%ld] cheap -> release %s\n",
+    (long)mypid, job->node->name);
+ may_continue_job(job);
+ } else
+ break;
+ }
 }
 
 /*-
@@ -712,19 +689,17 @@ continue_job(Job *job)
 void
 Job_Make(GNode *gn)
 {
- Job *job;
+ Job *job = availableJobs;      
 
- job = prepare_job(gn);
- if (!job)
- return;
- nJobs++;
+ assert(job != NULL);
+ availableJobs = availableJobs->next;
+ job_attach_node(job, gn);
  may_continue_job(job);
 }
 
 static void
 determine_job_next_step(Job *job)
 {
- bool okay;
  if (job->flags & JOB_IS_EXPENSIVE) {
  no_new_jobs = false;
  if (DEBUG(EXPENSIVE))
@@ -735,29 +710,11 @@ determine_job_next_step(Job *job)
  }
 
  if (job->exit_type != JOB_EXIT_OKAY || job->next_cmd == NULL)
- remove_job(job);
+ postprocess_job(job);
  else
  may_continue_job(job);
 }
 
-static void
-remove_job(Job *job)
-{
- nJobs--;
- postprocess_job(job);
- while (!no_new_jobs) {
- if (heldJobs != NULL) {
- job = heldJobs;
- heldJobs = heldJobs->next;
- if (DEBUG(EXPENSIVE))
- fprintf(stderr, "[%ld] cheap -> release %s\n",
-    (long)mypid, job->node->name);
- continue_job(job);
- } else
- break;
- }
-}
-
 /*
  * job = reap_finished_job(pid):
  * retrieve and remove a job from runningJobs, based on its pid
@@ -799,9 +756,10 @@ reap_jobs(void)
  if (job == NULL) {
  Punt("Child (%ld) not in table?", (long)pid);
  } else {
- job_handle_status(job, status);
+ handle_job_status(job, status);
  determine_job_next_step(job);
  }
+ may_continue_heldback_jobs();
  }
  /* sanity check, should not happen */
  if (pid == -1 && errno == ECHILD && runningJobs != NULL)
@@ -841,26 +799,6 @@ handle_running_jobs(void)
 }
 
 void
-handle_one_job(Job *job)
-{
- int stat;
- int status;
- sigset_t old;
-
- sigprocmask(SIG_BLOCK, &sigset, &old);
- while (1) {
- handle_all_signals();
- stat = waitpid(job->pid, &status, WNOHANG);
- if (stat == job->pid)
- break;
- sigsuspend(&emptyset);
- }
- runningJobs = NULL;
- job_handle_status(job, status);
- sigprocmask(SIG_SETMASK, &old, NULL);
-}
-
-static void
 loop_handle_running_jobs()
 {
  while (runningJobs != NULL)
@@ -868,16 +806,27 @@ loop_handle_running_jobs()
 }
 
 void
-Job_Init(int maxproc)
+Job_Init(int maxJobs)
 {
+ Job *j;
+ int i;
+
  runningJobs = NULL;
  heldJobs = NULL;
  errorJobs = NULL;
- maxJobs = maxproc;
+ availableJobs = NULL;
+ sequential = maxJobs == 1;
+
+ /* we allocate n+1 jobs, since we may need an extra job for
+ * running .INTERRUPT.  */
+ j = ereallocarray(NULL, sizeof(Job), maxJobs+1);
+ for (i = 0; i != maxJobs; i++) {
+ j[i].next = availableJobs;
+ availableJobs = &j[i];
+ }
+ extra_job = &j[maxJobs];
  mypid = getpid();
 
- nJobs = 0;
-
  aborting = 0;
  setup_all_signals();
 }
@@ -885,7 +834,7 @@ Job_Init(int maxproc)
 bool
 can_start_job(void)
 {
- if (aborting || nJobs >= maxJobs)
+ if (aborting || availableJobs == NULL)
  return false;
  else
  return true;
@@ -925,7 +874,8 @@ handle_fatal_signal(int signo)
  if (signo == SIGINT && !touchFlag) {
  if ((interrupt_node->type & OP_DUMMY) == 0) {
  ignoreErrors = false;
-
+ extra_job->next = availableJobs;
+ availableJobs = extra_job;
  Job_Make(interrupt_node);
  }
  }
@@ -942,40 +892,6 @@ handle_fatal_signal(int signo)
  exit(1);
 }
 
-/*
- *-----------------------------------------------------------------------
- * Job_Finish --
- * Do final processing such as the running of the commands
- * attached to the .END target.
- *
- * return true if fatal errors have happened.
- *-----------------------------------------------------------------------
- */
-bool
-Job_Finish(void)
-{
- bool problem = errorJobs != NULL;
-
- if ((end_node->type & OP_DUMMY) == 0) {
- if (problem) {
- Error("Errors reported so .END ignored");
- } else {
- Job_Make(end_node);
- loop_handle_running_jobs();
- }
- }
- return problem;
-}
-
-void
-Job_Begin(void)
-{
- if ((begin_node->type & OP_DUMMY) == 0) {
- Job_Make(begin_node);
- loop_handle_running_jobs();
- }
-}
-
 /*-
  *-----------------------------------------------------------------------
  * Job_Wait --
diff --git a/job.h b/job.h
index e8152d1..5356ee8 100644
--- a/job.h
+++ b/job.h
@@ -65,16 +65,6 @@ extern bool can_start_job(void);
  */
 extern bool Job_Empty(void);
 
-/* errors = Job_Finish();
- * final processing including running .END target if no errors.
- */
-extern bool Job_Finish(void);
-
-/* Job_Begin();
- * similarly, run .BEGIN job at start of job.
- */
-extern void Job_Begin(void);
-
 extern void Job_Wait(void);
 extern void Job_AbortAll(void);
 extern void print_errors(void);
@@ -84,6 +74,10 @@ extern void print_errors(void);
  * or a signal coming in.
  */
 extern void handle_running_jobs(void);
+/* loop_handle_running_jobs();
+ * handle running jobs until they're finished.
+ */
+extern void loop_handle_running_jobs(void);
 
 /* handle_all_signals();
  * if a signal was received, react accordingly.
@@ -93,11 +87,13 @@ extern void handle_running_jobs(void);
 extern void handle_all_signals(void);
 
 extern void determine_expensive_job(Job *);
-extern Job *runningJobs, *errorJobs;
+extern Job *runningJobs, *errorJobs, *availableJobs;
 extern void debug_job_printf(const char *, ...);
 extern void handle_one_job(Job *);
 extern int check_dying_signal(void);
 
 extern const char *basedirectory;
 
+extern bool sequential; /* True if we are running one single-job */
+
 #endif /* _JOB_H_ */
diff --git a/main.c b/main.c
index 9d3882a..ea0947c 100644
--- a/main.c
+++ b/main.c
@@ -57,15 +57,14 @@
 #include "pathnames.h"
 #include "init.h"
 #include "job.h"
-#include "compat.h"
 #include "targ.h"
 #include "suff.h"
 #include "str.h"
 #include "main.h"
 #include "lst.h"
 #include "memory.h"
-#include "make.h"
 #include "dump.h"
+#include "enginechoice.h"
 
 #define MAKEFLAGS ".MAKEFLAGS"
 
@@ -73,12 +72,12 @@ static LIST to_create; /* Targets to be made */
 Lst create = &to_create;
 bool allPrecious; /* .PRECIOUS given on line by itself */
 
-static bool noBuiltins; /* -r flag */
-static LIST makefiles; /* ordered list of makefiles to read */
-static LIST varstoprint; /* list of variables to print */
-int maxJobs; /* -j argument */
-bool compatMake; /* -B argument */
-static bool forceJobs = false;
+static bool noBuiltins; /* -r flag */
+static LIST makefiles; /* ordered list of makefiles to read */
+static LIST varstoprint; /* list of variables to print */
+static int optj; /* -j argument */
+static bool compatMake; /* -B argument */
+static bool forceJobs = false;
 int debug; /* -d flag */
 bool noExecute; /* -n flag */
 bool keepgoing; /* -k flag */
@@ -126,6 +125,12 @@ record_option(int c, const char *arg)
  Var_Append(MAKEFLAGS, arg);
 }
 
+void
+set_notparallel()
+{
+ compatMake = true;
+}
+
 static void
 posixParseOptLetter(int c)
 {
@@ -313,7 +318,7 @@ MainParseArgs(int argc, char **argv)
  const char *errstr;
 
  forceJobs = true;
- maxJobs = strtonum(optarg, 1, INT_MAX, &errstr);
+ optj = strtonum(optarg, 1, INT_MAX, &errstr);
  if (errstr != NULL) {
  fprintf(stderr,
     "make: illegal argument to -j option"
@@ -623,30 +628,25 @@ read_all_make_rules(bool noBuiltins, bool read_depend,
  Parse_End();
 }
 
+static void
+run_node(GNode *gn, bool *has_errors, bool *out_of_date)
+{
+ LIST l;
+
+ Lst_Init(&l);
+ Lst_AtEnd(&l, gn);
+ engine_run_list(&l, has_errors, out_of_date);
+ Lst_Destroy(&l, NOFREE);
+}
 
 int main(int, char **);
-/*-
- * main --
- * The main function, for obvious reasons. Initializes variables
- * and a few modules, then parses the arguments give it in the
- * environment and on the command line. Reads the system makefile
- * followed by either Makefile, makefile or the file given by the
- * -f argument. Sets the .MAKEFLAGS PMake variable based on all the
- * flags it has received by then uses either the Make or the Compat
- * module to create the initial list of targets.
- *
- * Results:
- * If -q was given, exits -1 if anything was out-of-date. Else it exits
- * 0.
- *
- * Side Effects:
- * The program exits when done. Targets are created. etc. etc. etc.
- */
+
 int
 main(int argc, char **argv)
 {
  static LIST targs; /* target nodes to create */
- bool outOfDate = true; /* false if all targets up to date */
+ bool outOfDate = false; /* false if all targets up to date */
+ bool errored = false; /* true if errors occurred */
  char *machine = figure_out_MACHINE();
  char *machine_arch = figure_out_MACHINE_ARCH();
  char *machine_cpu = figure_out_MACHINE_CPU();
@@ -665,6 +665,7 @@ main(int argc, char **argv)
  Static_Lst_Init(&makefiles);
  Static_Lst_Init(&varstoprint);
  Static_Lst_Init(&targs);
+ Static_Lst_Init(&special);
 
  beSilent = false; /* Print commands as executed */
  ignoreErrors = false; /* Pay attention to non-zero returns */
@@ -676,7 +677,7 @@ main(int argc, char **argv)
  touchFlag = false; /* Actually update targets */
  debug = 0; /* No debug verbosity, please. */
 
- maxJobs = DEFMAXJOBS;
+ optj = DEFMAXJOBS;
  compatMake = false; /* No compat mode */
 
 
@@ -757,6 +758,9 @@ main(int argc, char **argv)
 
  read_all_make_rules(noBuiltins, read_depend, &makefiles, &d);
 
+ if (compatMake)
+ optj = 1;
+
  Var_Append("MFLAGS", Var_Value(MAKEFLAGS));
 
  /* Install all the flags into the MAKEFLAGS env variable. */
@@ -796,25 +800,27 @@ main(int argc, char **argv)
  else
  Targ_FindList(&targs, create);
 
- Job_Init(maxJobs);
- /* If the user has defined a .BEGIN target, execute the commands
- * attached to it.  */
- if (!queryFlag)
- Job_Begin();
- if (compatMake)
- /* Compat_Init will take care of creating all the
- * targets as well as initializing the module.  */
- outOfDate =Compat_Run(&targs);
- else {
- /* Traverse the graph, checking on all the targets.  */
- outOfDate = Make_Run(&targs);
- }
+ choose_engine(compatMake);
+ Job_Init(optj);
+ if (!queryFlag && node_is_real(begin_node))
+ run_node(begin_node, &errored, &outOfDate);
+
+ if (!errored)
+ engine_run_list(&targs, &errored, &outOfDate);
+
+ if (!queryFlag && !errored && node_is_real(end_node))
+ run_node(end_node, &errored, &outOfDate);
  }
 
  /* print the graph now it's been processed if the user requested it */
  if (DEBUG(GRAPH2))
  post_mortem();
 
+ /* Note that we only hit this code if -k is used, otherwise we
+ * exited early in case of errors. */
+ if (errored)
+ Fatal("Errors while building");
+
  if (queryFlag && outOfDate)
  return 1;
  else
diff --git a/main.h b/main.h
index 469487e..3f5ce13 100644
--- a/main.h
+++ b/main.h
@@ -38,4 +38,7 @@ extern void Main_ParseArgLine(const char *);
  * .if make(...) statements. */
 extern Lst create;
 
+/* set_notparallel(): used to influence running mode from parse.c */
+extern void set_notparallel(void);
+
 #endif
diff --git a/make.c b/make.c
index 4ffdda6..1eff38e 100644
--- a/make.c
+++ b/make.c
@@ -71,6 +71,7 @@
 #include "suff.h"
 #include "var.h"
 #include "error.h"
+#include "expandchildren.h"
 #include "make.h"
 #include "gnode.h"
 #include "extern.h"
@@ -118,7 +119,7 @@ static bool randomize_queue;
 long random_delay = 0;
 
 bool
-no_jobs_left()
+nothing_left_to_build()
 {
  return Array_IsEmpty(&to_build);
 }
@@ -376,7 +377,13 @@ try_to_make_node(GNode *gn)
  return true;
  /* SIB: this is where commands should get prepared */
  Make_DoAllVar(gn);
- Job_Make(gn);
+ if (node_find_valid_commands(gn)) {
+ if (touchFlag)
+ Job_Touch(gn);
+ else
+ Job_Make(gn);
+ } else
+ node_failure(gn);
  } else {
  if (DEBUG(MAKE))
  printf("up-to-date\n");
@@ -504,6 +511,16 @@ add_targets_to_make(Lst todo)
  randomize_garray(&to_build);
 }
 
+void
+Make_Init()
+{
+ /* wild guess at initial sizes */
+ Array_Init(&to_build, 500);
+ Array_Init(&examine, 150);
+ Array_Init(&heldBack, 100);
+ ohash_init(&targets, 10, &gnode_info);
+}
+
 /*-
  *-----------------------------------------------------------------------
  * Make_Run --
@@ -516,25 +533,15 @@ add_targets_to_make(Lst todo)
  * calling on MakeStartJobs to keep the job table as full as
  * possible.
  *
- * Results:
- * true if work was done. false otherwise.
- *
  * Side Effects:
  * The must_make field of all nodes involved in the creation of the given
  * targets is set to 1. The to_build list is set to contain all the
  * 'leaves' of these subgraphs.
  *-----------------------------------------------------------------------
  */
-bool
-Make_Run(Lst targs) /* the initial list of targets */
+void
+Make_Run(Lst targs, bool *has_errors, bool *out_of_date)
 {
- bool problem; /* errors occurred */
-
- /* wild guess at initial sizes */
- Array_Init(&to_build, 500);
- Array_Init(&examine, 150);
- Array_Init(&heldBack, 100);
- ohash_init(&targets, 10, &gnode_info);
  if (DEBUG(PARALLEL))
  random_setup();
 
@@ -545,7 +552,8 @@ Make_Run(Lst targs) /* the initial list of targets */
  * the next loop... (we won't actually start any, of course,
  * this is just to see if any of the targets was out of date)
  */
- return MakeStartJobs();
+ if (MakeStartJobs())
+ *out_of_date = true;
  } else {
  /*
  * Initialization. At the moment, no jobs are running and until
@@ -572,8 +580,8 @@ Make_Run(Lst targs) /* the initial list of targets */
  (void)MakeStartJobs();
  }
 
- if (!queryFlag)
- problem = Job_Finish();
+ if (errorJobs != NULL)
+ *has_errors = true;
 
  /*
  * Print the final status of each target. E.g. if it wasn't made
@@ -581,13 +589,9 @@ Make_Run(Lst targs) /* the initial list of targets */
  */
  if (targets_contain_cycles()) {
  break_and_print_cycles(targs);
- problem = true;
+ *has_errors = true;
  }
  Lst_Every(targs, MakePrintStatus);
- if (problem)
- Fatal("Errors while building");
-
- return true;
 }
 
 /* round-about detection: assume make is bug-free, if there are targets
diff --git a/make.h b/make.h
index 91839fb..cb8e09e 100644
--- a/make.h
+++ b/make.h
@@ -41,8 +41,9 @@
  */
 
 extern void Make_Update(GNode *);
-extern bool Make_Run(Lst);
+extern void Make_Run(Lst, bool *, bool *);
+extern void Make_Init(void);
 extern long random_delay;
-extern bool no_jobs_left(void);
+extern bool nothing_left_to_build(void);
 
 #endif /* _MAKE_H_ */
diff --git a/parse.c b/parse.c
index 4c90d18..dfc2abc 100644
--- a/parse.c
+++ b/parse.c
@@ -181,40 +181,40 @@ static struct {
  const char *keyword;
  size_t sz;
  uint32_t hv;
- unsigned int type;
+ unsigned int special;
  unsigned int special_op;
 } specials[] = {
-    { P(NODE_EXEC), SPECIAL_EXEC | SPECIAL_TARGETSOURCE, OP_EXEC, },
-    { P(NODE_IGNORE), SPECIAL_IGNORE | SPECIAL_TARGETSOURCE, OP_IGNORE, },
-    { P(NODE_INCLUDES), SPECIAL_NOTHING | SPECIAL_TARGET, 0, },
-    { P(NODE_INVISIBLE),SPECIAL_INVISIBLE | SPECIAL_TARGETSOURCE,OP_INVISIBLE, },
-    { P(NODE_JOIN), SPECIAL_JOIN | SPECIAL_TARGETSOURCE, OP_JOIN, },
-    { P(NODE_LIBS), SPECIAL_NOTHING | SPECIAL_TARGET, 0, },
-    { P(NODE_MADE), SPECIAL_MADE | SPECIAL_TARGETSOURCE, OP_MADE, },
-    { P(NODE_MAIN), SPECIAL_MAIN | SPECIAL_TARGET, 0, },
-    { P(NODE_MAKE), SPECIAL_MAKE | SPECIAL_TARGETSOURCE, OP_MAKE, },
-    { P(NODE_MAKEFLAGS), SPECIAL_MFLAGS | SPECIAL_TARGET, 0, },
-    { P(NODE_MFLAGS), SPECIAL_MFLAGS | SPECIAL_TARGET, 0, },
-    { P(NODE_NOTMAIN), SPECIAL_NOTMAIN | SPECIAL_TARGETSOURCE, OP_NOTMAIN, },
-    { P(NODE_NOTPARALLEL),SPECIAL_NOTPARALLEL | SPECIAL_TARGET, 0, },
-    { P(NODE_NO_PARALLEL),SPECIAL_NOTPARALLEL | SPECIAL_TARGET, 0, },
-    { P(NODE_NULL), SPECIAL_NOTHING | SPECIAL_TARGET, 0, },
-    { P(NODE_OPTIONAL), SPECIAL_OPTIONAL | SPECIAL_TARGETSOURCE,OP_OPTIONAL, },
-    { P(NODE_ORDER), SPECIAL_ORDER | SPECIAL_TARGET, 0, },
-    { P(NODE_PARALLEL), SPECIAL_PARALLEL | SPECIAL_TARGET, 0, },
-    { P(NODE_PATH), SPECIAL_PATH | SPECIAL_TARGET, 0, },
-    { P(NODE_PHONY), SPECIAL_PHONY | SPECIAL_TARGETSOURCE, OP_PHONY, },
-    { P(NODE_PRECIOUS), SPECIAL_PRECIOUS | SPECIAL_TARGETSOURCE,OP_PRECIOUS, },
-    { P(NODE_RECURSIVE),SPECIAL_MAKE | SPECIAL_TARGETSOURCE, OP_MAKE, },
-    { P(NODE_SILENT), SPECIAL_SILENT | SPECIAL_TARGETSOURCE, OP_SILENT, },
-    { P(NODE_SINGLESHELL),SPECIAL_NOTHING | SPECIAL_TARGET, 0, },
-    { P(NODE_SUFFIXES), SPECIAL_SUFFIXES | SPECIAL_TARGET, 0, },
-    { P(NODE_USE), SPECIAL_USE | SPECIAL_TARGETSOURCE, OP_USE, },
-    { P(NODE_WAIT), SPECIAL_WAIT | SPECIAL_TARGETSOURCE, 0 },
-    { P(NODE_CHEAP), SPECIAL_CHEAP | SPECIAL_TARGETSOURCE, OP_CHEAP, },
-    { P(NODE_EXPENSIVE),SPECIAL_EXPENSIVE | SPECIAL_TARGETSOURCE,OP_EXPENSIVE, },
-    { P(NODE_POSIX), SPECIAL_NOTHING | SPECIAL_TARGET, 0 },
-    { P(NODE_SCCS_GET), SPECIAL_NOTHING | SPECIAL_TARGET, 0 },
+    { P(NODE_EXEC), SPECIAL_EXEC, OP_EXEC },
+    { P(NODE_IGNORE), SPECIAL_IGNORE, OP_IGNORE },
+    { P(NODE_INCLUDES), SPECIAL_NOTHING, 0 },
+    { P(NODE_INVISIBLE), SPECIAL_INVISIBLE, OP_INVISIBLE },
+    { P(NODE_JOIN), SPECIAL_JOIN, OP_JOIN },
+    { P(NODE_LIBS), SPECIAL_NOTHING, 0 },
+    { P(NODE_MADE), SPECIAL_MADE, OP_MADE },
+    { P(NODE_MAIN), SPECIAL_MAIN, 0 },
+    { P(NODE_MAKE), SPECIAL_MAKE, OP_MAKE },
+    { P(NODE_MAKEFLAGS), SPECIAL_MFLAGS, 0 },
+    { P(NODE_MFLAGS), SPECIAL_MFLAGS, 0 },
+    { P(NODE_NOTMAIN), SPECIAL_NOTMAIN, OP_NOTMAIN },
+    { P(NODE_NOTPARALLEL), SPECIAL_NOTPARALLEL, 0 },
+    { P(NODE_NO_PARALLEL), SPECIAL_NOTPARALLEL, 0 },
+    { P(NODE_NULL), SPECIAL_NOTHING, 0 },
+    { P(NODE_OPTIONAL), SPECIAL_OPTIONAL, OP_OPTIONAL },
+    { P(NODE_ORDER), SPECIAL_ORDER, 0 },
+    { P(NODE_PARALLEL), SPECIAL_PARALLEL, 0 },
+    { P(NODE_PATH), SPECIAL_PATH, 0 },
+    { P(NODE_PHONY), SPECIAL_PHONY, OP_PHONY },
+    { P(NODE_PRECIOUS), SPECIAL_PRECIOUS, OP_PRECIOUS },
+    { P(NODE_RECURSIVE), SPECIAL_MAKE, OP_MAKE },
+    { P(NODE_SILENT), SPECIAL_SILENT, OP_SILENT },
+    { P(NODE_SINGLESHELL), SPECIAL_NOTHING, 0 },
+    { P(NODE_SUFFIXES), SPECIAL_SUFFIXES, 0 },
+    { P(NODE_USE), SPECIAL_USE, OP_USE },
+    { P(NODE_WAIT), SPECIAL_WAIT, 0 },
+    { P(NODE_CHEAP), SPECIAL_CHEAP, OP_CHEAP },
+    { P(NODE_EXPENSIVE), SPECIAL_EXPENSIVE, OP_EXPENSIVE },
+    { P(NODE_POSIX), SPECIAL_NOTHING, 0 },
+    { P(NODE_SCCS_GET), SPECIAL_NOTHING, 0 },
 };
 
 #undef P
@@ -225,10 +225,9 @@ create_special_nodes()
  unsigned int i;
 
  for (i = 0; i < sizeof(specials)/sizeof(specials[0]); i++) {
- GNode *gn = Targ_FindNodeh(specials[i].keyword,
-    specials[i].sz, specials[i].hv, TARG_CREATE);
- gn->special = specials[i].type;
- gn->special_op = specials[i].special_op;
+ (void)Targ_mk_special_node(specials[i].keyword,
+    specials[i].sz, specials[i].hv,
+    OP_ZERO, specials[i].special, specials[i].special_op);
  }
 }
 
@@ -419,15 +418,13 @@ ParseDoSrc(
     const char *esrc)
 {
  GNode *gn = Targ_FindNodei(src, esrc, TARG_CREATE);
- if ((gn->special & SPECIAL_SOURCE) != 0) {
- if (gn->special_op) {
- Array_ForEach(targets, ParseDoSpecial, gn->special_op);
- return;
- } else {
- assert((gn->special & SPECIAL_MASK) == SPECIAL_WAIT);
- waiting++;
- return;
- }
+ if (gn->special_op) {
+ Array_ForEach(targets, ParseDoSpecial, gn->special_op);
+ return;
+ }
+ if (gn->special == SPECIAL_WAIT) {
+ waiting++;
+ return;
  }
 
  switch (specType) {
@@ -705,10 +702,10 @@ handle_special_targets(Lst paths)
 
  for (i = 0; i < gtargets.n; i++) {
  type = gtargets.a[i]->special;
- if ((type & SPECIAL_MASK) == SPECIAL_PATH) {
+ if (type == SPECIAL_PATH) {
  seen_path++;
  Lst_AtEnd(paths, find_suffix_path(gtargets.a[i]));
- } else if ((type & SPECIAL_TARGET) != 0)
+ } else if (type != 0)
  seen_special++;
  else
  seen_normal++;
@@ -734,7 +731,7 @@ handle_special_targets(Lst paths)
  dump_targets();
  return 0;
  } else if (seen_special == 1) {
- specType = gtargets.a[0]->special & SPECIAL_MASK;
+ specType = gtargets.a[0]->special;
  switch (specType) {
  case SPECIAL_MAIN:
  if (!Lst_IsEmpty(create)) {
@@ -742,13 +739,8 @@ handle_special_targets(Lst paths)
  }
  break;
  case SPECIAL_NOTPARALLEL:
- {
- extern int  maxJobs;
-
- maxJobs = 1;
- compatMake = 1;
+ set_notparallel();
  break;
- }
  case SPECIAL_ORDER:
  predecessor = NULL;
  break;
@@ -838,6 +830,7 @@ ParseDoDependency(const char *line) /* the line to parse */
  Array_Reset(&gsources);
 
  cp = parse_do_targets(&paths, &tOp, line);
+ assert(specType == SPECIAL_PATH || Lst_IsEmpty(&paths));
  if (cp == NULL || specType == SPECIAL_ERROR) {
  /* invalidate targets for further processing */
  Array_Reset(&gtargets);
@@ -856,19 +849,15 @@ ParseDoDependency(const char *line) /* the line to parse */
 
  line = cp;
 
- /*
- * Several special targets take different actions if present with no
- * sources:
- * a .SUFFIXES line with no sources clears out all old suffixes
- * a .PRECIOUS line makes all targets precious
- * a .IGNORE line ignores errors for all targets
- * a .SILENT line creates silence when making all targets
- * a .PATH removes all directories from the search path(s).
- */
+ /* Several special targets have specific semantics with no source:
+ * .SUFFIXES clears out all old suffixes
+ * .PRECIOUS/.IGNORE/.SILENT
+ * apply to all target
+ * .PATH clears out all search paths.  */
  if (!*line) {
  switch (specType) {
  case SPECIAL_SUFFIXES:
- Suff_ClearSuffixes();
+ Suff_DisableAllSuffixes();
  break;
  case SPECIAL_PRECIOUS:
  allPrecious = true;
@@ -886,42 +875,26 @@ ParseDoDependency(const char *line) /* the line to parse */
  break;
  }
  } else if (specType == SPECIAL_MFLAGS) {
- /* Call on functions in main.c to deal with these arguments */
  Main_ParseArgLine(line);
  return;
  } else if (specType == SPECIAL_NOTPARALLEL) {
  return;
  }
 
- /*
- * NOW GO FOR THE SOURCES
- */
+ /* NOW GO FOR THE SOURCES */
  if (specType == SPECIAL_SUFFIXES || specType == SPECIAL_PATH ||
     specType == SPECIAL_NOTHING) {
  while (*line) {
-    /*
-     * If the target was one that doesn't take files as its
-     * sources but takes something like suffixes, we take each
-     * space-separated word on the line as a something and deal
-     * with it accordingly.
-     *
-     * If the target was .SUFFIXES, we take each source as a
-     * suffix and add it to the list of suffixes maintained by
-     * the Suff module.
-     *
-     * If the target was a .PATH, we add the source as a
-     * directory to search on the search path.
+    /* Some special targets take a list of space-separated
+     * words.  For each word,
      *
-     * If it was .INCLUDES, the source is taken to be the
-     * suffix of files which will be #included and whose search
-     * path should be present in the .INCLUDES variable.
+     * if .SUFFIXES, add it to the list of suffixes maintained
+     * by suff.c.
      *
-     * If it was .LIBS, the source is taken to be the suffix of
-     * files which are considered libraries and whose search
-     * path should be present in the .LIBS variable.
+     * if .PATHS, add it as a directory on the main search path.
      *
-     * If it was .NULL, the source is the suffix to use when a
-     * file has no valid suffix.
+     * if .LIBS/.INCLUDE/.NULL... this has been deprecated,
+     * ignore
      */
     while (*cp && !ISSPACE(*cp))
     cp++;
@@ -937,6 +910,7 @@ ParseDoDependency(const char *line) /* the line to parse */
      ln = Lst_Adv(ln))
     Dir_AddDiri(Lst_Datum(ln), line, cp);
     break;
+    Lst_Destroy(&paths, NOFREE);
     }
     default:
     break;
@@ -947,7 +921,6 @@ ParseDoDependency(const char *line) /* the line to parse */
  cp++;
     line = cp;
  }
- Lst_Destroy(&paths, NOFREE);
  } else {
  while (*line) {
  /*
@@ -1642,12 +1615,12 @@ Parse_File(const char *filename, FILE *stream)
  bool expectingCommands = false;
  bool commands_seen = false;
 
- /* somewhat permanent spaces to shave time */
- BUFFER buf;
- BUFFER copy;
+ /* permanent spaces to shave time */
+ static BUFFER buf;
+ static BUFFER copy;
 
- Buf_Init(&buf, MAKE_BSIZE);
- Buf_Init(&copy, MAKE_BSIZE);
+ Buf_Reinit(&buf, MAKE_BSIZE);
+ Buf_Reinit(&copy, MAKE_BSIZE);
 
  Parse_FromFile(filename, stream);
  do {
@@ -1687,8 +1660,6 @@ Parse_File(const char *filename, FILE *stream)
  Cond_End();
 
  Parse_ReportErrors();
- Buf_Destroy(&buf);
- Buf_Destroy(&copy);
 }
 
 void
diff --git a/suff.c b/suff.c
index 94f0160..e03b745 100644
--- a/suff.c
+++ b/suff.c
@@ -39,28 +39,9 @@
  * suff.c --
  * Functions to maintain suffix lists and find implicit dependents
  * using suffix transformation rules
- *
- * Interface:
- * Suff_Init Initialize all things to do with suffixes.
- *
- * Suff_ClearSuffixes Clear out all the suffixes.
- *
- * Suff_AddSuffix Add the passed string as another known suffix.
- *
- * Suff_ParseAsTransform Line might be a suffix line, check it.
- * If it's not, return NULL. Otherwise, add
- * another transformation to the suffix graph.
- * Returns GNode suitable for framing, I mean,
- * tacking commands, attributes, etc. on.
- *
- * Suff_FindDeps Find implicit sources for and the location of
- * a target based on its suffix. Returns the
- * bottom-most node added to the graph or NULL
- * if the target had no implicit sources.
  */
 
 #include <ctype.h>
-#include <signal.h>
 #include <stddef.h>
 #include <stdint.h>
 #include <stdio.h>
@@ -70,9 +51,7 @@
 #include "config.h"
 #include "defines.h"
 #include "dir.h"
-#include "direxpand.h"
 #include "engine.h"
-#include "arch.h"
 #include "suff.h"
 #include "var.h"
 #include "targ.h"
@@ -81,9 +60,9 @@
 #include "lst.h"
 #include "memory.h"
 #include "gnode.h"
-#include "make.h"
 #include "stats.h"
 #include "dump.h"
+#include "expandchildren.h"
 
 /* XXX the suffixes hash is stored using a specific hash function, suitable
  * for looking up suffixes in reverse.
@@ -91,7 +70,7 @@
 static struct ohash suffixes;
 
 /* We remember the longest suffix, so we don't need to look beyond that.  */
-size_t maxLen;
+size_t maxLen = 0U;
 static LIST srclist;
 
 /* Transforms (.c.o) are stored in another hash, independently from suffixes.
@@ -172,7 +151,6 @@ static Suff *new_suffixi(const char *, const char *);
 static void reverse_hash_add_char(uint32_t *, const char *);
 static uint32_t reverse_hashi(const char *, const char **);
 static unsigned int reverse_slot(struct ohash *, const char *, const char **);
-static void clear_suffixes(void);
 static void record_possible_suffix(Suff *, GNode *, char *, Lst, Lst);
 static void record_possible_suffixes(GNode *, Lst, Lst);
 static Suff *find_suffix_as_suffix(Lst, const char *, const char *);
@@ -184,9 +162,6 @@ static bool SuffRemoveSrc(Lst);
 static void SuffAddLevel(Lst, Src *);
 static Src *SuffFindThem(Lst, Lst);
 static Src *SuffFindCmds(Src *, Lst);
-static void SuffExpandChildren(LstNode, GNode *);
-static void SuffExpandVarChildren(LstNode, GNode *, GNode *);
-static void SuffExpandWildChildren(LstNode, GNode *, GNode *);
 static bool SuffApplyTransform(GNode *, GNode *, Suff *, Suff *);
 static void SuffFindDeps(GNode *, Lst);
 static void SuffFindArchiveDeps(GNode *, Lst);
@@ -348,18 +323,11 @@ SuffInsert(Lst l, Suff *s)
  }
 }
 
-/*-
- *-----------------------------------------------------------------------
- * Suff_ClearSuffixes --
- * Nuke the list of suffixes but keep all transformation
- * rules around.
- *
- * Side Effects:
- * Current suffixes are reset
- *-----------------------------------------------------------------------
- */
-static void
-clear_suffixes(void)
+/* Suff_DisableAllSuffixes
+ * mark all current suffixes as inactive, and reset precedence
+ * computation.  */
+void
+Suff_DisableAllSuffixes(void)
 {
  unsigned int i;
  Suff *s;
@@ -372,12 +340,6 @@ clear_suffixes(void)
  maxLen = 0;
 }
 
-void
-Suff_ClearSuffixes(void)
-{
- clear_suffixes();
-}
-
 
 /* okay = parse_transform(str, &src, &targ);
  * try parsing a string as a transformation rule, returns true if
@@ -488,6 +450,18 @@ find_best_suffix(const char *s, const char *e)
  return best;
 }
 
+Lst
+find_best_path(const char *name)
+{
+ Suff *s = find_best_suffix(name, name + strlen(name));
+ if (s != NULL) {
+ if (DEBUG(SUFF))
+ printf("suffix is \"%s\"...", s->name);
+ return &s->searchPath;
+ } else
+ return defaultPath;
+}
+
 /*-
  *-----------------------------------------------------------------------
  * Suff_ParseAsTransform --
@@ -523,7 +497,7 @@ Suff_ParseAsTransform(const char *line, const char *end)
 
  gn->type = OP_TRANSFORM;
  if (s->flags & SUFF_PATH) {
- gn->special = SPECIAL_PATH | SPECIAL_TARGET;
+ gn->special = SPECIAL_PATH;
  gn->suffix = t;
  }
 
@@ -612,7 +586,7 @@ build_suffixes_graph(void)
     gn = ohash_next(&transforms, &i)) {
      if (Lst_IsEmpty(&gn->commands) && Lst_IsEmpty(&gn->children))
  continue;
- if ((gn->special & SPECIAL_MASK) == SPECIAL_PATH)
+ if (gn->special == SPECIAL_PATH)
  continue;
      if (parse_transform(gn->name, &s, &s2)) {
  SuffInsert(&s2->children, s);
@@ -629,11 +603,7 @@ build_suffixes_graph(void)
  *
  * Side Effects:
  * The searchPath field of all the suffixes is extended by the
- * directories in defaultPath. If paths were specified for the
- * ".h" suffix, the directories are stuffed into a global variable
- * called ".INCLUDES" with each directory preceded by a -I. The same
- * is done for the ".a" suffix, except the variable is called
- * ".LIBS" and the flag is -L.
+ * directories in defaultPath.
  *-----------------------------------------------------------------------
  */
 static void
@@ -912,203 +882,6 @@ SuffFindCmds(Src *targ, Lst slst)
  return NULL;
 }
 
-static void
-SuffLinkParent(GNode *cgn, GNode *pgn)
-{
- Lst_AtEnd(&cgn->parents, pgn);
- if (!has_been_built(cgn))
- pgn->children_left++;
- else if ( ! (cgn->type & (OP_EXEC|OP_USE))) {
- if (cgn->built_status == REBUILT)
- pgn->child_rebuilt = true;
- (void)Make_TimeStamp(pgn, cgn);
- }
-}
-
-static void
-SuffExpandVarChildren(LstNode after, GNode *cgn, GNode *pgn)
-{
- GNode *gn; /* New source 8) */
- char *cp; /* Expanded value */
- LIST members;
-
-
- if (DEBUG(SUFF))
- printf("Expanding \"%s\"...", cgn->name);
-
- cp = Var_Subst(cgn->name, &pgn->localvars, true);
- if (cp == NULL) {
- printf("Problem substituting in %s", cgn->name);
- printf("\n");
- return;
- }
-
- Lst_Init(&members);
-
- if (cgn->type & OP_ARCHV) {
- /*
- * Node was an archive(member) target, so we want to call
- * on the Arch module to find the nodes for us, expanding
- * variables in the parent's context.
- */
- const char *sacrifice = (const char *)cp;
-
- (void)Arch_ParseArchive(&sacrifice, &members, &pgn->localvars);
- } else {
- /* Break the result into a vector of strings whose nodes
- * we can find, then add those nodes to the members list.
- * Unfortunately, we can't use brk_string because it
- * doesn't understand about variable specifications with
- * spaces in them...  */
- const char *start, *cp2;
-
- for (start = cp; *start == ' ' || *start == '\t'; start++)
- continue;
- for (cp2 = start; *cp2 != '\0';) {
- if (ISSPACE(*cp2)) {
- /* White-space -- terminate element, find the
- * node, add it, skip any further spaces.  */
- gn = Targ_FindNodei(start, cp2, TARG_CREATE);
- cp2++;
- Lst_AtEnd(&members, gn);
- while (ISSPACE(*cp2))
- cp2++;
- /* Adjust cp2 for increment at start of loop,
- * but set start to first non-space.  */
- start = cp2;
- } else if (*cp2 == '$')
- /* Start of a variable spec -- contact variable
- * module to find the end so we can skip over
- * it.  */
- Var_ParseSkip(&cp2, &pgn->localvars);
- else if (*cp2 == '\\' && cp2[1] != '\0')
- /* Escaped something -- skip over it.  */
- cp2+=2;
- else
- cp2++;
-    }
-
-    if (cp2 != start) {
-    /* Stuff left over -- add it to the list too.  */
-    gn = Targ_FindNodei(start, cp2, TARG_CREATE);
-    Lst_AtEnd(&members, gn);
-    }
- }
- /* Add all elements of the members list to the parent node.  */
- while ((gn = Lst_DeQueue(&members)) != NULL) {
- if (DEBUG(SUFF))
- printf("%s...", gn->name);
- if (Lst_Member(&pgn->children, gn) == NULL) {
- Lst_Append(&pgn->children, after, gn);
- after = Lst_Adv(after);
- SuffLinkParent(gn, pgn);
- }
- }
- /* Free the result.  */
- free(cp);
- if (DEBUG(SUFF))
- printf("\n");
-}
-
-static void
-SuffExpandWildChildren(LstNode after, GNode *cgn, GNode *pgn)
-{
- Suff *s;
- char *cp; /* Expanded value */
-
- LIST exp; /* List of expansions */
- Lst path; /* Search path along which to expand */
-
- if (DEBUG(SUFF))
- printf("Wildcard expanding \"%s\"...", cgn->name);
-
- /* Find a path along which to expand the word.
- *
- * If the word has a known suffix, use that path.
- * If it has no known suffix and we're allowed to use the null
- * suffix, use its path.
- * Else use the default system search path.  */
- s = find_best_suffix(cgn->name, cgn->name + strlen(cgn->name));
-
- if (s != NULL) {
- if (DEBUG(SUFF))
- printf("suffix is \"%s\"...", s->name);
- path = &s->searchPath;
- } else
- /* Use default search path.  */
- path = defaultPath;
-
- /* Expand the word along the chosen path. */
- Lst_Init(&exp);
- Dir_Expand(cgn->name, path, &exp);
-
- /* Fetch next expansion off the list and find its GNode.  */
- while ((cp = Lst_DeQueue(&exp)) != NULL) {
- GNode *gn; /* New source 8) */
- if (DEBUG(SUFF))
- printf("%s...", cp);
- gn = Targ_FindNode(cp, TARG_CREATE);
-
- /* If gn isn't already a child of the parent, make it so and
- * up the parent's count of children to build.  */
- if (Lst_Member(&pgn->children, gn) == NULL) {
- Lst_Append(&pgn->children, after, gn);
- after = Lst_Adv(after);
- SuffLinkParent(gn, pgn);
- }
- }
-
- if (DEBUG(SUFF))
- printf("\n");
-}
-
-/*-
- *-----------------------------------------------------------------------
- * SuffExpandChildren --
- * Expand the names of any children of a given node that contain
- * variable invocations or file wildcards into actual targets.
- *
- * Side Effects:
- * The expanded node is removed from the parent's list of children,
- * and the parent's children to build counter is decremented,
- *      but other nodes may be added.
- *-----------------------------------------------------------------------
- */
-static void
-SuffExpandChildren(LstNode ln, /* LstNode of child, so we can replace it */
-    GNode *pgn)
-{
- GNode *cgn = Lst_Datum(ln);
-
- /* First do variable expansion -- this takes precedence over wildcard
- * expansion. If the result contains wildcards, they'll be gotten to
- * later since the resulting words are tacked on to the end of the
- * children list.  */
- if (strchr(cgn->name, '$') != NULL)
- SuffExpandVarChildren(ln, cgn, pgn);
- else if (Dir_HasWildcards(cgn->name))
- SuffExpandWildChildren(ln, cgn, pgn);
- else
-    /* Third case: nothing to expand.  */
- return;
-
- /* Since the source was expanded, remove it from the list of children to
- * keep it from being processed.  */
- pgn->children_left--;
- Lst_Remove(&pgn->children, ln);
-}
-
-void
-expand_children_from(GNode *parent, LstNode from)
-{
- LstNode np, ln;
-
- for (ln = from; ln != NULL; ln = np) {
- np = Lst_Adv(ln);
- SuffExpandChildren(ln, parent);
- }
-}
-
 /*-
  *-----------------------------------------------------------------------
  * SuffApplyTransform --
@@ -1140,7 +913,7 @@ SuffApplyTransform(
  if (Lst_AddNew(&tGn->children, sGn)) {
  /* Not already linked, so form the proper links between the
  * target and source.  */
- SuffLinkParent(sGn, tGn);
+ LinkParent(sGn, tGn);
  }
 
  if ((sGn->type & OP_OPMASK) == OP_DOUBLEDEP) {
@@ -1154,7 +927,7 @@ SuffApplyTransform(
  if (Lst_AddNew(&tGn->children, gn)) {
  /* Not already linked, so form the proper links
  * between the target and source.  */
- SuffLinkParent(gn, tGn);
+ LinkParent(gn, tGn);
  }
  }
  }
@@ -1247,7 +1020,7 @@ SuffFindArchiveDeps(
 
  /* Create the link between the two nodes right off. */
  if (Lst_AddNew(&gn->children, mem))
- SuffLinkParent(mem, gn);
+ LinkParent(mem, gn);
 
  /* Copy variables from member node to this one.  */
  Var(TARGET_INDEX, gn) = Var(TARGET_INDEX, mem);
@@ -1685,19 +1458,14 @@ Suff_Init(void)
  Static_Lst_Init(&srclist);
  ohash_init(&transforms, 4, &gnode_info);
 
- /*
- * Create null suffix for single-suffix rules (POSIX). The thing doesn't
- * actually go on the suffix list or everyone will think that's its
- * suffix.
- */
+ /* Create null suffix for single-suffix rules (POSIX). The thing doesn't
+ * actually go on the suffix list as it matches everything.  */
  emptySuff = new_suffix("");
- make_suffix_known(emptySuff);
+ emptySuff->flags = SUFF_ACTIVE;
+ emptySuff->order = 0;
  Dir_Concat(&emptySuff->searchPath, defaultPath);
  ohash_init(&suffixes, 4, &suff_info);
- order = 0;
- clear_suffixes();
  special_path_hack();
-
 }
 
 
diff --git a/suff.h b/suff.h
index 2fe7923..79e65d2 100644
--- a/suff.h
+++ b/suff.h
@@ -3,7 +3,7 @@
 /* $OpenBSD: suff.h,v 1.10 2012/12/06 14:30:35 espie Exp $ */
 
 /*
- * Copyright (c) 2001 Marc Espie.
+ * Copyright (c) 2001-2019 Marc Espie.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -27,16 +27,40 @@
  * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-extern void Suff_ClearSuffixes(void);
+extern void Suff_Init(void);
+
+/* Suff_DisableAllSuffixes():
+ * disable current suffixes and the corresponding rules.
+ * They may be re-activated by adding a suffix anew.  */
+extern void Suff_DisableAllSuffixes(void);
+/* gn = Suff_ParseAsTransform(line, eline):
+ * Try parsing a [line,eline[ as a suffix transformation
+ * (.a.b or .a). If successful, returns a gn we can add
+ * commands to (this is actually a transform kept on a
+ * separate hash from normal targets).  Otherwise returns NULL. */
 extern GNode *Suff_ParseAsTransform(const char *, const char *);
+/* Suff_AddSuffixi(name, ename):
+ * add the passed string interval [name,ename[ as a known
+ * suffix. */
 extern void Suff_AddSuffixi(const char *, const char *);
-extern void Suff_FindDeps(GNode *);
-extern void Suff_Init(void);
+/* process_suffixes_after_makefile_is_read():
+ * finish setting up the transformation graph for Suff_FindDep
+ * and the .PATH.sfx paths get the default path appended for
+ * find_suffix_path().  */
 extern void process_suffixes_after_makefile_is_read(void);
+/* Suff_FindDeps(gn):
+ * find implicit dependencies for gn and fill out corresponding
+ * fields. */
+extern void Suff_FindDeps(GNode *);
+/* l = find_suffix_path(gn):
+ * returns the path associated with a gn, either because of its
+ * suffix, or the default path.  */
 extern Lst find_suffix_path(GNode *);
+/* Suff_PrintAll():
+ * displays all suffix information. */
 extern void Suff_PrintAll(void);
-extern void expand_children_from(GNode *, LstNode);
-#define expand_all_children(gn) \
-    expand_children_from(gn, Lst_First(&(gn)->children))
-
+/* path = find_best_path(name):
+ * find the best path for the name, according to known suffixes.
+ */
+extern Lst find_best_path(const char *name);
 #endif
diff --git a/targ.c b/targ.c
index b2d2489..bba0192 100644
--- a/targ.c
+++ b/targ.c
@@ -122,8 +122,11 @@ struct ohash_info gnode_info = {
  offsetof(GNode, name), NULL, hash_calloc, hash_free, element_alloc
 };
 
-#define Targ_FindConstantNode(n, f) Targ_FindNodeh(n, sizeof(n), K_##n, f)
+static GNode *Targ_mk_node(const char *, const char *, unsigned int,
+    unsigned char, unsigned int);
 
+#define Targ_mk_constant(n, type) \
+    Targ_mk_special_node(n, sizeof(n), K_##n, type, SPECIAL_NONE, 0)
 
 GNode *begin_node, *end_node, *interrupt_node, *DEFAULT;
 
@@ -132,27 +135,28 @@ Targ_Init(void)
 {
  /* A small make file already creates 200 targets.  */
  ohash_init(&targets, 10, &gnode_info);
- begin_node = Targ_FindConstantNode(NODE_BEGIN, TARG_CREATE);
- begin_node->type |= OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT;
- end_node = Targ_FindConstantNode(NODE_END, TARG_CREATE);
- end_node->type |= OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT;
- interrupt_node = Targ_FindConstantNode(NODE_INTERRUPT, TARG_CREATE);
- interrupt_node->type |= OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT;
- DEFAULT = Targ_FindConstantNode(NODE_DEFAULT, TARG_CREATE);
- DEFAULT->type |= OP_DUMMY | OP_NOTMAIN| OP_TRANSFORM | OP_NODEFAULT;
+ begin_node = Targ_mk_constant(NODE_BEGIN,
+    OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT);
+ end_node = Targ_mk_constant(NODE_END,
+    OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT);
+ interrupt_node = Targ_mk_constant(NODE_INTERRUPT,
+    OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT);
+ DEFAULT = Targ_mk_constant(NODE_DEFAULT,
+    OP_DUMMY | OP_NOTMAIN| OP_TRANSFORM | OP_NODEFAULT);
 
 }
 
-GNode *
-Targ_NewGNi(const char *name, const char *ename)
+static GNode *
+Targ_mk_node(const char *name, const char *ename,
+    unsigned int type, unsigned char special, unsigned int special_op)
 {
  GNode *gn;
 
  gn = ohash_create_entry(&gnode_info, name, &ename);
  gn->path = NULL;
- gn->type = OP_ZERO;
- gn->special = SPECIAL_NONE;
- gn->special_op = 0;
+ gn->type = type;
+ gn->special = special;
+ gn->special_op = special_op;
  gn->children_left = 0;
  gn->must_make = false;
  gn->built_status = UNKNOWN;
@@ -183,20 +187,20 @@ Targ_NewGNi(const char *name, const char *ename)
 }
 
 GNode *
-Targ_FindNodei(const char *name, const char *ename, int flags)
+Targ_NewGNi(const char *name, const char *ename)
 {
- uint32_t hv;
-
- hv = ohash_interval(name, &ename);
- return Targ_FindNodeih(name, ename, hv, flags);
+ return Targ_mk_node(name, ename, OP_ZERO, SPECIAL_NONE, 0);
 }
 
 GNode *
-Targ_FindNodeih(const char *name, const char *ename, uint32_t hv, int flags)
+Targ_FindNodei(const char *name, const char *ename, int flags)
 {
+ uint32_t hv;
  GNode *gn;
  unsigned int slot;
 
+ hv = ohash_interval(name, &ename);
+
  slot = ohash_lookup_interval(&targets, name, ename, hv);
 
  gn = ohash_find(&targets, slot);
@@ -209,6 +213,24 @@ Targ_FindNodeih(const char *name, const char *ename, uint32_t hv, int flags)
  return gn;
 }
 
+GNode *
+Targ_mk_special_node(const char *name, size_t n, uint32_t hv,
+    unsigned int type, unsigned char special, unsigned int special_op)
+{
+ GNode *gn;
+ unsigned int slot;
+ const char *ename = name + n - 1;
+
+ slot = ohash_lookup_interval(&targets, name, ename, hv);
+
+ assert(ohash_find(&targets, slot) == NULL);
+
+ gn = Targ_mk_node(name, ename, type, special, special_op);
+ ohash_insert(&targets, slot, gn);
+
+ return gn;
+}
+
 void
 Targ_FindList(Lst nodes, Lst names)
 {
@@ -255,6 +277,12 @@ Targ_Precious(GNode *gn)
  return false;
 }
 
+bool
+node_is_real(GNode *gn)
+{
+ return (gn->type & OP_DUMMY) == 0;
+}
+
 void
 Targ_PrintCmd(void *p)
 {
@@ -321,9 +349,3 @@ targets_hash()
 {
  return &targets;
 }
-
-GNode *
-Targ_FindNodeh(const char *name, size_t n, uint32_t hv, int flags)
-{
- return Targ_FindNodeih(name, name + n - 1, hv, flags);
-}
diff --git a/targ.h b/targ.h
index 78517e3..69b527c 100644
--- a/targ.h
+++ b/targ.h
@@ -46,17 +46,10 @@ extern GNode *Targ_FindNodei(const char *, const char *, int);
 
 
 
-/* set of helpers for constant nodes */
-extern GNode *Targ_FindNodeih(const char *, const char *, uint32_t, int);
+/* helper for constant nodes */
+extern GNode *Targ_mk_special_node(const char *, size_t, uint32_t,
+    unsigned int, unsigned char, unsigned int);
 
-__only_inline GNode *
-Targ_FindNodeh(const char *, size_t, uint32_t, int);
-
-__only_inline GNode *
-Targ_FindNodeh(const char *name, size_t n, uint32_t hv, int flags)
-{
- return Targ_FindNodeih(name, name + n - 1, hv, flags);
-}
 extern void Targ_FindList(Lst, Lst);
 extern bool Targ_Ignore(GNode *);
 extern bool Targ_Silent(GNode *);
@@ -64,6 +57,7 @@ extern bool Targ_Precious(GNode *);
 extern void Targ_PrintCmd(void *);
 extern void Targ_PrintType(int);
 extern void Targ_PrintGraph(int);
+extern bool node_is_real(GNode *);
 
 extern GNode *begin_node, *end_node, *interrupt_node, *DEFAULT;
 struct ohash_info;

Reply | Threaded
Open this post in threaded view
|

Re: MAKE: redux patch

Marc Espie-2
On Thu, Jan 09, 2020 at 01:09:59PM +0100, Marc Espie wrote:

> So my development branch is getting a bit too far
> ahead compared to what's committed.
>
> Here's a big diff to test.  Comments as to what's going on
> and the changes this contains:
>
> - buffer changes: add support for "permanent static buffers"
> that don't get reinit'd all the time (that's a mostly trivial
> change)
>
> - parser change: SPECIAL_TARGET and SPECIAL_SOURCE are not
> really needed (figured out in netbsd)
> - (related): create the special nodes once in a simpler way
>
> - a bit of reorg: take the code for expand_children out
> of suff.c proper, as this file is always fairly unwieldy.
> This only requires Link_Parent, and find_best_path, which
> are suff-independent.
>
> - engine change: stop mixing parallel and compat so much.
> In particular, don't run Make_Update when in compat mode (which
> would happen because handle_all_signals goes into the main loop)
>
> - (related): abstract the engine running into operations in
> 'enginechoice.c' so that it's perfectly clear what part is compatMake
> and what part is not.   Have it report out_of_date and errors in
> a simple way.  Don't mix up .BEGIN/.END handling.
>
> - (related): job.c determines whether to use the expensive heuristics
> or not depending on how many jobs we run.
>
> - jobrunner cleanup: we have no need for maxjobs, it's enough to move
> stuff around from available into running or error... do the move to
> error later so that everything is in the same location.
>
> - with the above changes: the special case of running_one_job is no
> longer needed at all.
>
> - feature coming from bmake: treat .BEGIN/.END more as normal targets.
> More specifically, just invoke the engine on them if they exist.
> This makes it possible to have
> .BEGIN: somethingelse
> which does make a lot of sense, actually.
>
> I would really need this to go in so that I can keep pushing forward.
>
> I think I might take out the "old" parallel engine (which is somewhat
> broken, and frankly, I don't get how it can work) because modifying
> compat to actually queue stuff and build it  is a distinct possibility
> now.  e.g., having an actual parallel engine that works.

Bleh, I forgot to synch two patches I already committed. Here's a patch
that applies cleanly.

diff --git a/Makefile b/Makefile
index 0cd84fc..90747de 100644
--- a/Makefile
+++ b/Makefile
@@ -16,8 +16,8 @@ CFLAGS+=${CDEFS}
 HOSTCFLAGS+=${CDEFS}
 
 SRCS= arch.c buf.c cmd_exec.c compat.c cond.c dir.c direxpand.c dump.c \
- engine.c \
- error.c for.c init.c job.c lowparse.c main.c make.c memory.c parse.c \
+ engine.c enginechoice.c error.c expandchildren.c \
+ for.c init.c job.c lowparse.c main.c make.c memory.c parse.c \
  parsevar.c str.c stats.c suff.c targ.c targequiv.c timestamp.c \
  var.c varmodifiers.c varname.c
 
diff --git a/arch.c b/arch.c
index 85e8e7e..02c18b5 100644
--- a/arch.c
+++ b/arch.c
@@ -195,11 +195,10 @@ bool
 Arch_ParseArchive(const char **line, Lst nodes, SymTable *ctxt)
 {
  bool result;
- BUFFER expand;
+ static BUFFER expand;
 
- Buf_Init(&expand, MAKE_BSIZE);
+ Buf_Reinit(&expand, MAKE_BSIZE);
  result = parse_archive(&expand, line, nodes, ctxt);
- Buf_Destroy(&expand);
  return result;
 }
 
diff --git a/buf.c b/buf.c
index 74cbfe8..931bd7e 100644
--- a/buf.c
+++ b/buf.c
@@ -153,6 +153,15 @@ Buf_printf(Buffer bp, const char *fmt, ...)
  bp->inPtr += n;
 }
 
+void
+Buf_Reinit(Buffer bp, size_t size)
+{
+ if (bp->buffer == NULL)
+ Buf_Init(bp, size);
+ else
+ Buf_Reset(bp);
+}
+
 void
 Buf_Init(Buffer bp, size_t size)
 {
diff --git a/buf.h b/buf.h
index 1b56b27..20ea56a 100644
--- a/buf.h
+++ b/buf.h
@@ -106,6 +106,9 @@ extern void Buf_AddChars(Buffer, size_t, const char *);
  * Initializes a buffer, to hold approximately init chars.
  * Set init to 0 if you have no idea.  */
 extern void Buf_Init(Buffer, size_t);
+/* Buf_Reinit(buf, init);
+ * Initializes/reset a static buffer */
+extern void Buf_Reinit(Buffer, size_t);
 /* Buf_Destroy(buf);
  * Nukes a buffer and all its resources. */
 #define Buf_Destroy(bp) ((void)free((bp)->buffer))
diff --git a/compat.c b/compat.c
index 7ecb75f..fd78d78 100644
--- a/compat.c
+++ b/compat.c
@@ -193,14 +193,12 @@ CompatMake(void *gnp, /* The node to make */
  /* copy over what we just did */
  gn->built_status = sib->built_status;
 
- if (gn->built_status != ERROR) {
- /* If the node was built successfully, mark it so,
+ if (gn->built_status == REBUILT) {
+ /* If the node was built successfully,
  * update its modification time and timestamp all
  * its parents.
  * This is to keep its state from affecting that of
  * its parent.  */
- gn->built_status = REBUILT;
- sib->built_status = REBUILT;
  /* This is what Make does and it's actually a good
  * thing, as it allows rules like
  *
@@ -266,12 +264,20 @@ CompatMake(void *gnp, /* The node to make */
  }
 }
 
-bool
-Compat_Run(Lst targs) /* List of target nodes to re-create */
+void
+Compat_Init()
+{
+}
+
+void
+Compat_Update(GNode *gn)
+{
+}
+
+void
+Compat_Run(Lst targs, bool *has_errors, bool *out_of_date)
 {
  GNode  *gn = NULL; /* Current root target */
- int  errors;   /* Number of targets not built due to errors */
- bool out_of_date = false;
 
  /* For each entry in the list of targets to create, call CompatMake on
  * it to create the thing. CompatMake will leave the 'built_status'
@@ -283,7 +289,6 @@ Compat_Run(Lst targs) /* List of target nodes to re-create */
  *    ABORTED    gn was not built because one of its
  *                          dependencies could not be built due
  *          to errors.  */
- errors = 0;
  while ((gn = Lst_DeQueue(targs)) != NULL) {
  CompatMake(gn, NULL);
 
@@ -292,15 +297,10 @@ Compat_Run(Lst targs) /* List of target nodes to re-create */
  else if (gn->built_status == ABORTED) {
  printf("`%s' not remade because of errors.\n",
     gn->name);
- out_of_date = true;
- errors++;
+ *out_of_date = true;
+ *has_errors = true;
  } else {
- out_of_date = true;
+ *out_of_date = true;
  }
  }
-
- /* If the user has defined a .END target, run its commands.  */
- if (errors == 0 && !queryFlag)
- run_gnode(end_node);
- return out_of_date;
 }
diff --git a/compat.h b/compat.h
index 2420abd..0aa2de9 100644
--- a/compat.h
+++ b/compat.h
@@ -35,9 +35,11 @@
  *    - friendly variable substitution.
  */
 
-/* out_of_date = Compat_Run(to_create);
+/* Compat_Run(to_create, &has_errors, &out_of_date);
  * Run the actual make engine, to create targets that need to,
- * return true if any target is out of date. */
-extern bool Compat_Run(Lst);
+ * return info about what we did. */
+extern void Compat_Run(Lst, bool *, bool *);
+extern void Compat_Init(void);
+extern void Compat_Update(GNode *);
 
 #endif
diff --git a/dump.c b/dump.c
index 0f79c33..b3820eb 100644
--- a/dump.c
+++ b/dump.c
@@ -104,7 +104,7 @@ TargPrintNode(GNode *gn, bool full)
 {
  if (OP_NOP(gn->type))
  return;
- switch((gn->special & SPECIAL_MASK)) {
+ switch(gn->special) {
  case SPECIAL_SUFFIXES:
  case SPECIAL_PHONY:
  case SPECIAL_ORDER:
diff --git a/engine.c b/engine.c
index 7c786eb..7a9e912 100644
--- a/engine.c
+++ b/engine.c
@@ -603,8 +603,6 @@ run_command(const char *cmd, bool errCheck)
  _exit(1);
 }
 
-static Job myjob;
-
 void
 job_attach_node(Job *job, GNode *node)
 {
@@ -617,7 +615,7 @@ job_attach_node(Job *job, GNode *node)
 }
 
 void
-job_handle_status(Job *job, int status)
+handle_job_status(Job *job, int status)
 {
  bool silent;
  int dying;
@@ -666,19 +664,21 @@ job_handle_status(Job *job, int status)
  printf(" in target '%s'", job->node->name);
  if (job->flags & JOB_ERRCHECK) {
  job->node->built_status = ERROR;
- /* compute expensive status if we really want it */
- if ((job->flags & JOB_SILENT) && job == &myjob)
- determine_expensive_job(job);
  if (!keepgoing) {
  if (!silent)
  printf("\n");
- job->next = errorJobs;
- errorJobs = job;
+ job->flags |= JOB_KEEPERROR;
  /* XXX don't free the command */
  return;
  }
  printf(", line %lu of %s", job->location->lineno,
     job->location->fname);
+ /* Parallel make already determined whether
+ * JOB_IS_EXPENSIVE, perform the computation for
+ * sequential make to figure out whether to display the
+ * command or not.  */
+ if ((job->flags & JOB_SILENT) && sequential)
+ determine_expensive_job(job);
  if ((job->flags & (JOB_SILENT | JOB_IS_EXPENSIVE))
     == JOB_SILENT)
  printf(": %s", job->cmd);
@@ -699,17 +699,12 @@ job_handle_status(Job *job, int status)
 int
 run_gnode(GNode *gn)
 {
+ Job *j;
  if (!gn || (gn->type & OP_DUMMY))
  return NOSUCHNODE;
 
- job_attach_node(&myjob, gn);
- while (myjob.exit_type == JOB_EXIT_OKAY) {
- bool finished = job_run_next(&myjob);
- if (finished)
- break;
- handle_one_job(&myjob);
- }
-
+ Job_Make(gn);
+ loop_handle_running_jobs();
  return gn->built_status;
 }
 
@@ -792,7 +787,7 @@ do_run_command(Job *job, const char *pre)
  * and there's nothing left to do.
  */
  if (random_delay)
- if (!(runningJobs == NULL && no_jobs_left()))
+ if (!(runningJobs == NULL && nothing_left_to_build()))
  usleep(arc4random_uniform(random_delay));
  run_command(cmd, errCheck);
  /*NOTREACHED*/
diff --git a/engine.h b/engine.h
index cb96139..ce7ebd4 100644
--- a/engine.h
+++ b/engine.h
@@ -103,19 +103,20 @@ struct Job_ {
  struct Job_ *next; /* singly linked list */
  pid_t pid; /* Current command process id */
  Location *location;
- int exit_type; /* last child exit or signal */
-#define JOB_EXIT_OKAY 0
-#define JOB_EXIT_BAD 1
-#define JOB_SIGNALED 2
  int code; /* exit status or signal code */
+ unsigned short exit_type; /* last child exit or signal */
+#define JOB_EXIT_OKAY 0
+#define JOB_EXIT_BAD 1
+#define JOB_SIGNALED 2
+ unsigned short flags;
+#define JOB_SILENT 0x001 /* Command was silent */
+#define JOB_IS_EXPENSIVE 0x002
+#define JOB_LOST 0x004 /* sent signal to non-existing pid ? */
+#define JOB_ERRCHECK 0x008 /* command wants errcheck */
+#define JOB_KEEPERROR 0x010 /* should place job on error list */
  LstNode next_cmd; /* Next command to run */
  char *cmd; /* Last command run */
  GNode *node;     /* Target of this job */
- unsigned short flags;
-#define JOB_SILENT 0x001 /* Command was silent */
-#define JOB_IS_EXPENSIVE 0x002
-#define JOB_LOST 0x004 /* sent signal to non-existing pid ? */
-#define JOB_ERRCHECK 0x008 /* command wants errcheck */
 };
 
 /* Continuation-style running commands for the parallel engine */
@@ -131,10 +132,10 @@ extern void job_attach_node(Job *, GNode *);
  */
 extern bool job_run_next(Job *);
 
-/* job_handle_status(job, waitstatus):
+/* handle_job_status(job, waitstatus):
  * process a wait return value corresponding to a job, display
  * messages and set job status accordingly.
  */
-extern void job_handle_status(Job *, int);
+extern void handle_job_status(Job *, int);
 
 #endif
diff --git a/enginechoice.c b/enginechoice.c
new file mode 100644
index 0000000..32ba046
--- /dev/null
+++ b/enginechoice.c
@@ -0,0 +1,32 @@
+#include "config.h"
+#include "defines.h"
+#include "compat.h"
+#include "make.h"
+
+struct engine {
+ void (*run_list)(Lst, bool *, bool *);
+ void (*node_updated)(GNode *);
+ void (*init)(void);
+}
+ compat_engine = { Compat_Run, Compat_Update, Compat_Init },
+ parallel_engine = { Make_Run, Make_Update, Make_Init },
+ *engine;
+
+void
+choose_engine(bool compat)
+{
+ engine = compat ? &compat_engine: &parallel_engine;
+ engine->init();
+}
+
+void
+engine_run_list(Lst l, bool *has_errors, bool *out_of_date)
+{
+ engine->run_list(l, has_errors, out_of_date);
+}
+
+void
+engine_node_updated(GNode *gn)
+{
+ engine->node_updated(gn);
+}
diff --git a/enginechoice.h b/enginechoice.h
new file mode 100644
index 0000000..6441231
--- /dev/null
+++ b/enginechoice.h
@@ -0,0 +1,8 @@
+#ifndef ENGINECHOICE_H
+#define ENGINECHOICE_H
+
+extern void engine_run_list(Lst, bool *, bool *);
+extern void engine_node_updated(GNode *);
+extern void choose_engine(bool);
+
+#endif
diff --git a/expandchildren.c b/expandchildren.c
new file mode 100644
index 0000000..28a1004
--- /dev/null
+++ b/expandchildren.c
@@ -0,0 +1,246 @@
+/* $OpenBSD$ */
+/* $NetBSD: suff.c,v 1.13 1996/11/06 17:59:25 christos Exp $ */
+
+/*
+ * Copyright (c) 1988, 1989, 1990, 1993
+ * The Regents of the University of California.  All rights reserved.
+ * Copyright (c) 1989 by Berkeley Softworks
+ * All rights reserved.
+ *
+ * This code is derived from software contributed to Berkeley by
+ * Adam de Boor.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the University nor the names of its contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+/*-
+ * expandchildren.c --
+ * Dealing with final children expansion before building stuff
+ */
+
+#include <ctype.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include "config.h"
+#include "defines.h"
+#include "direxpand.h"
+#include "engine.h"
+#include "arch.h"
+#include "expandchildren.h"
+#include "var.h"
+#include "targ.h"
+#include "lst.h"
+#include "gnode.h"
+#include "suff.h"
+
+static void ExpandChildren(LstNode, GNode *);
+static void ExpandVarChildren(LstNode, GNode *, GNode *);
+static void ExpandWildChildren(LstNode, GNode *, GNode *);
+
+void
+LinkParent(GNode *cgn, GNode *pgn)
+{
+ Lst_AtEnd(&cgn->parents, pgn);
+ if (!has_been_built(cgn))
+ pgn->children_left++;
+ else if ( ! (cgn->type & (OP_EXEC|OP_USE))) {
+ if (cgn->built_status == REBUILT)
+ pgn->child_rebuilt = true;
+ (void)Make_TimeStamp(pgn, cgn);
+ }
+}
+
+static void
+ExpandVarChildren(LstNode after, GNode *cgn, GNode *pgn)
+{
+ GNode *gn; /* New source 8) */
+ char *cp; /* Expanded value */
+ LIST members;
+
+
+ if (DEBUG(SUFF))
+ printf("Expanding \"%s\"...", cgn->name);
+
+ cp = Var_Subst(cgn->name, &pgn->localvars, true);
+ if (cp == NULL) {
+ printf("Problem substituting in %s", cgn->name);
+ printf("\n");
+ return;
+ }
+
+ Lst_Init(&members);
+
+ if (cgn->type & OP_ARCHV) {
+ /*
+ * Node was an archive(member) target, so we want to call
+ * on the Arch module to find the nodes for us, expanding
+ * variables in the parent's context.
+ */
+ const char *sacrifice = (const char *)cp;
+
+ (void)Arch_ParseArchive(&sacrifice, &members, &pgn->localvars);
+ } else {
+ /* Break the result into a vector of strings whose nodes
+ * we can find, then add those nodes to the members list.
+ * Unfortunately, we can't use brk_string because it
+ * doesn't understand about variable specifications with
+ * spaces in them...  */
+ const char *start, *cp2;
+
+ for (start = cp; *start == ' ' || *start == '\t'; start++)
+ continue;
+ for (cp2 = start; *cp2 != '\0';) {
+ if (ISSPACE(*cp2)) {
+ /* White-space -- terminate element, find the
+ * node, add it, skip any further spaces.  */
+ gn = Targ_FindNodei(start, cp2, TARG_CREATE);
+ cp2++;
+ Lst_AtEnd(&members, gn);
+ while (ISSPACE(*cp2))
+ cp2++;
+ /* Adjust cp2 for increment at start of loop,
+ * but set start to first non-space.  */
+ start = cp2;
+ } else if (*cp2 == '$')
+ /* Start of a variable spec -- contact variable
+ * module to find the end so we can skip over
+ * it.  */
+ Var_ParseSkip(&cp2, &pgn->localvars);
+ else if (*cp2 == '\\' && cp2[1] != '\0')
+ /* Escaped something -- skip over it.  */
+ cp2+=2;
+ else
+ cp2++;
+    }
+
+    if (cp2 != start) {
+    /* Stuff left over -- add it to the list too.  */
+    gn = Targ_FindNodei(start, cp2, TARG_CREATE);
+    Lst_AtEnd(&members, gn);
+    }
+ }
+ /* Add all elements of the members list to the parent node.  */
+ while ((gn = Lst_DeQueue(&members)) != NULL) {
+ if (DEBUG(SUFF))
+ printf("%s...", gn->name);
+ if (Lst_Member(&pgn->children, gn) == NULL) {
+ Lst_Append(&pgn->children, after, gn);
+ after = Lst_Adv(after);
+ LinkParent(gn, pgn);
+ }
+ }
+ /* Free the result.  */
+ free(cp);
+ if (DEBUG(SUFF))
+ printf("\n");
+}
+
+static void
+ExpandWildChildren(LstNode after, GNode *cgn, GNode *pgn)
+{
+ char *cp; /* Expanded value */
+
+ LIST exp; /* List of expansions */
+ Lst path; /* Search path along which to expand */
+
+ if (DEBUG(SUFF))
+ printf("Wildcard expanding \"%s\"...", cgn->name);
+
+ /* Find a path along which to expand the word: if
+ * the word has a known suffix, use the path for that suffix,
+ * otherwise use the default path. */
+ path = find_best_path(cgn->name);
+
+ /* Expand the word along the chosen path. */
+ Lst_Init(&exp);
+ Dir_Expand(cgn->name, path, &exp);
+
+ /* Fetch next expansion off the list and find its GNode.  */
+ while ((cp = Lst_DeQueue(&exp)) != NULL) {
+ GNode *gn; /* New source 8) */
+ if (DEBUG(SUFF))
+ printf("%s...", cp);
+ gn = Targ_FindNode(cp, TARG_CREATE);
+
+ /* If gn isn't already a child of the parent, make it so and
+ * up the parent's count of children to build.  */
+ if (Lst_Member(&pgn->children, gn) == NULL) {
+ Lst_Append(&pgn->children, after, gn);
+ after = Lst_Adv(after);
+ LinkParent(gn, pgn);
+ }
+ }
+
+ if (DEBUG(SUFF))
+ printf("\n");
+}
+
+/*-
+ *-----------------------------------------------------------------------
+ * ExpandChildren --
+ * Expand the names of any children of a given node that contain
+ * variable invocations or file wildcards into actual targets.
+ *
+ * Side Effects:
+ * The expanded node is removed from the parent's list of children,
+ * and the parent's children to build counter is decremented,
+ *      but other nodes may be added.
+ *-----------------------------------------------------------------------
+ */
+static void
+ExpandChildren(LstNode ln, /* LstNode of child, so we can replace it */
+    GNode *pgn)
+{
+ GNode *cgn = Lst_Datum(ln);
+
+ /* First do variable expansion -- this takes precedence over wildcard
+ * expansion. If the result contains wildcards, they'll be gotten to
+ * later since the resulting words are tacked on to the end of the
+ * children list.  */
+ if (strchr(cgn->name, '$') != NULL)
+ ExpandVarChildren(ln, cgn, pgn);
+ else if (Dir_HasWildcards(cgn->name))
+ ExpandWildChildren(ln, cgn, pgn);
+ else
+    /* Third case: nothing to expand.  */
+ return;
+
+ /* Since the source was expanded, remove it from the list of children to
+ * keep it from being processed.  */
+ pgn->children_left--;
+ Lst_Remove(&pgn->children, ln);
+}
+
+void
+expand_children_from(GNode *parent, LstNode from)
+{
+ LstNode np, ln;
+
+ for (ln = from; ln != NULL; ln = np) {
+ np = Lst_Adv(ln);
+ ExpandChildren(ln, parent);
+ }
+}
diff --git a/expandchildren.h b/expandchildren.h
new file mode 100644
index 0000000..2c5212e
--- /dev/null
+++ b/expandchildren.h
@@ -0,0 +1,16 @@
+#ifndef EXPANDCHILDREN_H
+#define EXPANDCHILDREN_H
+/* $OpenBSD: suff.h,v 1.10 2012/12/06 14:30:35 espie Exp $ */
+
+extern void LinkParent(GNode *, GNode *);
+
+/* partial expansion of children. */
+extern void expand_children_from(GNode *, LstNode);
+/* expand_all_children(gn):
+ * figure out all variable/wildcards expansions in gn.
+ * TODO pretty sure this is independent from the main suff module.
+ */
+#define expand_all_children(gn) \
+    expand_children_from(gn, Lst_First(&(gn)->children))
+
+#endif
diff --git a/extern.h b/extern.h
index 89797d6..e461ecb 100644
--- a/extern.h
+++ b/extern.h
@@ -40,7 +40,6 @@
  * from: @(#)nonints.h 8.3 (Berkeley) 3/19/94
  */
 
-extern bool compatMake; /* True if we are make compatible */
 extern bool ignoreErrors; /* True if should ignore all errors */
 extern bool beSilent; /* True if should print no commands */
 extern bool noExecute; /* True if should execute nothing */
diff --git a/gnode.h b/gnode.h
index d014d87..283fead 100644
--- a/gnode.h
+++ b/gnode.h
@@ -65,32 +65,34 @@
  *   to create this target.
  */
 
+/* constants for specials
+ * Most of these values are only handled by parse.c.
+ * In many cases, there is a corresponding OP_* flag
+ */
 #define SPECIAL_NONE 0U
-#define SPECIAL_PATH 21U
-#define SPECIAL_MASK 63U
-#define SPECIAL_TARGET 64U
-#define SPECIAL_SOURCE 128U
-#define SPECIAL_TARGETSOURCE (SPECIAL_TARGET|SPECIAL_SOURCE)
+#define SPECIAL_PATH 62U /* handled by parse.c and suff.c */
 
-#define SPECIAL_EXEC 4U
+#define SPECIAL_EXEC 4U
 #define SPECIAL_IGNORE 5U
-#define SPECIAL_NOTHING 6U
-#define SPECIAL_INVISIBLE 8U
+#define SPECIAL_NOTHING 6U /* this is used for things we
+ * recognize for compatibility but
+ * don't do anything with... */
+#define SPECIAL_INVISIBLE 8U
 #define SPECIAL_JOIN 9U
 #define SPECIAL_MADE 11U
 #define SPECIAL_MAIN 12U
 #define SPECIAL_MAKE 13U
 #define SPECIAL_MFLAGS 14U
-#define SPECIAL_NOTMAIN 15U
-#define SPECIAL_NOTPARALLEL 16U
-#define SPECIAL_OPTIONAL 18U
+#define SPECIAL_NOTMAIN 15U
+#define SPECIAL_NOTPARALLEL 16U
+#define SPECIAL_OPTIONAL 18U
 #define SPECIAL_ORDER 19U
 #define SPECIAL_PARALLEL 20U
 #define SPECIAL_PHONY 22U
 #define SPECIAL_PRECIOUS 23U
 #define SPECIAL_SILENT 25U
 #define SPECIAL_SUFFIXES 27U
-#define SPECIAL_USE 28U
+#define SPECIAL_USE 28U
 #define SPECIAL_WAIT 29U
 #define SPECIAL_NOPATH 30U
 #define SPECIAL_ERROR 31U
diff --git a/job.c b/job.c
index 1846e6a..0a2180f 100644
--- a/job.c
+++ b/job.c
@@ -70,19 +70,11 @@
  *
  * Job_Init Called to initialize this module.
  *
- * Job_Begin execute commands attached to the .BEGIN target
- * if any.
- *
  * can_start_job Return true if we can start job
  *
  * Job_Empty Return true if the job table is completely
  * empty.
  *
- * Job_Finish Perform any final processing which needs doing.
- * This includes the execution of any commands
- * which have been/were attached to the .END
- * target.
- *
  * Job_AbortAll Abort all current jobs. It doesn't
  * handle output or do anything for the jobs,
  * just kills them.
@@ -113,23 +105,24 @@
 #include "lst.h"
 #include "gnode.h"
 #include "memory.h"
-#include "make.h"
 #include "buf.h"
+#include "enginechoice.h"
 
 static int aborting = 0;    /* why is the make aborting? */
 #define ABORT_ERROR 1    /* Because of an error */
 #define ABORT_INTERRUPT 2    /* Because it was interrupted */
 #define ABORT_WAIT 3    /* Waiting for jobs to finish */
 
-static int maxJobs; /* The most children we can run at once */
-static int nJobs; /* Number of jobs already allocated */
 static bool no_new_jobs; /* Mark recursive shit so we shouldn't start
  * something else at the same time
  */
+bool sequential;
 Job *runningJobs; /* Jobs currently running a process */
 Job *errorJobs; /* Jobs in error at end */
+Job *availableJobs; /* Pool of available jobs */
 static Job *heldJobs; /* Jobs not running yet because of expensive */
 static pid_t mypid; /* Used for printing debugging messages */
+static Job *extra_job; /* Needed for .INTERRUPT */
 
 static volatile sig_atomic_t got_fatal;
 
@@ -141,16 +134,11 @@ static sigset_t sigset, emptyset;
 static void handle_fatal_signal(int);
 static void handle_siginfo(void);
 static void postprocess_job(Job *);
-static Job *prepare_job(GNode *);
 static void determine_job_next_step(Job *);
-static void remove_job(Job *);
 static void may_continue_job(Job *);
-static void continue_job(Job *);
 static Job *reap_finished_job(pid_t);
 static bool reap_jobs(void);
-static void may_continue_heldback_jobs();
 
-static void loop_handle_running_jobs(void);
 static bool expensive_job(Job *);
 static bool expensive_command(const char *);
 static void setup_signal(int);
@@ -542,10 +530,15 @@ postprocess_job(Job *job)
  * non-zero status that we shouldn't ignore, we call
  * Make_Update to update the parents. */
  job->node->built_status = REBUILT;
- Make_Update(job->node);
- free(job);
- } else if (job->exit_type != JOB_EXIT_OKAY && keepgoing)
- free(job);
+ engine_node_updated(job->node);
+ }
+ if (job->flags & JOB_KEEPERROR) {
+ job->next = errorJobs;
+ errorJobs = job;
+ } else {
+ job->next = availableJobs;
+ availableJobs = job;
+ }
 
  if (errorJobs != NULL && aborting != ABORT_INTERRUPT)
  aborting = ABORT_ERROR;
@@ -569,10 +562,10 @@ postprocess_job(Job *job)
  * is set, so jobs that would fork new processes are accumulated in the
  * heldJobs list instead.
  *
- * This heuristics is also used on error exit: we display silent commands
- * that failed, unless those ARE expensive commands: expensive commands
- * are likely to not be failing by themselves, but to be the result of
- * a cascade of failures in descendant makes.
+ * XXX This heuristics is also used on error exit: we display silent commands
+ * that failed, unless those ARE expensive commands: expensive commands are
+ * likely to not be failing by themselves, but to be the result of a cascade of
+ * failures in descendant makes.
  */
 void
 determine_expensive_job(Job *job)
@@ -648,35 +641,6 @@ expensive_command(const char *s)
  return false;
 }
 
-static Job *
-prepare_job(GNode *gn)
-{
- /* a new job is prepared unless its commands are bogus (we don't
- * have anything for it), or if we're in touch mode.
- *
- * Note that even in noexec mode, some commands may still run
- * thanks to the +cmd construct.
- */
- if (node_find_valid_commands(gn)) {
- if (touchFlag) {
- Job_Touch(gn);
- return NULL;
- } else {
- Job *job;      
-
- job = emalloc(sizeof(Job));
- if (job == NULL)
- Punt("can't create job: out of memory");
-
- job_attach_node(job, gn);
- return job;
- }
- } else {
- node_failure(gn);
- return NULL;
- }
-}
-
 static void
 may_continue_job(Job *job)
 {
@@ -686,18 +650,29 @@ may_continue_job(Job *job)
     (long)mypid, job->node->name);
  job->next = heldJobs;
  heldJobs = job;
- } else
- continue_job(job);
+ } else {
+ bool finished = job_run_next(job);
+ if (finished)
+ postprocess_job(job);
+ else if (!sequential)
+ determine_expensive_job(job);
+ }
 }
 
 static void
-continue_job(Job *job)
+may_continue_heldback_jobs()
 {
- bool finished = job_run_next(job);
- if (finished)
- remove_job(job);
- else
- determine_expensive_job(job);
+ while (!no_new_jobs) {
+ if (heldJobs != NULL) {
+ Job *job = heldJobs;
+ heldJobs = heldJobs->next;
+ if (DEBUG(EXPENSIVE))
+ fprintf(stderr, "[%ld] cheap -> release %s\n",
+    (long)mypid, job->node->name);
+ may_continue_job(job);
+ } else
+ break;
+ }
 }
 
 /*-
@@ -714,19 +689,17 @@ continue_job(Job *job)
 void
 Job_Make(GNode *gn)
 {
- Job *job;
+ Job *job = availableJobs;      
 
- job = prepare_job(gn);
- if (!job)
- return;
- nJobs++;
+ assert(job != NULL);
+ availableJobs = availableJobs->next;
+ job_attach_node(job, gn);
  may_continue_job(job);
 }
 
 static void
 determine_job_next_step(Job *job)
 {
- bool okay;
  if (job->flags & JOB_IS_EXPENSIVE) {
  no_new_jobs = false;
  if (DEBUG(EXPENSIVE))
@@ -737,34 +710,11 @@ determine_job_next_step(Job *job)
  }
 
  if (job->exit_type != JOB_EXIT_OKAY || job->next_cmd == NULL)
- remove_job(job);
+ postprocess_job(job);
  else
  may_continue_job(job);
 }
 
-static void
-remove_job(Job *job)
-{
- nJobs--;
- postprocess_job(job);
-}
-
-static void
-may_continue_heldback_jobs()
-{
- while (!no_new_jobs) {
- if (heldJobs != NULL) {
- Job *job = heldJobs;
- heldJobs = heldJobs->next;
- if (DEBUG(EXPENSIVE))
- fprintf(stderr, "[%ld] cheap -> release %s\n",
-    (long)mypid, job->node->name);
- continue_job(job);
- } else
- break;
- }
-}
-
 /*
  * job = reap_finished_job(pid):
  * retrieve and remove a job from runningJobs, based on its pid
@@ -806,7 +756,7 @@ reap_jobs(void)
  if (job == NULL) {
  Punt("Child (%ld) not in table?", (long)pid);
  } else {
- job_handle_status(job, status);
+ handle_job_status(job, status);
  determine_job_next_step(job);
  }
  may_continue_heldback_jobs();
@@ -849,26 +799,6 @@ handle_running_jobs(void)
 }
 
 void
-handle_one_job(Job *job)
-{
- int stat;
- int status;
- sigset_t old;
-
- sigprocmask(SIG_BLOCK, &sigset, &old);
- while (1) {
- handle_all_signals();
- stat = waitpid(job->pid, &status, WNOHANG);
- if (stat == job->pid)
- break;
- sigsuspend(&emptyset);
- }
- runningJobs = NULL;
- job_handle_status(job, status);
- sigprocmask(SIG_SETMASK, &old, NULL);
-}
-
-static void
 loop_handle_running_jobs()
 {
  while (runningJobs != NULL)
@@ -876,16 +806,27 @@ loop_handle_running_jobs()
 }
 
 void
-Job_Init(int maxproc)
+Job_Init(int maxJobs)
 {
+ Job *j;
+ int i;
+
  runningJobs = NULL;
  heldJobs = NULL;
  errorJobs = NULL;
- maxJobs = maxproc;
+ availableJobs = NULL;
+ sequential = maxJobs == 1;
+
+ /* we allocate n+1 jobs, since we may need an extra job for
+ * running .INTERRUPT.  */
+ j = ereallocarray(NULL, sizeof(Job), maxJobs+1);
+ for (i = 0; i != maxJobs; i++) {
+ j[i].next = availableJobs;
+ availableJobs = &j[i];
+ }
+ extra_job = &j[maxJobs];
  mypid = getpid();
 
- nJobs = 0;
-
  aborting = 0;
  setup_all_signals();
 }
@@ -893,7 +834,7 @@ Job_Init(int maxproc)
 bool
 can_start_job(void)
 {
- if (aborting || nJobs >= maxJobs)
+ if (aborting || availableJobs == NULL)
  return false;
  else
  return true;
@@ -933,7 +874,8 @@ handle_fatal_signal(int signo)
  if (signo == SIGINT && !touchFlag) {
  if ((interrupt_node->type & OP_DUMMY) == 0) {
  ignoreErrors = false;
-
+ extra_job->next = availableJobs;
+ availableJobs = extra_job;
  Job_Make(interrupt_node);
  }
  }
@@ -950,40 +892,6 @@ handle_fatal_signal(int signo)
  exit(1);
 }
 
-/*
- *-----------------------------------------------------------------------
- * Job_Finish --
- * Do final processing such as the running of the commands
- * attached to the .END target.
- *
- * return true if fatal errors have happened.
- *-----------------------------------------------------------------------
- */
-bool
-Job_Finish(void)
-{
- bool problem = errorJobs != NULL;
-
- if ((end_node->type & OP_DUMMY) == 0) {
- if (problem) {
- Error("Errors reported so .END ignored");
- } else {
- Job_Make(end_node);
- loop_handle_running_jobs();
- }
- }
- return problem;
-}
-
-void
-Job_Begin(void)
-{
- if ((begin_node->type & OP_DUMMY) == 0) {
- Job_Make(begin_node);
- loop_handle_running_jobs();
- }
-}
-
 /*-
  *-----------------------------------------------------------------------
  * Job_Wait --
diff --git a/job.h b/job.h
index e8152d1..5356ee8 100644
--- a/job.h
+++ b/job.h
@@ -65,16 +65,6 @@ extern bool can_start_job(void);
  */
 extern bool Job_Empty(void);
 
-/* errors = Job_Finish();
- * final processing including running .END target if no errors.
- */
-extern bool Job_Finish(void);
-
-/* Job_Begin();
- * similarly, run .BEGIN job at start of job.
- */
-extern void Job_Begin(void);
-
 extern void Job_Wait(void);
 extern void Job_AbortAll(void);
 extern void print_errors(void);
@@ -84,6 +74,10 @@ extern void print_errors(void);
  * or a signal coming in.
  */
 extern void handle_running_jobs(void);
+/* loop_handle_running_jobs();
+ * handle running jobs until they're finished.
+ */
+extern void loop_handle_running_jobs(void);
 
 /* handle_all_signals();
  * if a signal was received, react accordingly.
@@ -93,11 +87,13 @@ extern void handle_running_jobs(void);
 extern void handle_all_signals(void);
 
 extern void determine_expensive_job(Job *);
-extern Job *runningJobs, *errorJobs;
+extern Job *runningJobs, *errorJobs, *availableJobs;
 extern void debug_job_printf(const char *, ...);
 extern void handle_one_job(Job *);
 extern int check_dying_signal(void);
 
 extern const char *basedirectory;
 
+extern bool sequential; /* True if we are running one single-job */
+
 #endif /* _JOB_H_ */
diff --git a/main.c b/main.c
index cb79cb1..ea0947c 100644
--- a/main.c
+++ b/main.c
@@ -57,15 +57,14 @@
 #include "pathnames.h"
 #include "init.h"
 #include "job.h"
-#include "compat.h"
 #include "targ.h"
 #include "suff.h"
 #include "str.h"
 #include "main.h"
 #include "lst.h"
 #include "memory.h"
-#include "make.h"
 #include "dump.h"
+#include "enginechoice.h"
 
 #define MAKEFLAGS ".MAKEFLAGS"
 
@@ -73,12 +72,12 @@ static LIST to_create; /* Targets to be made */
 Lst create = &to_create;
 bool allPrecious; /* .PRECIOUS given on line by itself */
 
-static bool noBuiltins; /* -r flag */
-static LIST makefiles; /* ordered list of makefiles to read */
-static LIST varstoprint; /* list of variables to print */
-int maxJobs; /* -j argument */
-bool compatMake; /* -B argument */
-static bool forceJobs = false;
+static bool noBuiltins; /* -r flag */
+static LIST makefiles; /* ordered list of makefiles to read */
+static LIST varstoprint; /* list of variables to print */
+static int optj; /* -j argument */
+static bool compatMake; /* -B argument */
+static bool forceJobs = false;
 int debug; /* -d flag */
 bool noExecute; /* -n flag */
 bool keepgoing; /* -k flag */
@@ -126,6 +125,12 @@ record_option(int c, const char *arg)
  Var_Append(MAKEFLAGS, arg);
 }
 
+void
+set_notparallel()
+{
+ compatMake = true;
+}
+
 static void
 posixParseOptLetter(int c)
 {
@@ -313,7 +318,7 @@ MainParseArgs(int argc, char **argv)
  const char *errstr;
 
  forceJobs = true;
- maxJobs = strtonum(optarg, 1, INT_MAX, &errstr);
+ optj = strtonum(optarg, 1, INT_MAX, &errstr);
  if (errstr != NULL) {
  fprintf(stderr,
     "make: illegal argument to -j option"
@@ -623,30 +628,25 @@ read_all_make_rules(bool noBuiltins, bool read_depend,
  Parse_End();
 }
 
+static void
+run_node(GNode *gn, bool *has_errors, bool *out_of_date)
+{
+ LIST l;
+
+ Lst_Init(&l);
+ Lst_AtEnd(&l, gn);
+ engine_run_list(&l, has_errors, out_of_date);
+ Lst_Destroy(&l, NOFREE);
+}
 
 int main(int, char **);
-/*-
- * main --
- * The main function, for obvious reasons. Initializes variables
- * and a few modules, then parses the arguments give it in the
- * environment and on the command line. Reads the system makefile
- * followed by either Makefile, makefile or the file given by the
- * -f argument. Sets the .MAKEFLAGS PMake variable based on all the
- * flags it has received by then uses either the Make or the Compat
- * module to create the initial list of targets.
- *
- * Results:
- * If -q was given, exits -1 if anything was out-of-date. Else it exits
- * 0.
- *
- * Side Effects:
- * The program exits when done. Targets are created. etc. etc. etc.
- */
+
 int
 main(int argc, char **argv)
 {
  static LIST targs; /* target nodes to create */
- bool outOfDate = true; /* false if all targets up to date */
+ bool outOfDate = false; /* false if all targets up to date */
+ bool errored = false; /* true if errors occurred */
  char *machine = figure_out_MACHINE();
  char *machine_arch = figure_out_MACHINE_ARCH();
  char *machine_cpu = figure_out_MACHINE_CPU();
@@ -665,6 +665,7 @@ main(int argc, char **argv)
  Static_Lst_Init(&makefiles);
  Static_Lst_Init(&varstoprint);
  Static_Lst_Init(&targs);
+ Static_Lst_Init(&special);
 
  beSilent = false; /* Print commands as executed */
  ignoreErrors = false; /* Pay attention to non-zero returns */
@@ -676,7 +677,7 @@ main(int argc, char **argv)
  touchFlag = false; /* Actually update targets */
  debug = 0; /* No debug verbosity, please. */
 
- maxJobs = DEFMAXJOBS;
+ optj = DEFMAXJOBS;
  compatMake = false; /* No compat mode */
 
 
@@ -757,6 +758,9 @@ main(int argc, char **argv)
 
  read_all_make_rules(noBuiltins, read_depend, &makefiles, &d);
 
+ if (compatMake)
+ optj = 1;
+
  Var_Append("MFLAGS", Var_Value(MAKEFLAGS));
 
  /* Install all the flags into the MAKEFLAGS env variable. */
@@ -796,25 +800,27 @@ main(int argc, char **argv)
  else
  Targ_FindList(&targs, create);
 
- Job_Init(maxJobs);
- /* If the user has defined a .BEGIN target, execute the commands
- * attached to it.  */
- if (!queryFlag)
- Job_Begin();
- if (compatMake)
- /* Compat_Init will take care of creating all the
- * targets as well as initializing the module.  */
- outOfDate = Compat_Run(&targs);
- else {
- /* Traverse the graph, checking on all the targets.  */
- outOfDate = Make_Run(&targs);
- }
+ choose_engine(compatMake);
+ Job_Init(optj);
+ if (!queryFlag && node_is_real(begin_node))
+ run_node(begin_node, &errored, &outOfDate);
+
+ if (!errored)
+ engine_run_list(&targs, &errored, &outOfDate);
+
+ if (!queryFlag && !errored && node_is_real(end_node))
+ run_node(end_node, &errored, &outOfDate);
  }
 
  /* print the graph now it's been processed if the user requested it */
  if (DEBUG(GRAPH2))
  post_mortem();
 
+ /* Note that we only hit this code if -k is used, otherwise we
+ * exited early in case of errors. */
+ if (errored)
+ Fatal("Errors while building");
+
  if (queryFlag && outOfDate)
  return 1;
  else
diff --git a/main.h b/main.h
index 469487e..3f5ce13 100644
--- a/main.h
+++ b/main.h
@@ -38,4 +38,7 @@ extern void Main_ParseArgLine(const char *);
  * .if make(...) statements. */
 extern Lst create;
 
+/* set_notparallel(): used to influence running mode from parse.c */
+extern void set_notparallel(void);
+
 #endif
diff --git a/make.c b/make.c
index 4ffdda6..1eff38e 100644
--- a/make.c
+++ b/make.c
@@ -71,6 +71,7 @@
 #include "suff.h"
 #include "var.h"
 #include "error.h"
+#include "expandchildren.h"
 #include "make.h"
 #include "gnode.h"
 #include "extern.h"
@@ -118,7 +119,7 @@ static bool randomize_queue;
 long random_delay = 0;
 
 bool
-no_jobs_left()
+nothing_left_to_build()
 {
  return Array_IsEmpty(&to_build);
 }
@@ -376,7 +377,13 @@ try_to_make_node(GNode *gn)
  return true;
  /* SIB: this is where commands should get prepared */
  Make_DoAllVar(gn);
- Job_Make(gn);
+ if (node_find_valid_commands(gn)) {
+ if (touchFlag)
+ Job_Touch(gn);
+ else
+ Job_Make(gn);
+ } else
+ node_failure(gn);
  } else {
  if (DEBUG(MAKE))
  printf("up-to-date\n");
@@ -504,6 +511,16 @@ add_targets_to_make(Lst todo)
  randomize_garray(&to_build);
 }
 
+void
+Make_Init()
+{
+ /* wild guess at initial sizes */
+ Array_Init(&to_build, 500);
+ Array_Init(&examine, 150);
+ Array_Init(&heldBack, 100);
+ ohash_init(&targets, 10, &gnode_info);
+}
+
 /*-
  *-----------------------------------------------------------------------
  * Make_Run --
@@ -516,25 +533,15 @@ add_targets_to_make(Lst todo)
  * calling on MakeStartJobs to keep the job table as full as
  * possible.
  *
- * Results:
- * true if work was done. false otherwise.
- *
  * Side Effects:
  * The must_make field of all nodes involved in the creation of the given
  * targets is set to 1. The to_build list is set to contain all the
  * 'leaves' of these subgraphs.
  *-----------------------------------------------------------------------
  */
-bool
-Make_Run(Lst targs) /* the initial list of targets */
+void
+Make_Run(Lst targs, bool *has_errors, bool *out_of_date)
 {
- bool problem; /* errors occurred */
-
- /* wild guess at initial sizes */
- Array_Init(&to_build, 500);
- Array_Init(&examine, 150);
- Array_Init(&heldBack, 100);
- ohash_init(&targets, 10, &gnode_info);
  if (DEBUG(PARALLEL))
  random_setup();
 
@@ -545,7 +552,8 @@ Make_Run(Lst targs) /* the initial list of targets */
  * the next loop... (we won't actually start any, of course,
  * this is just to see if any of the targets was out of date)
  */
- return MakeStartJobs();
+ if (MakeStartJobs())
+ *out_of_date = true;
  } else {
  /*
  * Initialization. At the moment, no jobs are running and until
@@ -572,8 +580,8 @@ Make_Run(Lst targs) /* the initial list of targets */
  (void)MakeStartJobs();
  }
 
- if (!queryFlag)
- problem = Job_Finish();
+ if (errorJobs != NULL)
+ *has_errors = true;
 
  /*
  * Print the final status of each target. E.g. if it wasn't made
@@ -581,13 +589,9 @@ Make_Run(Lst targs) /* the initial list of targets */
  */
  if (targets_contain_cycles()) {
  break_and_print_cycles(targs);
- problem = true;
+ *has_errors = true;
  }
  Lst_Every(targs, MakePrintStatus);
- if (problem)
- Fatal("Errors while building");
-
- return true;
 }
 
 /* round-about detection: assume make is bug-free, if there are targets
diff --git a/make.h b/make.h
index 91839fb..cb8e09e 100644
--- a/make.h
+++ b/make.h
@@ -41,8 +41,9 @@
  */
 
 extern void Make_Update(GNode *);
-extern bool Make_Run(Lst);
+extern void Make_Run(Lst, bool *, bool *);
+extern void Make_Init(void);
 extern long random_delay;
-extern bool no_jobs_left(void);
+extern bool nothing_left_to_build(void);
 
 #endif /* _MAKE_H_ */
diff --git a/parse.c b/parse.c
index 4c90d18..dfc2abc 100644
--- a/parse.c
+++ b/parse.c
@@ -181,40 +181,40 @@ static struct {
  const char *keyword;
  size_t sz;
  uint32_t hv;
- unsigned int type;
+ unsigned int special;
  unsigned int special_op;
 } specials[] = {
-    { P(NODE_EXEC), SPECIAL_EXEC | SPECIAL_TARGETSOURCE, OP_EXEC, },
-    { P(NODE_IGNORE), SPECIAL_IGNORE | SPECIAL_TARGETSOURCE, OP_IGNORE, },
-    { P(NODE_INCLUDES), SPECIAL_NOTHING | SPECIAL_TARGET, 0, },
-    { P(NODE_INVISIBLE),SPECIAL_INVISIBLE | SPECIAL_TARGETSOURCE,OP_INVISIBLE, },
-    { P(NODE_JOIN), SPECIAL_JOIN | SPECIAL_TARGETSOURCE, OP_JOIN, },
-    { P(NODE_LIBS), SPECIAL_NOTHING | SPECIAL_TARGET, 0, },
-    { P(NODE_MADE), SPECIAL_MADE | SPECIAL_TARGETSOURCE, OP_MADE, },
-    { P(NODE_MAIN), SPECIAL_MAIN | SPECIAL_TARGET, 0, },
-    { P(NODE_MAKE), SPECIAL_MAKE | SPECIAL_TARGETSOURCE, OP_MAKE, },
-    { P(NODE_MAKEFLAGS), SPECIAL_MFLAGS | SPECIAL_TARGET, 0, },
-    { P(NODE_MFLAGS), SPECIAL_MFLAGS | SPECIAL_TARGET, 0, },
-    { P(NODE_NOTMAIN), SPECIAL_NOTMAIN | SPECIAL_TARGETSOURCE, OP_NOTMAIN, },
-    { P(NODE_NOTPARALLEL),SPECIAL_NOTPARALLEL | SPECIAL_TARGET, 0, },
-    { P(NODE_NO_PARALLEL),SPECIAL_NOTPARALLEL | SPECIAL_TARGET, 0, },
-    { P(NODE_NULL), SPECIAL_NOTHING | SPECIAL_TARGET, 0, },
-    { P(NODE_OPTIONAL), SPECIAL_OPTIONAL | SPECIAL_TARGETSOURCE,OP_OPTIONAL, },
-    { P(NODE_ORDER), SPECIAL_ORDER | SPECIAL_TARGET, 0, },
-    { P(NODE_PARALLEL), SPECIAL_PARALLEL | SPECIAL_TARGET, 0, },
-    { P(NODE_PATH), SPECIAL_PATH | SPECIAL_TARGET, 0, },
-    { P(NODE_PHONY), SPECIAL_PHONY | SPECIAL_TARGETSOURCE, OP_PHONY, },
-    { P(NODE_PRECIOUS), SPECIAL_PRECIOUS | SPECIAL_TARGETSOURCE,OP_PRECIOUS, },
-    { P(NODE_RECURSIVE),SPECIAL_MAKE | SPECIAL_TARGETSOURCE, OP_MAKE, },
-    { P(NODE_SILENT), SPECIAL_SILENT | SPECIAL_TARGETSOURCE, OP_SILENT, },
-    { P(NODE_SINGLESHELL),SPECIAL_NOTHING | SPECIAL_TARGET, 0, },
-    { P(NODE_SUFFIXES), SPECIAL_SUFFIXES | SPECIAL_TARGET, 0, },
-    { P(NODE_USE), SPECIAL_USE | SPECIAL_TARGETSOURCE, OP_USE, },
-    { P(NODE_WAIT), SPECIAL_WAIT | SPECIAL_TARGETSOURCE, 0 },
-    { P(NODE_CHEAP), SPECIAL_CHEAP | SPECIAL_TARGETSOURCE, OP_CHEAP, },
-    { P(NODE_EXPENSIVE),SPECIAL_EXPENSIVE | SPECIAL_TARGETSOURCE,OP_EXPENSIVE, },
-    { P(NODE_POSIX), SPECIAL_NOTHING | SPECIAL_TARGET, 0 },
-    { P(NODE_SCCS_GET), SPECIAL_NOTHING | SPECIAL_TARGET, 0 },
+    { P(NODE_EXEC), SPECIAL_EXEC, OP_EXEC },
+    { P(NODE_IGNORE), SPECIAL_IGNORE, OP_IGNORE },
+    { P(NODE_INCLUDES), SPECIAL_NOTHING, 0 },
+    { P(NODE_INVISIBLE), SPECIAL_INVISIBLE, OP_INVISIBLE },
+    { P(NODE_JOIN), SPECIAL_JOIN, OP_JOIN },
+    { P(NODE_LIBS), SPECIAL_NOTHING, 0 },
+    { P(NODE_MADE), SPECIAL_MADE, OP_MADE },
+    { P(NODE_MAIN), SPECIAL_MAIN, 0 },
+    { P(NODE_MAKE), SPECIAL_MAKE, OP_MAKE },
+    { P(NODE_MAKEFLAGS), SPECIAL_MFLAGS, 0 },
+    { P(NODE_MFLAGS), SPECIAL_MFLAGS, 0 },
+    { P(NODE_NOTMAIN), SPECIAL_NOTMAIN, OP_NOTMAIN },
+    { P(NODE_NOTPARALLEL), SPECIAL_NOTPARALLEL, 0 },
+    { P(NODE_NO_PARALLEL), SPECIAL_NOTPARALLEL, 0 },
+    { P(NODE_NULL), SPECIAL_NOTHING, 0 },
+    { P(NODE_OPTIONAL), SPECIAL_OPTIONAL, OP_OPTIONAL },
+    { P(NODE_ORDER), SPECIAL_ORDER, 0 },
+    { P(NODE_PARALLEL), SPECIAL_PARALLEL, 0 },
+    { P(NODE_PATH), SPECIAL_PATH, 0 },
+    { P(NODE_PHONY), SPECIAL_PHONY, OP_PHONY },
+    { P(NODE_PRECIOUS), SPECIAL_PRECIOUS, OP_PRECIOUS },
+    { P(NODE_RECURSIVE), SPECIAL_MAKE, OP_MAKE },
+    { P(NODE_SILENT), SPECIAL_SILENT, OP_SILENT },
+    { P(NODE_SINGLESHELL), SPECIAL_NOTHING, 0 },
+    { P(NODE_SUFFIXES), SPECIAL_SUFFIXES, 0 },
+    { P(NODE_USE), SPECIAL_USE, OP_USE },
+    { P(NODE_WAIT), SPECIAL_WAIT, 0 },
+    { P(NODE_CHEAP), SPECIAL_CHEAP, OP_CHEAP },
+    { P(NODE_EXPENSIVE), SPECIAL_EXPENSIVE, OP_EXPENSIVE },
+    { P(NODE_POSIX), SPECIAL_NOTHING, 0 },
+    { P(NODE_SCCS_GET), SPECIAL_NOTHING, 0 },
 };
 
 #undef P
@@ -225,10 +225,9 @@ create_special_nodes()
  unsigned int i;
 
  for (i = 0; i < sizeof(specials)/sizeof(specials[0]); i++) {
- GNode *gn = Targ_FindNodeh(specials[i].keyword,
-    specials[i].sz, specials[i].hv, TARG_CREATE);
- gn->special = specials[i].type;
- gn->special_op = specials[i].special_op;
+ (void)Targ_mk_special_node(specials[i].keyword,
+    specials[i].sz, specials[i].hv,
+    OP_ZERO, specials[i].special, specials[i].special_op);
  }
 }
 
@@ -419,15 +418,13 @@ ParseDoSrc(
     const char *esrc)
 {
  GNode *gn = Targ_FindNodei(src, esrc, TARG_CREATE);
- if ((gn->special & SPECIAL_SOURCE) != 0) {
- if (gn->special_op) {
- Array_ForEach(targets, ParseDoSpecial, gn->special_op);
- return;
- } else {
- assert((gn->special & SPECIAL_MASK) == SPECIAL_WAIT);
- waiting++;
- return;
- }
+ if (gn->special_op) {
+ Array_ForEach(targets, ParseDoSpecial, gn->special_op);
+ return;
+ }
+ if (gn->special == SPECIAL_WAIT) {
+ waiting++;
+ return;
  }
 
  switch (specType) {
@@ -705,10 +702,10 @@ handle_special_targets(Lst paths)
 
  for (i = 0; i < gtargets.n; i++) {
  type = gtargets.a[i]->special;
- if ((type & SPECIAL_MASK) == SPECIAL_PATH) {
+ if (type == SPECIAL_PATH) {
  seen_path++;
  Lst_AtEnd(paths, find_suffix_path(gtargets.a[i]));
- } else if ((type & SPECIAL_TARGET) != 0)
+ } else if (type != 0)
  seen_special++;
  else
  seen_normal++;
@@ -734,7 +731,7 @@ handle_special_targets(Lst paths)
  dump_targets();
  return 0;
  } else if (seen_special == 1) {
- specType = gtargets.a[0]->special & SPECIAL_MASK;
+ specType = gtargets.a[0]->special;
  switch (specType) {
  case SPECIAL_MAIN:
  if (!Lst_IsEmpty(create)) {
@@ -742,13 +739,8 @@ handle_special_targets(Lst paths)
  }
  break;
  case SPECIAL_NOTPARALLEL:
- {
- extern int  maxJobs;
-
- maxJobs = 1;
- compatMake = 1;
+ set_notparallel();
  break;
- }
  case SPECIAL_ORDER:
  predecessor = NULL;
  break;
@@ -838,6 +830,7 @@ ParseDoDependency(const char *line) /* the line to parse */
  Array_Reset(&gsources);
 
  cp = parse_do_targets(&paths, &tOp, line);
+ assert(specType == SPECIAL_PATH || Lst_IsEmpty(&paths));
  if (cp == NULL || specType == SPECIAL_ERROR) {
  /* invalidate targets for further processing */
  Array_Reset(&gtargets);
@@ -856,19 +849,15 @@ ParseDoDependency(const char *line) /* the line to parse */
 
  line = cp;
 
- /*
- * Several special targets take different actions if present with no
- * sources:
- * a .SUFFIXES line with no sources clears out all old suffixes
- * a .PRECIOUS line makes all targets precious
- * a .IGNORE line ignores errors for all targets
- * a .SILENT line creates silence when making all targets
- * a .PATH removes all directories from the search path(s).
- */
+ /* Several special targets have specific semantics with no source:
+ * .SUFFIXES clears out all old suffixes
+ * .PRECIOUS/.IGNORE/.SILENT
+ * apply to all target
+ * .PATH clears out all search paths.  */
  if (!*line) {
  switch (specType) {
  case SPECIAL_SUFFIXES:
- Suff_ClearSuffixes();
+ Suff_DisableAllSuffixes();
  break;
  case SPECIAL_PRECIOUS:
  allPrecious = true;
@@ -886,42 +875,26 @@ ParseDoDependency(const char *line) /* the line to parse */
  break;
  }
  } else if (specType == SPECIAL_MFLAGS) {
- /* Call on functions in main.c to deal with these arguments */
  Main_ParseArgLine(line);
  return;
  } else if (specType == SPECIAL_NOTPARALLEL) {
  return;
  }
 
- /*
- * NOW GO FOR THE SOURCES
- */
+ /* NOW GO FOR THE SOURCES */
  if (specType == SPECIAL_SUFFIXES || specType == SPECIAL_PATH ||
     specType == SPECIAL_NOTHING) {
  while (*line) {
-    /*
-     * If the target was one that doesn't take files as its
-     * sources but takes something like suffixes, we take each
-     * space-separated word on the line as a something and deal
-     * with it accordingly.
-     *
-     * If the target was .SUFFIXES, we take each source as a
-     * suffix and add it to the list of suffixes maintained by
-     * the Suff module.
-     *
-     * If the target was a .PATH, we add the source as a
-     * directory to search on the search path.
+    /* Some special targets take a list of space-separated
+     * words.  For each word,
      *
-     * If it was .INCLUDES, the source is taken to be the
-     * suffix of files which will be #included and whose search
-     * path should be present in the .INCLUDES variable.
+     * if .SUFFIXES, add it to the list of suffixes maintained
+     * by suff.c.
      *
-     * If it was .LIBS, the source is taken to be the suffix of
-     * files which are considered libraries and whose search
-     * path should be present in the .LIBS variable.
+     * if .PATHS, add it as a directory on the main search path.
      *
-     * If it was .NULL, the source is the suffix to use when a
-     * file has no valid suffix.
+     * if .LIBS/.INCLUDE/.NULL... this has been deprecated,
+     * ignore
      */
     while (*cp && !ISSPACE(*cp))
     cp++;
@@ -937,6 +910,7 @@ ParseDoDependency(const char *line) /* the line to parse */
      ln = Lst_Adv(ln))
     Dir_AddDiri(Lst_Datum(ln), line, cp);
     break;
+    Lst_Destroy(&paths, NOFREE);
     }
     default:
     break;
@@ -947,7 +921,6 @@ ParseDoDependency(const char *line) /* the line to parse */
  cp++;
     line = cp;
  }
- Lst_Destroy(&paths, NOFREE);
  } else {
  while (*line) {
  /*
@@ -1642,12 +1615,12 @@ Parse_File(const char *filename, FILE *stream)
  bool expectingCommands = false;
  bool commands_seen = false;
 
- /* somewhat permanent spaces to shave time */
- BUFFER buf;
- BUFFER copy;
+ /* permanent spaces to shave time */
+ static BUFFER buf;
+ static BUFFER copy;
 
- Buf_Init(&buf, MAKE_BSIZE);
- Buf_Init(&copy, MAKE_BSIZE);
+ Buf_Reinit(&buf, MAKE_BSIZE);
+ Buf_Reinit(&copy, MAKE_BSIZE);
 
  Parse_FromFile(filename, stream);
  do {
@@ -1687,8 +1660,6 @@ Parse_File(const char *filename, FILE *stream)
  Cond_End();
 
  Parse_ReportErrors();
- Buf_Destroy(&buf);
- Buf_Destroy(&copy);
 }
 
 void
diff --git a/suff.c b/suff.c
index 94f0160..e03b745 100644
--- a/suff.c
+++ b/suff.c
@@ -39,28 +39,9 @@
  * suff.c --
  * Functions to maintain suffix lists and find implicit dependents
  * using suffix transformation rules
- *
- * Interface:
- * Suff_Init Initialize all things to do with suffixes.
- *
- * Suff_ClearSuffixes Clear out all the suffixes.
- *
- * Suff_AddSuffix Add the passed string as another known suffix.
- *
- * Suff_ParseAsTransform Line might be a suffix line, check it.
- * If it's not, return NULL. Otherwise, add
- * another transformation to the suffix graph.
- * Returns GNode suitable for framing, I mean,
- * tacking commands, attributes, etc. on.
- *
- * Suff_FindDeps Find implicit sources for and the location of
- * a target based on its suffix. Returns the
- * bottom-most node added to the graph or NULL
- * if the target had no implicit sources.
  */
 
 #include <ctype.h>
-#include <signal.h>
 #include <stddef.h>
 #include <stdint.h>
 #include <stdio.h>
@@ -70,9 +51,7 @@
 #include "config.h"
 #include "defines.h"
 #include "dir.h"
-#include "direxpand.h"
 #include "engine.h"
-#include "arch.h"
 #include "suff.h"
 #include "var.h"
 #include "targ.h"
@@ -81,9 +60,9 @@
 #include "lst.h"
 #include "memory.h"
 #include "gnode.h"
-#include "make.h"
 #include "stats.h"
 #include "dump.h"
+#include "expandchildren.h"
 
 /* XXX the suffixes hash is stored using a specific hash function, suitable
  * for looking up suffixes in reverse.
@@ -91,7 +70,7 @@
 static struct ohash suffixes;
 
 /* We remember the longest suffix, so we don't need to look beyond that.  */
-size_t maxLen;
+size_t maxLen = 0U;
 static LIST srclist;
 
 /* Transforms (.c.o) are stored in another hash, independently from suffixes.
@@ -172,7 +151,6 @@ static Suff *new_suffixi(const char *, const char *);
 static void reverse_hash_add_char(uint32_t *, const char *);
 static uint32_t reverse_hashi(const char *, const char **);
 static unsigned int reverse_slot(struct ohash *, const char *, const char **);
-static void clear_suffixes(void);
 static void record_possible_suffix(Suff *, GNode *, char *, Lst, Lst);
 static void record_possible_suffixes(GNode *, Lst, Lst);
 static Suff *find_suffix_as_suffix(Lst, const char *, const char *);
@@ -184,9 +162,6 @@ static bool SuffRemoveSrc(Lst);
 static void SuffAddLevel(Lst, Src *);
 static Src *SuffFindThem(Lst, Lst);
 static Src *SuffFindCmds(Src *, Lst);
-static void SuffExpandChildren(LstNode, GNode *);
-static void SuffExpandVarChildren(LstNode, GNode *, GNode *);
-static void SuffExpandWildChildren(LstNode, GNode *, GNode *);
 static bool SuffApplyTransform(GNode *, GNode *, Suff *, Suff *);
 static void SuffFindDeps(GNode *, Lst);
 static void SuffFindArchiveDeps(GNode *, Lst);
@@ -348,18 +323,11 @@ SuffInsert(Lst l, Suff *s)
  }
 }
 
-/*-
- *-----------------------------------------------------------------------
- * Suff_ClearSuffixes --
- * Nuke the list of suffixes but keep all transformation
- * rules around.
- *
- * Side Effects:
- * Current suffixes are reset
- *-----------------------------------------------------------------------
- */
-static void
-clear_suffixes(void)
+/* Suff_DisableAllSuffixes
+ * mark all current suffixes as inactive, and reset precedence
+ * computation.  */
+void
+Suff_DisableAllSuffixes(void)
 {
  unsigned int i;
  Suff *s;
@@ -372,12 +340,6 @@ clear_suffixes(void)
  maxLen = 0;
 }
 
-void
-Suff_ClearSuffixes(void)
-{
- clear_suffixes();
-}
-
 
 /* okay = parse_transform(str, &src, &targ);
  * try parsing a string as a transformation rule, returns true if
@@ -488,6 +450,18 @@ find_best_suffix(const char *s, const char *e)
  return best;
 }
 
+Lst
+find_best_path(const char *name)
+{
+ Suff *s = find_best_suffix(name, name + strlen(name));
+ if (s != NULL) {
+ if (DEBUG(SUFF))
+ printf("suffix is \"%s\"...", s->name);
+ return &s->searchPath;
+ } else
+ return defaultPath;
+}
+
 /*-
  *-----------------------------------------------------------------------
  * Suff_ParseAsTransform --
@@ -523,7 +497,7 @@ Suff_ParseAsTransform(const char *line, const char *end)
 
  gn->type = OP_TRANSFORM;
  if (s->flags & SUFF_PATH) {
- gn->special = SPECIAL_PATH | SPECIAL_TARGET;
+ gn->special = SPECIAL_PATH;
  gn->suffix = t;
  }
 
@@ -612,7 +586,7 @@ build_suffixes_graph(void)
     gn = ohash_next(&transforms, &i)) {
      if (Lst_IsEmpty(&gn->commands) && Lst_IsEmpty(&gn->children))
  continue;
- if ((gn->special & SPECIAL_MASK) == SPECIAL_PATH)
+ if (gn->special == SPECIAL_PATH)
  continue;
      if (parse_transform(gn->name, &s, &s2)) {
  SuffInsert(&s2->children, s);
@@ -629,11 +603,7 @@ build_suffixes_graph(void)
  *
  * Side Effects:
  * The searchPath field of all the suffixes is extended by the
- * directories in defaultPath. If paths were specified for the
- * ".h" suffix, the directories are stuffed into a global variable
- * called ".INCLUDES" with each directory preceded by a -I. The same
- * is done for the ".a" suffix, except the variable is called
- * ".LIBS" and the flag is -L.
+ * directories in defaultPath.
  *-----------------------------------------------------------------------
  */
 static void
@@ -912,203 +882,6 @@ SuffFindCmds(Src *targ, Lst slst)
  return NULL;
 }
 
-static void
-SuffLinkParent(GNode *cgn, GNode *pgn)
-{
- Lst_AtEnd(&cgn->parents, pgn);
- if (!has_been_built(cgn))
- pgn->children_left++;
- else if ( ! (cgn->type & (OP_EXEC|OP_USE))) {
- if (cgn->built_status == REBUILT)
- pgn->child_rebuilt = true;
- (void)Make_TimeStamp(pgn, cgn);
- }
-}
-
-static void
-SuffExpandVarChildren(LstNode after, GNode *cgn, GNode *pgn)
-{
- GNode *gn; /* New source 8) */
- char *cp; /* Expanded value */
- LIST members;
-
-
- if (DEBUG(SUFF))
- printf("Expanding \"%s\"...", cgn->name);
-
- cp = Var_Subst(cgn->name, &pgn->localvars, true);
- if (cp == NULL) {
- printf("Problem substituting in %s", cgn->name);
- printf("\n");
- return;
- }
-
- Lst_Init(&members);
-
- if (cgn->type & OP_ARCHV) {
- /*
- * Node was an archive(member) target, so we want to call
- * on the Arch module to find the nodes for us, expanding
- * variables in the parent's context.
- */
- const char *sacrifice = (const char *)cp;
-
- (void)Arch_ParseArchive(&sacrifice, &members, &pgn->localvars);
- } else {
- /* Break the result into a vector of strings whose nodes
- * we can find, then add those nodes to the members list.
- * Unfortunately, we can't use brk_string because it
- * doesn't understand about variable specifications with
- * spaces in them...  */
- const char *start, *cp2;
-
- for (start = cp; *start == ' ' || *start == '\t'; start++)
- continue;
- for (cp2 = start; *cp2 != '\0';) {
- if (ISSPACE(*cp2)) {
- /* White-space -- terminate element, find the
- * node, add it, skip any further spaces.  */
- gn = Targ_FindNodei(start, cp2, TARG_CREATE);
- cp2++;
- Lst_AtEnd(&members, gn);
- while (ISSPACE(*cp2))
- cp2++;
- /* Adjust cp2 for increment at start of loop,
- * but set start to first non-space.  */
- start = cp2;
- } else if (*cp2 == '$')
- /* Start of a variable spec -- contact variable
- * module to find the end so we can skip over
- * it.  */
- Var_ParseSkip(&cp2, &pgn->localvars);
- else if (*cp2 == '\\' && cp2[1] != '\0')
- /* Escaped something -- skip over it.  */
- cp2+=2;
- else
- cp2++;
-    }
-
-    if (cp2 != start) {
-    /* Stuff left over -- add it to the list too.  */
-    gn = Targ_FindNodei(start, cp2, TARG_CREATE);
-    Lst_AtEnd(&members, gn);
-    }
- }
- /* Add all elements of the members list to the parent node.  */
- while ((gn = Lst_DeQueue(&members)) != NULL) {
- if (DEBUG(SUFF))
- printf("%s...", gn->name);
- if (Lst_Member(&pgn->children, gn) == NULL) {
- Lst_Append(&pgn->children, after, gn);
- after = Lst_Adv(after);
- SuffLinkParent(gn, pgn);
- }
- }
- /* Free the result.  */
- free(cp);
- if (DEBUG(SUFF))
- printf("\n");
-}
-
-static void
-SuffExpandWildChildren(LstNode after, GNode *cgn, GNode *pgn)
-{
- Suff *s;
- char *cp; /* Expanded value */
-
- LIST exp; /* List of expansions */
- Lst path; /* Search path along which to expand */
-
- if (DEBUG(SUFF))
- printf("Wildcard expanding \"%s\"...", cgn->name);
-
- /* Find a path along which to expand the word.
- *
- * If the word has a known suffix, use that path.
- * If it has no known suffix and we're allowed to use the null
- * suffix, use its path.
- * Else use the default system search path.  */
- s = find_best_suffix(cgn->name, cgn->name + strlen(cgn->name));
-
- if (s != NULL) {
- if (DEBUG(SUFF))
- printf("suffix is \"%s\"...", s->name);
- path = &s->searchPath;
- } else
- /* Use default search path.  */
- path = defaultPath;
-
- /* Expand the word along the chosen path. */
- Lst_Init(&exp);
- Dir_Expand(cgn->name, path, &exp);
-
- /* Fetch next expansion off the list and find its GNode.  */
- while ((cp = Lst_DeQueue(&exp)) != NULL) {
- GNode *gn; /* New source 8) */
- if (DEBUG(SUFF))
- printf("%s...", cp);
- gn = Targ_FindNode(cp, TARG_CREATE);
-
- /* If gn isn't already a child of the parent, make it so and
- * up the parent's count of children to build.  */
- if (Lst_Member(&pgn->children, gn) == NULL) {
- Lst_Append(&pgn->children, after, gn);
- after = Lst_Adv(after);
- SuffLinkParent(gn, pgn);
- }
- }
-
- if (DEBUG(SUFF))
- printf("\n");
-}
-
-/*-
- *-----------------------------------------------------------------------
- * SuffExpandChildren --
- * Expand the names of any children of a given node that contain
- * variable invocations or file wildcards into actual targets.
- *
- * Side Effects:
- * The expanded node is removed from the parent's list of children,
- * and the parent's children to build counter is decremented,
- *      but other nodes may be added.
- *-----------------------------------------------------------------------
- */
-static void
-SuffExpandChildren(LstNode ln, /* LstNode of child, so we can replace it */
-    GNode *pgn)
-{
- GNode *cgn = Lst_Datum(ln);
-
- /* First do variable expansion -- this takes precedence over wildcard
- * expansion. If the result contains wildcards, they'll be gotten to
- * later since the resulting words are tacked on to the end of the
- * children list.  */
- if (strchr(cgn->name, '$') != NULL)
- SuffExpandVarChildren(ln, cgn, pgn);
- else if (Dir_HasWildcards(cgn->name))
- SuffExpandWildChildren(ln, cgn, pgn);
- else
-    /* Third case: nothing to expand.  */
- return;
-
- /* Since the source was expanded, remove it from the list of children to
- * keep it from being processed.  */
- pgn->children_left--;
- Lst_Remove(&pgn->children, ln);
-}
-
-void
-expand_children_from(GNode *parent, LstNode from)
-{
- LstNode np, ln;
-
- for (ln = from; ln != NULL; ln = np) {
- np = Lst_Adv(ln);
- SuffExpandChildren(ln, parent);
- }
-}
-
 /*-
  *-----------------------------------------------------------------------
  * SuffApplyTransform --
@@ -1140,7 +913,7 @@ SuffApplyTransform(
  if (Lst_AddNew(&tGn->children, sGn)) {
  /* Not already linked, so form the proper links between the
  * target and source.  */
- SuffLinkParent(sGn, tGn);
+ LinkParent(sGn, tGn);
  }
 
  if ((sGn->type & OP_OPMASK) == OP_DOUBLEDEP) {
@@ -1154,7 +927,7 @@ SuffApplyTransform(
  if (Lst_AddNew(&tGn->children, gn)) {
  /* Not already linked, so form the proper links
  * between the target and source.  */
- SuffLinkParent(gn, tGn);
+ LinkParent(gn, tGn);
  }
  }
  }
@@ -1247,7 +1020,7 @@ SuffFindArchiveDeps(
 
  /* Create the link between the two nodes right off. */
  if (Lst_AddNew(&gn->children, mem))
- SuffLinkParent(mem, gn);
+ LinkParent(mem, gn);
 
  /* Copy variables from member node to this one.  */
  Var(TARGET_INDEX, gn) = Var(TARGET_INDEX, mem);
@@ -1685,19 +1458,14 @@ Suff_Init(void)
  Static_Lst_Init(&srclist);
  ohash_init(&transforms, 4, &gnode_info);
 
- /*
- * Create null suffix for single-suffix rules (POSIX). The thing doesn't
- * actually go on the suffix list or everyone will think that's its
- * suffix.
- */
+ /* Create null suffix for single-suffix rules (POSIX). The thing doesn't
+ * actually go on the suffix list as it matches everything.  */
  emptySuff = new_suffix("");
- make_suffix_known(emptySuff);
+ emptySuff->flags = SUFF_ACTIVE;
+ emptySuff->order = 0;
  Dir_Concat(&emptySuff->searchPath, defaultPath);
  ohash_init(&suffixes, 4, &suff_info);
- order = 0;
- clear_suffixes();
  special_path_hack();
-
 }
 
 
diff --git a/suff.h b/suff.h
index 2fe7923..79e65d2 100644
--- a/suff.h
+++ b/suff.h
@@ -3,7 +3,7 @@
 /* $OpenBSD: suff.h,v 1.10 2012/12/06 14:30:35 espie Exp $ */
 
 /*
- * Copyright (c) 2001 Marc Espie.
+ * Copyright (c) 2001-2019 Marc Espie.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -27,16 +27,40 @@
  * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-extern void Suff_ClearSuffixes(void);
+extern void Suff_Init(void);
+
+/* Suff_DisableAllSuffixes():
+ * disable current suffixes and the corresponding rules.
+ * They may be re-activated by adding a suffix anew.  */
+extern void Suff_DisableAllSuffixes(void);
+/* gn = Suff_ParseAsTransform(line, eline):
+ * Try parsing a [line,eline[ as a suffix transformation
+ * (.a.b or .a). If successful, returns a gn we can add
+ * commands to (this is actually a transform kept on a
+ * separate hash from normal targets).  Otherwise returns NULL. */
 extern GNode *Suff_ParseAsTransform(const char *, const char *);
+/* Suff_AddSuffixi(name, ename):
+ * add the passed string interval [name,ename[ as a known
+ * suffix. */
 extern void Suff_AddSuffixi(const char *, const char *);
-extern void Suff_FindDeps(GNode *);
-extern void Suff_Init(void);
+/* process_suffixes_after_makefile_is_read():
+ * finish setting up the transformation graph for Suff_FindDep
+ * and the .PATH.sfx paths get the default path appended for
+ * find_suffix_path().  */
 extern void process_suffixes_after_makefile_is_read(void);
+/* Suff_FindDeps(gn):
+ * find implicit dependencies for gn and fill out corresponding
+ * fields. */
+extern void Suff_FindDeps(GNode *);
+/* l = find_suffix_path(gn):
+ * returns the path associated with a gn, either because of its
+ * suffix, or the default path.  */
 extern Lst find_suffix_path(GNode *);
+/* Suff_PrintAll():
+ * displays all suffix information. */
 extern void Suff_PrintAll(void);
-extern void expand_children_from(GNode *, LstNode);
-#define expand_all_children(gn) \
-    expand_children_from(gn, Lst_First(&(gn)->children))
-
+/* path = find_best_path(name):
+ * find the best path for the name, according to known suffixes.
+ */
+extern Lst find_best_path(const char *name);
 #endif
diff --git a/targ.c b/targ.c
index b2d2489..bba0192 100644
--- a/targ.c
+++ b/targ.c
@@ -122,8 +122,11 @@ struct ohash_info gnode_info = {
  offsetof(GNode, name), NULL, hash_calloc, hash_free, element_alloc
 };
 
-#define Targ_FindConstantNode(n, f) Targ_FindNodeh(n, sizeof(n), K_##n, f)
+static GNode *Targ_mk_node(const char *, const char *, unsigned int,
+    unsigned char, unsigned int);
 
+#define Targ_mk_constant(n, type) \
+    Targ_mk_special_node(n, sizeof(n), K_##n, type, SPECIAL_NONE, 0)
 
 GNode *begin_node, *end_node, *interrupt_node, *DEFAULT;
 
@@ -132,27 +135,28 @@ Targ_Init(void)
 {
  /* A small make file already creates 200 targets.  */
  ohash_init(&targets, 10, &gnode_info);
- begin_node = Targ_FindConstantNode(NODE_BEGIN, TARG_CREATE);
- begin_node->type |= OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT;
- end_node = Targ_FindConstantNode(NODE_END, TARG_CREATE);
- end_node->type |= OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT;
- interrupt_node = Targ_FindConstantNode(NODE_INTERRUPT, TARG_CREATE);
- interrupt_node->type |= OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT;
- DEFAULT = Targ_FindConstantNode(NODE_DEFAULT, TARG_CREATE);
- DEFAULT->type |= OP_DUMMY | OP_NOTMAIN| OP_TRANSFORM | OP_NODEFAULT;
+ begin_node = Targ_mk_constant(NODE_BEGIN,
+    OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT);
+ end_node = Targ_mk_constant(NODE_END,
+    OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT);
+ interrupt_node = Targ_mk_constant(NODE_INTERRUPT,
+    OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT);
+ DEFAULT = Targ_mk_constant(NODE_DEFAULT,
+    OP_DUMMY | OP_NOTMAIN| OP_TRANSFORM | OP_NODEFAULT);
 
 }
 
-GNode *
-Targ_NewGNi(const char *name, const char *ename)
+static GNode *
+Targ_mk_node(const char *name, const char *ename,
+    unsigned int type, unsigned char special, unsigned int special_op)
 {
  GNode *gn;
 
  gn = ohash_create_entry(&gnode_info, name, &ename);
  gn->path = NULL;
- gn->type = OP_ZERO;
- gn->special = SPECIAL_NONE;
- gn->special_op = 0;
+ gn->type = type;
+ gn->special = special;
+ gn->special_op = special_op;
  gn->children_left = 0;
  gn->must_make = false;
  gn->built_status = UNKNOWN;
@@ -183,20 +187,20 @@ Targ_NewGNi(const char *name, const char *ename)
 }
 
 GNode *
-Targ_FindNodei(const char *name, const char *ename, int flags)
+Targ_NewGNi(const char *name, const char *ename)
 {
- uint32_t hv;
-
- hv = ohash_interval(name, &ename);
- return Targ_FindNodeih(name, ename, hv, flags);
+ return Targ_mk_node(name, ename, OP_ZERO, SPECIAL_NONE, 0);
 }
 
 GNode *
-Targ_FindNodeih(const char *name, const char *ename, uint32_t hv, int flags)
+Targ_FindNodei(const char *name, const char *ename, int flags)
 {
+ uint32_t hv;
  GNode *gn;
  unsigned int slot;
 
+ hv = ohash_interval(name, &ename);
+
  slot = ohash_lookup_interval(&targets, name, ename, hv);
 
  gn = ohash_find(&targets, slot);
@@ -209,6 +213,24 @@ Targ_FindNodeih(const char *name, const char *ename, uint32_t hv, int flags)
  return gn;
 }
 
+GNode *
+Targ_mk_special_node(const char *name, size_t n, uint32_t hv,
+    unsigned int type, unsigned char special, unsigned int special_op)
+{
+ GNode *gn;
+ unsigned int slot;
+ const char *ename = name + n - 1;
+
+ slot = ohash_lookup_interval(&targets, name, ename, hv);
+
+ assert(ohash_find(&targets, slot) == NULL);
+
+ gn = Targ_mk_node(name, ename, type, special, special_op);
+ ohash_insert(&targets, slot, gn);
+
+ return gn;
+}
+
 void
 Targ_FindList(Lst nodes, Lst names)
 {
@@ -255,6 +277,12 @@ Targ_Precious(GNode *gn)
  return false;
 }
 
+bool
+node_is_real(GNode *gn)
+{
+ return (gn->type & OP_DUMMY) == 0;
+}
+
 void
 Targ_PrintCmd(void *p)
 {
@@ -321,9 +349,3 @@ targets_hash()
 {
  return &targets;
 }
-
-GNode *
-Targ_FindNodeh(const char *name, size_t n, uint32_t hv, int flags)
-{
- return Targ_FindNodeih(name, name + n - 1, hv, flags);
-}
diff --git a/targ.h b/targ.h
index 78517e3..69b527c 100644
--- a/targ.h
+++ b/targ.h
@@ -46,17 +46,10 @@ extern GNode *Targ_FindNodei(const char *, const char *, int);
 
 
 
-/* set of helpers for constant nodes */
-extern GNode *Targ_FindNodeih(const char *, const char *, uint32_t, int);
+/* helper for constant nodes */
+extern GNode *Targ_mk_special_node(const char *, size_t, uint32_t,
+    unsigned int, unsigned char, unsigned int);
 
-__only_inline GNode *
-Targ_FindNodeh(const char *, size_t, uint32_t, int);
-
-__only_inline GNode *
-Targ_FindNodeh(const char *name, size_t n, uint32_t hv, int flags)
-{
- return Targ_FindNodeih(name, name + n - 1, hv, flags);
-}
 extern void Targ_FindList(Lst, Lst);
 extern bool Targ_Ignore(GNode *);
 extern bool Targ_Silent(GNode *);
@@ -64,6 +57,7 @@ extern bool Targ_Precious(GNode *);
 extern void Targ_PrintCmd(void *);
 extern void Targ_PrintType(int);
 extern void Targ_PrintGraph(int);
+extern bool node_is_real(GNode *);
 
 extern GNode *begin_node, *end_node, *interrupt_node, *DEFAULT;
 struct ohash_info;

Reply | Threaded
Open this post in threaded view
|

Re: MAKE: redux patch

Alexander Bluhm
On Fri, Jan 10, 2020 at 01:58:47PM +0100, Marc Espie wrote:
> Bleh, I forgot to synch two patches I already committed. Here's a patch
> that applies cleanly.

I did run this make through a full regress.  It seems that make
regress in /usr/src/regress/usr.sbin/ldapd/ exits with an error.

http://bluhm.genua.de/regress/results/2020-01-10T14%3A11%3A58Z/logs/usr.sbin/ldapd/make.log

*** Error 1 in /usr/src/regress/usr.sbin/ldapd (Makefile:44 '.END': @[ -f /usr/src/regress/usr.sbin/ldapd/obj/ldapd.pid ] &&  kill $(cat /us...)

I have never seen this problem before.

bluhm

> diff --git a/Makefile b/Makefile
> index 0cd84fc..90747de 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -16,8 +16,8 @@ CFLAGS+=${CDEFS}
>  HOSTCFLAGS+=${CDEFS}
>
>  SRCS= arch.c buf.c cmd_exec.c compat.c cond.c dir.c direxpand.c dump.c \
> - engine.c \
> - error.c for.c init.c job.c lowparse.c main.c make.c memory.c parse.c \
> + engine.c enginechoice.c error.c expandchildren.c \
> + for.c init.c job.c lowparse.c main.c make.c memory.c parse.c \
>   parsevar.c str.c stats.c suff.c targ.c targequiv.c timestamp.c \
>   var.c varmodifiers.c varname.c
>
> diff --git a/arch.c b/arch.c
> index 85e8e7e..02c18b5 100644
> --- a/arch.c
> +++ b/arch.c
> @@ -195,11 +195,10 @@ bool
>  Arch_ParseArchive(const char **line, Lst nodes, SymTable *ctxt)
>  {
>   bool result;
> - BUFFER expand;
> + static BUFFER expand;
>
> - Buf_Init(&expand, MAKE_BSIZE);
> + Buf_Reinit(&expand, MAKE_BSIZE);
>   result = parse_archive(&expand, line, nodes, ctxt);
> - Buf_Destroy(&expand);
>   return result;
>  }
>
> diff --git a/buf.c b/buf.c
> index 74cbfe8..931bd7e 100644
> --- a/buf.c
> +++ b/buf.c
> @@ -153,6 +153,15 @@ Buf_printf(Buffer bp, const char *fmt, ...)
>   bp->inPtr += n;
>  }
>
> +void
> +Buf_Reinit(Buffer bp, size_t size)
> +{
> + if (bp->buffer == NULL)
> + Buf_Init(bp, size);
> + else
> + Buf_Reset(bp);
> +}
> +
>  void
>  Buf_Init(Buffer bp, size_t size)
>  {
> diff --git a/buf.h b/buf.h
> index 1b56b27..20ea56a 100644
> --- a/buf.h
> +++ b/buf.h
> @@ -106,6 +106,9 @@ extern void Buf_AddChars(Buffer, size_t, const char *);
>   * Initializes a buffer, to hold approximately init chars.
>   * Set init to 0 if you have no idea.  */
>  extern void Buf_Init(Buffer, size_t);
> +/* Buf_Reinit(buf, init);
> + * Initializes/reset a static buffer */
> +extern void Buf_Reinit(Buffer, size_t);
>  /* Buf_Destroy(buf);
>   * Nukes a buffer and all its resources. */
>  #define Buf_Destroy(bp) ((void)free((bp)->buffer))
> diff --git a/compat.c b/compat.c
> index 7ecb75f..fd78d78 100644
> --- a/compat.c
> +++ b/compat.c
> @@ -193,14 +193,12 @@ CompatMake(void *gnp, /* The node to make */
>   /* copy over what we just did */
>   gn->built_status = sib->built_status;
>
> - if (gn->built_status != ERROR) {
> - /* If the node was built successfully, mark it so,
> + if (gn->built_status == REBUILT) {
> + /* If the node was built successfully,
>   * update its modification time and timestamp all
>   * its parents.
>   * This is to keep its state from affecting that of
>   * its parent.  */
> - gn->built_status = REBUILT;
> - sib->built_status = REBUILT;
>   /* This is what Make does and it's actually a good
>   * thing, as it allows rules like
>   *
> @@ -266,12 +264,20 @@ CompatMake(void *gnp, /* The node to make */
>   }
>  }
>
> -bool
> -Compat_Run(Lst targs) /* List of target nodes to re-create */
> +void
> +Compat_Init()
> +{
> +}
> +
> +void
> +Compat_Update(GNode *gn)
> +{
> +}
> +
> +void
> +Compat_Run(Lst targs, bool *has_errors, bool *out_of_date)
>  {
>   GNode  *gn = NULL; /* Current root target */
> - int  errors;   /* Number of targets not built due to errors */
> - bool out_of_date = false;
>
>   /* For each entry in the list of targets to create, call CompatMake on
>   * it to create the thing. CompatMake will leave the 'built_status'
> @@ -283,7 +289,6 @@ Compat_Run(Lst targs) /* List of target nodes to re-create */
>   *    ABORTED    gn was not built because one of its
>   *                          dependencies could not be built due
>   *          to errors.  */
> - errors = 0;
>   while ((gn = Lst_DeQueue(targs)) != NULL) {
>   CompatMake(gn, NULL);
>
> @@ -292,15 +297,10 @@ Compat_Run(Lst targs) /* List of target nodes to re-create */
>   else if (gn->built_status == ABORTED) {
>   printf("`%s' not remade because of errors.\n",
>      gn->name);
> - out_of_date = true;
> - errors++;
> + *out_of_date = true;
> + *has_errors = true;
>   } else {
> - out_of_date = true;
> + *out_of_date = true;
>   }
>   }
> -
> - /* If the user has defined a .END target, run its commands.  */
> - if (errors == 0 && !queryFlag)
> - run_gnode(end_node);
> - return out_of_date;
>  }
> diff --git a/compat.h b/compat.h
> index 2420abd..0aa2de9 100644
> --- a/compat.h
> +++ b/compat.h
> @@ -35,9 +35,11 @@
>   *    - friendly variable substitution.
>   */
>
> -/* out_of_date = Compat_Run(to_create);
> +/* Compat_Run(to_create, &has_errors, &out_of_date);
>   * Run the actual make engine, to create targets that need to,
> - * return true if any target is out of date. */
> -extern bool Compat_Run(Lst);
> + * return info about what we did. */
> +extern void Compat_Run(Lst, bool *, bool *);
> +extern void Compat_Init(void);
> +extern void Compat_Update(GNode *);
>
>  #endif
> diff --git a/dump.c b/dump.c
> index 0f79c33..b3820eb 100644
> --- a/dump.c
> +++ b/dump.c
> @@ -104,7 +104,7 @@ TargPrintNode(GNode *gn, bool full)
>  {
>   if (OP_NOP(gn->type))
>   return;
> - switch((gn->special & SPECIAL_MASK)) {
> + switch(gn->special) {
>   case SPECIAL_SUFFIXES:
>   case SPECIAL_PHONY:
>   case SPECIAL_ORDER:
> diff --git a/engine.c b/engine.c
> index 7c786eb..7a9e912 100644
> --- a/engine.c
> +++ b/engine.c
> @@ -603,8 +603,6 @@ run_command(const char *cmd, bool errCheck)
>   _exit(1);
>  }
>
> -static Job myjob;
> -
>  void
>  job_attach_node(Job *job, GNode *node)
>  {
> @@ -617,7 +615,7 @@ job_attach_node(Job *job, GNode *node)
>  }
>
>  void
> -job_handle_status(Job *job, int status)
> +handle_job_status(Job *job, int status)
>  {
>   bool silent;
>   int dying;
> @@ -666,19 +664,21 @@ job_handle_status(Job *job, int status)
>   printf(" in target '%s'", job->node->name);
>   if (job->flags & JOB_ERRCHECK) {
>   job->node->built_status = ERROR;
> - /* compute expensive status if we really want it */
> - if ((job->flags & JOB_SILENT) && job == &myjob)
> - determine_expensive_job(job);
>   if (!keepgoing) {
>   if (!silent)
>   printf("\n");
> - job->next = errorJobs;
> - errorJobs = job;
> + job->flags |= JOB_KEEPERROR;
>   /* XXX don't free the command */
>   return;
>   }
>   printf(", line %lu of %s", job->location->lineno,
>      job->location->fname);
> + /* Parallel make already determined whether
> + * JOB_IS_EXPENSIVE, perform the computation for
> + * sequential make to figure out whether to display the
> + * command or not.  */
> + if ((job->flags & JOB_SILENT) && sequential)
> + determine_expensive_job(job);
>   if ((job->flags & (JOB_SILENT | JOB_IS_EXPENSIVE))
>      == JOB_SILENT)
>   printf(": %s", job->cmd);
> @@ -699,17 +699,12 @@ job_handle_status(Job *job, int status)
>  int
>  run_gnode(GNode *gn)
>  {
> + Job *j;
>   if (!gn || (gn->type & OP_DUMMY))
>   return NOSUCHNODE;
>
> - job_attach_node(&myjob, gn);
> - while (myjob.exit_type == JOB_EXIT_OKAY) {
> - bool finished = job_run_next(&myjob);
> - if (finished)
> - break;
> - handle_one_job(&myjob);
> - }
> -
> + Job_Make(gn);
> + loop_handle_running_jobs();
>   return gn->built_status;
>  }
>
> @@ -792,7 +787,7 @@ do_run_command(Job *job, const char *pre)
>   * and there's nothing left to do.
>   */
>   if (random_delay)
> - if (!(runningJobs == NULL && no_jobs_left()))
> + if (!(runningJobs == NULL && nothing_left_to_build()))
>   usleep(arc4random_uniform(random_delay));
>   run_command(cmd, errCheck);
>   /*NOTREACHED*/
> diff --git a/engine.h b/engine.h
> index cb96139..ce7ebd4 100644
> --- a/engine.h
> +++ b/engine.h
> @@ -103,19 +103,20 @@ struct Job_ {
>   struct Job_ *next; /* singly linked list */
>   pid_t pid; /* Current command process id */
>   Location *location;
> - int exit_type; /* last child exit or signal */
> -#define JOB_EXIT_OKAY 0
> -#define JOB_EXIT_BAD 1
> -#define JOB_SIGNALED 2
>   int code; /* exit status or signal code */
> + unsigned short exit_type; /* last child exit or signal */
> +#define JOB_EXIT_OKAY 0
> +#define JOB_EXIT_BAD 1
> +#define JOB_SIGNALED 2
> + unsigned short flags;
> +#define JOB_SILENT 0x001 /* Command was silent */
> +#define JOB_IS_EXPENSIVE 0x002
> +#define JOB_LOST 0x004 /* sent signal to non-existing pid ? */
> +#define JOB_ERRCHECK 0x008 /* command wants errcheck */
> +#define JOB_KEEPERROR 0x010 /* should place job on error list */
>   LstNode next_cmd; /* Next command to run */
>   char *cmd; /* Last command run */
>   GNode *node;     /* Target of this job */
> - unsigned short flags;
> -#define JOB_SILENT 0x001 /* Command was silent */
> -#define JOB_IS_EXPENSIVE 0x002
> -#define JOB_LOST 0x004 /* sent signal to non-existing pid ? */
> -#define JOB_ERRCHECK 0x008 /* command wants errcheck */
>  };
>
>  /* Continuation-style running commands for the parallel engine */
> @@ -131,10 +132,10 @@ extern void job_attach_node(Job *, GNode *);
>   */
>  extern bool job_run_next(Job *);
>
> -/* job_handle_status(job, waitstatus):
> +/* handle_job_status(job, waitstatus):
>   * process a wait return value corresponding to a job, display
>   * messages and set job status accordingly.
>   */
> -extern void job_handle_status(Job *, int);
> +extern void handle_job_status(Job *, int);
>
>  #endif
> diff --git a/enginechoice.c b/enginechoice.c
> new file mode 100644
> index 0000000..32ba046
> --- /dev/null
> +++ b/enginechoice.c
> @@ -0,0 +1,32 @@
> +#include "config.h"
> +#include "defines.h"
> +#include "compat.h"
> +#include "make.h"
> +
> +struct engine {
> + void (*run_list)(Lst, bool *, bool *);
> + void (*node_updated)(GNode *);
> + void (*init)(void);
> +}
> + compat_engine = { Compat_Run, Compat_Update, Compat_Init },
> + parallel_engine = { Make_Run, Make_Update, Make_Init },
> + *engine;
> +
> +void
> +choose_engine(bool compat)
> +{
> + engine = compat ? &compat_engine: &parallel_engine;
> + engine->init();
> +}
> +
> +void
> +engine_run_list(Lst l, bool *has_errors, bool *out_of_date)
> +{
> + engine->run_list(l, has_errors, out_of_date);
> +}
> +
> +void
> +engine_node_updated(GNode *gn)
> +{
> + engine->node_updated(gn);
> +}
> diff --git a/enginechoice.h b/enginechoice.h
> new file mode 100644
> index 0000000..6441231
> --- /dev/null
> +++ b/enginechoice.h
> @@ -0,0 +1,8 @@
> +#ifndef ENGINECHOICE_H
> +#define ENGINECHOICE_H
> +
> +extern void engine_run_list(Lst, bool *, bool *);
> +extern void engine_node_updated(GNode *);
> +extern void choose_engine(bool);
> +
> +#endif
> diff --git a/expandchildren.c b/expandchildren.c
> new file mode 100644
> index 0000000..28a1004
> --- /dev/null
> +++ b/expandchildren.c
> @@ -0,0 +1,246 @@
> +/* $OpenBSD$ */
> +/* $NetBSD: suff.c,v 1.13 1996/11/06 17:59:25 christos Exp $ */
> +
> +/*
> + * Copyright (c) 1988, 1989, 1990, 1993
> + * The Regents of the University of California.  All rights reserved.
> + * Copyright (c) 1989 by Berkeley Softworks
> + * All rights reserved.
> + *
> + * This code is derived from software contributed to Berkeley by
> + * Adam de Boor.
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + * 1. Redistributions of source code must retain the above copyright
> + *    notice, this list of conditions and the following disclaimer.
> + * 2. Redistributions in binary form must reproduce the above copyright
> + *    notice, this list of conditions and the following disclaimer in the
> + *    documentation and/or other materials provided with the distribution.
> + * 3. Neither the name of the University nor the names of its contributors
> + *    may be used to endorse or promote products derived from this software
> + *    without specific prior written permission.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
> + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> + * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
> + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
> + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
> + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
> + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> + * SUCH DAMAGE.
> + */
> +
> +/*-
> + * expandchildren.c --
> + * Dealing with final children expansion before building stuff
> + */
> +
> +#include <ctype.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include "config.h"
> +#include "defines.h"
> +#include "direxpand.h"
> +#include "engine.h"
> +#include "arch.h"
> +#include "expandchildren.h"
> +#include "var.h"
> +#include "targ.h"
> +#include "lst.h"
> +#include "gnode.h"
> +#include "suff.h"
> +
> +static void ExpandChildren(LstNode, GNode *);
> +static void ExpandVarChildren(LstNode, GNode *, GNode *);
> +static void ExpandWildChildren(LstNode, GNode *, GNode *);
> +
> +void
> +LinkParent(GNode *cgn, GNode *pgn)
> +{
> + Lst_AtEnd(&cgn->parents, pgn);
> + if (!has_been_built(cgn))
> + pgn->children_left++;
> + else if ( ! (cgn->type & (OP_EXEC|OP_USE))) {
> + if (cgn->built_status == REBUILT)
> + pgn->child_rebuilt = true;
> + (void)Make_TimeStamp(pgn, cgn);
> + }
> +}
> +
> +static void
> +ExpandVarChildren(LstNode after, GNode *cgn, GNode *pgn)
> +{
> + GNode *gn; /* New source 8) */
> + char *cp; /* Expanded value */
> + LIST members;
> +
> +
> + if (DEBUG(SUFF))
> + printf("Expanding \"%s\"...", cgn->name);
> +
> + cp = Var_Subst(cgn->name, &pgn->localvars, true);
> + if (cp == NULL) {
> + printf("Problem substituting in %s", cgn->name);
> + printf("\n");
> + return;
> + }
> +
> + Lst_Init(&members);
> +
> + if (cgn->type & OP_ARCHV) {
> + /*
> + * Node was an archive(member) target, so we want to call
> + * on the Arch module to find the nodes for us, expanding
> + * variables in the parent's context.
> + */
> + const char *sacrifice = (const char *)cp;
> +
> + (void)Arch_ParseArchive(&sacrifice, &members, &pgn->localvars);
> + } else {
> + /* Break the result into a vector of strings whose nodes
> + * we can find, then add those nodes to the members list.
> + * Unfortunately, we can't use brk_string because it
> + * doesn't understand about variable specifications with
> + * spaces in them...  */
> + const char *start, *cp2;
> +
> + for (start = cp; *start == ' ' || *start == '\t'; start++)
> + continue;
> + for (cp2 = start; *cp2 != '\0';) {
> + if (ISSPACE(*cp2)) {
> + /* White-space -- terminate element, find the
> + * node, add it, skip any further spaces.  */
> + gn = Targ_FindNodei(start, cp2, TARG_CREATE);
> + cp2++;
> + Lst_AtEnd(&members, gn);
> + while (ISSPACE(*cp2))
> + cp2++;
> + /* Adjust cp2 for increment at start of loop,
> + * but set start to first non-space.  */
> + start = cp2;
> + } else if (*cp2 == '$')
> + /* Start of a variable spec -- contact variable
> + * module to find the end so we can skip over
> + * it.  */
> + Var_ParseSkip(&cp2, &pgn->localvars);
> + else if (*cp2 == '\\' && cp2[1] != '\0')
> + /* Escaped something -- skip over it.  */
> + cp2+=2;
> + else
> + cp2++;
> +    }
> +
> +    if (cp2 != start) {
> +    /* Stuff left over -- add it to the list too.  */
> +    gn = Targ_FindNodei(start, cp2, TARG_CREATE);
> +    Lst_AtEnd(&members, gn);
> +    }
> + }
> + /* Add all elements of the members list to the parent node.  */
> + while ((gn = Lst_DeQueue(&members)) != NULL) {
> + if (DEBUG(SUFF))
> + printf("%s...", gn->name);
> + if (Lst_Member(&pgn->children, gn) == NULL) {
> + Lst_Append(&pgn->children, after, gn);
> + after = Lst_Adv(after);
> + LinkParent(gn, pgn);
> + }
> + }
> + /* Free the result.  */
> + free(cp);
> + if (DEBUG(SUFF))
> + printf("\n");
> +}
> +
> +static void
> +ExpandWildChildren(LstNode after, GNode *cgn, GNode *pgn)
> +{
> + char *cp; /* Expanded value */
> +
> + LIST exp; /* List of expansions */
> + Lst path; /* Search path along which to expand */
> +
> + if (DEBUG(SUFF))
> + printf("Wildcard expanding \"%s\"...", cgn->name);
> +
> + /* Find a path along which to expand the word: if
> + * the word has a known suffix, use the path for that suffix,
> + * otherwise use the default path. */
> + path = find_best_path(cgn->name);
> +
> + /* Expand the word along the chosen path. */
> + Lst_Init(&exp);
> + Dir_Expand(cgn->name, path, &exp);
> +
> + /* Fetch next expansion off the list and find its GNode.  */
> + while ((cp = Lst_DeQueue(&exp)) != NULL) {
> + GNode *gn; /* New source 8) */
> + if (DEBUG(SUFF))
> + printf("%s...", cp);
> + gn = Targ_FindNode(cp, TARG_CREATE);
> +
> + /* If gn isn't already a child of the parent, make it so and
> + * up the parent's count of children to build.  */
> + if (Lst_Member(&pgn->children, gn) == NULL) {
> + Lst_Append(&pgn->children, after, gn);
> + after = Lst_Adv(after);
> + LinkParent(gn, pgn);
> + }
> + }
> +
> + if (DEBUG(SUFF))
> + printf("\n");
> +}
> +
> +/*-
> + *-----------------------------------------------------------------------
> + * ExpandChildren --
> + * Expand the names of any children of a given node that contain
> + * variable invocations or file wildcards into actual targets.
> + *
> + * Side Effects:
> + * The expanded node is removed from the parent's list of children,
> + * and the parent's children to build counter is decremented,
> + *      but other nodes may be added.
> + *-----------------------------------------------------------------------
> + */
> +static void
> +ExpandChildren(LstNode ln, /* LstNode of child, so we can replace it */
> +    GNode *pgn)
> +{
> + GNode *cgn = Lst_Datum(ln);
> +
> + /* First do variable expansion -- this takes precedence over wildcard
> + * expansion. If the result contains wildcards, they'll be gotten to
> + * later since the resulting words are tacked on to the end of the
> + * children list.  */
> + if (strchr(cgn->name, '$') != NULL)
> + ExpandVarChildren(ln, cgn, pgn);
> + else if (Dir_HasWildcards(cgn->name))
> + ExpandWildChildren(ln, cgn, pgn);
> + else
> +    /* Third case: nothing to expand.  */
> + return;
> +
> + /* Since the source was expanded, remove it from the list of children to
> + * keep it from being processed.  */
> + pgn->children_left--;
> + Lst_Remove(&pgn->children, ln);
> +}
> +
> +void
> +expand_children_from(GNode *parent, LstNode from)
> +{
> + LstNode np, ln;
> +
> + for (ln = from; ln != NULL; ln = np) {
> + np = Lst_Adv(ln);
> + ExpandChildren(ln, parent);
> + }
> +}
> diff --git a/expandchildren.h b/expandchildren.h
> new file mode 100644
> index 0000000..2c5212e
> --- /dev/null
> +++ b/expandchildren.h
> @@ -0,0 +1,16 @@
> +#ifndef EXPANDCHILDREN_H
> +#define EXPANDCHILDREN_H
> +/* $OpenBSD: suff.h,v 1.10 2012/12/06 14:30:35 espie Exp $ */
> +
> +extern void LinkParent(GNode *, GNode *);
> +
> +/* partial expansion of children. */
> +extern void expand_children_from(GNode *, LstNode);
> +/* expand_all_children(gn):
> + * figure out all variable/wildcards expansions in gn.
> + * TODO pretty sure this is independent from the main suff module.
> + */
> +#define expand_all_children(gn) \
> +    expand_children_from(gn, Lst_First(&(gn)->children))
> +
> +#endif
> diff --git a/extern.h b/extern.h
> index 89797d6..e461ecb 100644
> --- a/extern.h
> +++ b/extern.h
> @@ -40,7 +40,6 @@
>   * from: @(#)nonints.h 8.3 (Berkeley) 3/19/94
>   */
>
> -extern bool compatMake; /* True if we are make compatible */
>  extern bool ignoreErrors; /* True if should ignore all errors */
>  extern bool beSilent; /* True if should print no commands */
>  extern bool noExecute; /* True if should execute nothing */
> diff --git a/gnode.h b/gnode.h
> index d014d87..283fead 100644
> --- a/gnode.h
> +++ b/gnode.h
> @@ -65,32 +65,34 @@
>   *   to create this target.
>   */
>
> +/* constants for specials
> + * Most of these values are only handled by parse.c.
> + * In many cases, there is a corresponding OP_* flag
> + */
>  #define SPECIAL_NONE 0U
> -#define SPECIAL_PATH 21U
> -#define SPECIAL_MASK 63U
> -#define SPECIAL_TARGET 64U
> -#define SPECIAL_SOURCE 128U
> -#define SPECIAL_TARGETSOURCE (SPECIAL_TARGET|SPECIAL_SOURCE)
> +#define SPECIAL_PATH 62U /* handled by parse.c and suff.c */
>
> -#define SPECIAL_EXEC 4U
> +#define SPECIAL_EXEC 4U
>  #define SPECIAL_IGNORE 5U
> -#define SPECIAL_NOTHING 6U
> -#define SPECIAL_INVISIBLE 8U
> +#define SPECIAL_NOTHING 6U /* this is used for things we
> + * recognize for compatibility but
> + * don't do anything with... */
> +#define SPECIAL_INVISIBLE 8U
>  #define SPECIAL_JOIN 9U
>  #define SPECIAL_MADE 11U
>  #define SPECIAL_MAIN 12U
>  #define SPECIAL_MAKE 13U
>  #define SPECIAL_MFLAGS 14U
> -#define SPECIAL_NOTMAIN 15U
> -#define SPECIAL_NOTPARALLEL 16U
> -#define SPECIAL_OPTIONAL 18U
> +#define SPECIAL_NOTMAIN 15U
> +#define SPECIAL_NOTPARALLEL 16U
> +#define SPECIAL_OPTIONAL 18U
>  #define SPECIAL_ORDER 19U
>  #define SPECIAL_PARALLEL 20U
>  #define SPECIAL_PHONY 22U
>  #define SPECIAL_PRECIOUS 23U
>  #define SPECIAL_SILENT 25U
>  #define SPECIAL_SUFFIXES 27U
> -#define SPECIAL_USE 28U
> +#define SPECIAL_USE 28U
>  #define SPECIAL_WAIT 29U
>  #define SPECIAL_NOPATH 30U
>  #define SPECIAL_ERROR 31U
> diff --git a/job.c b/job.c
> index 1846e6a..0a2180f 100644
> --- a/job.c
> +++ b/job.c
> @@ -70,19 +70,11 @@
>   *
>   * Job_Init Called to initialize this module.
>   *
> - * Job_Begin execute commands attached to the .BEGIN target
> - * if any.
> - *
>   * can_start_job Return true if we can start job
>   *
>   * Job_Empty Return true if the job table is completely
>   * empty.
>   *
> - * Job_Finish Perform any final processing which needs doing.
> - * This includes the execution of any commands
> - * which have been/were attached to the .END
> - * target.
> - *
>   * Job_AbortAll Abort all current jobs. It doesn't
>   * handle output or do anything for the jobs,
>   * just kills them.
> @@ -113,23 +105,24 @@
>  #include "lst.h"
>  #include "gnode.h"
>  #include "memory.h"
> -#include "make.h"
>  #include "buf.h"
> +#include "enginechoice.h"
>
>  static int aborting = 0;    /* why is the make aborting? */
>  #define ABORT_ERROR 1    /* Because of an error */
>  #define ABORT_INTERRUPT 2    /* Because it was interrupted */
>  #define ABORT_WAIT 3    /* Waiting for jobs to finish */
>
> -static int maxJobs; /* The most children we can run at once */
> -static int nJobs; /* Number of jobs already allocated */
>  static bool no_new_jobs; /* Mark recursive shit so we shouldn't start
>   * something else at the same time
>   */
> +bool sequential;
>  Job *runningJobs; /* Jobs currently running a process */
>  Job *errorJobs; /* Jobs in error at end */
> +Job *availableJobs; /* Pool of available jobs */
>  static Job *heldJobs; /* Jobs not running yet because of expensive */
>  static pid_t mypid; /* Used for printing debugging messages */
> +static Job *extra_job; /* Needed for .INTERRUPT */
>
>  static volatile sig_atomic_t got_fatal;
>
> @@ -141,16 +134,11 @@ static sigset_t sigset, emptyset;
>  static void handle_fatal_signal(int);
>  static void handle_siginfo(void);
>  static void postprocess_job(Job *);
> -static Job *prepare_job(GNode *);
>  static void determine_job_next_step(Job *);
> -static void remove_job(Job *);
>  static void may_continue_job(Job *);
> -static void continue_job(Job *);
>  static Job *reap_finished_job(pid_t);
>  static bool reap_jobs(void);
> -static void may_continue_heldback_jobs();
>
> -static void loop_handle_running_jobs(void);
>  static bool expensive_job(Job *);
>  static bool expensive_command(const char *);
>  static void setup_signal(int);
> @@ -542,10 +530,15 @@ postprocess_job(Job *job)
>   * non-zero status that we shouldn't ignore, we call
>   * Make_Update to update the parents. */
>   job->node->built_status = REBUILT;
> - Make_Update(job->node);
> - free(job);
> - } else if (job->exit_type != JOB_EXIT_OKAY && keepgoing)
> - free(job);
> + engine_node_updated(job->node);
> + }
> + if (job->flags & JOB_KEEPERROR) {
> + job->next = errorJobs;
> + errorJobs = job;
> + } else {
> + job->next = availableJobs;
> + availableJobs = job;
> + }
>
>   if (errorJobs != NULL && aborting != ABORT_INTERRUPT)
>   aborting = ABORT_ERROR;
> @@ -569,10 +562,10 @@ postprocess_job(Job *job)
>   * is set, so jobs that would fork new processes are accumulated in the
>   * heldJobs list instead.
>   *
> - * This heuristics is also used on error exit: we display silent commands
> - * that failed, unless those ARE expensive commands: expensive commands
> - * are likely to not be failing by themselves, but to be the result of
> - * a cascade of failures in descendant makes.
> + * XXX This heuristics is also used on error exit: we display silent commands
> + * that failed, unless those ARE expensive commands: expensive commands are
> + * likely to not be failing by themselves, but to be the result of a cascade of
> + * failures in descendant makes.
>   */
>  void
>  determine_expensive_job(Job *job)
> @@ -648,35 +641,6 @@ expensive_command(const char *s)
>   return false;
>  }
>
> -static Job *
> -prepare_job(GNode *gn)
> -{
> - /* a new job is prepared unless its commands are bogus (we don't
> - * have anything for it), or if we're in touch mode.
> - *
> - * Note that even in noexec mode, some commands may still run
> - * thanks to the +cmd construct.
> - */
> - if (node_find_valid_commands(gn)) {
> - if (touchFlag) {
> - Job_Touch(gn);
> - return NULL;
> - } else {
> - Job *job;
> -
> - job = emalloc(sizeof(Job));
> - if (job == NULL)
> - Punt("can't create job: out of memory");
> -
> - job_attach_node(job, gn);
> - return job;
> - }
> - } else {
> - node_failure(gn);
> - return NULL;
> - }
> -}
> -
>  static void
>  may_continue_job(Job *job)
>  {
> @@ -686,18 +650,29 @@ may_continue_job(Job *job)
>      (long)mypid, job->node->name);
>   job->next = heldJobs;
>   heldJobs = job;
> - } else
> - continue_job(job);
> + } else {
> + bool finished = job_run_next(job);
> + if (finished)
> + postprocess_job(job);
> + else if (!sequential)
> + determine_expensive_job(job);
> + }
>  }
>
>  static void
> -continue_job(Job *job)
> +may_continue_heldback_jobs()
>  {
> - bool finished = job_run_next(job);
> - if (finished)
> - remove_job(job);
> - else
> - determine_expensive_job(job);
> + while (!no_new_jobs) {
> + if (heldJobs != NULL) {
> + Job *job = heldJobs;
> + heldJobs = heldJobs->next;
> + if (DEBUG(EXPENSIVE))
> + fprintf(stderr, "[%ld] cheap -> release %s\n",
> +    (long)mypid, job->node->name);
> + may_continue_job(job);
> + } else
> + break;
> + }
>  }
>
>  /*-
> @@ -714,19 +689,17 @@ continue_job(Job *job)
>  void
>  Job_Make(GNode *gn)
>  {
> - Job *job;
> + Job *job = availableJobs;
>
> - job = prepare_job(gn);
> - if (!job)
> - return;
> - nJobs++;
> + assert(job != NULL);
> + availableJobs = availableJobs->next;
> + job_attach_node(job, gn);
>   may_continue_job(job);
>  }
>
>  static void
>  determine_job_next_step(Job *job)
>  {
> - bool okay;
>   if (job->flags & JOB_IS_EXPENSIVE) {
>   no_new_jobs = false;
>   if (DEBUG(EXPENSIVE))
> @@ -737,34 +710,11 @@ determine_job_next_step(Job *job)
>   }
>
>   if (job->exit_type != JOB_EXIT_OKAY || job->next_cmd == NULL)
> - remove_job(job);
> + postprocess_job(job);
>   else
>   may_continue_job(job);
>  }
>
> -static void
> -remove_job(Job *job)
> -{
> - nJobs--;
> - postprocess_job(job);
> -}
> -
> -static void
> -may_continue_heldback_jobs()
> -{
> - while (!no_new_jobs) {
> - if (heldJobs != NULL) {
> - Job *job = heldJobs;
> - heldJobs = heldJobs->next;
> - if (DEBUG(EXPENSIVE))
> - fprintf(stderr, "[%ld] cheap -> release %s\n",
> -    (long)mypid, job->node->name);
> - continue_job(job);
> - } else
> - break;
> - }
> -}
> -
>  /*
>   * job = reap_finished_job(pid):
>   * retrieve and remove a job from runningJobs, based on its pid
> @@ -806,7 +756,7 @@ reap_jobs(void)
>   if (job == NULL) {
>   Punt("Child (%ld) not in table?", (long)pid);
>   } else {
> - job_handle_status(job, status);
> + handle_job_status(job, status);
>   determine_job_next_step(job);
>   }
>   may_continue_heldback_jobs();
> @@ -849,26 +799,6 @@ handle_running_jobs(void)
>  }
>
>  void
> -handle_one_job(Job *job)
> -{
> - int stat;
> - int status;
> - sigset_t old;
> -
> - sigprocmask(SIG_BLOCK, &sigset, &old);
> - while (1) {
> - handle_all_signals();
> - stat = waitpid(job->pid, &status, WNOHANG);
> - if (stat == job->pid)
> - break;
> - sigsuspend(&emptyset);
> - }
> - runningJobs = NULL;
> - job_handle_status(job, status);
> - sigprocmask(SIG_SETMASK, &old, NULL);
> -}
> -
> -static void
>  loop_handle_running_jobs()
>  {
>   while (runningJobs != NULL)
> @@ -876,16 +806,27 @@ loop_handle_running_jobs()
>  }
>
>  void
> -Job_Init(int maxproc)
> +Job_Init(int maxJobs)
>  {
> + Job *j;
> + int i;
> +
>   runningJobs = NULL;
>   heldJobs = NULL;
>   errorJobs = NULL;
> - maxJobs = maxproc;
> + availableJobs = NULL;
> + sequential = maxJobs == 1;
> +
> + /* we allocate n+1 jobs, since we may need an extra job for
> + * running .INTERRUPT.  */
> + j = ereallocarray(NULL, sizeof(Job), maxJobs+1);
> + for (i = 0; i != maxJobs; i++) {
> + j[i].next = availableJobs;
> + availableJobs = &j[i];
> + }
> + extra_job = &j[maxJobs];
>   mypid = getpid();
>
> - nJobs = 0;
> -
>   aborting = 0;
>   setup_all_signals();
>  }
> @@ -893,7 +834,7 @@ Job_Init(int maxproc)
>  bool
>  can_start_job(void)
>  {
> - if (aborting || nJobs >= maxJobs)
> + if (aborting || availableJobs == NULL)
>   return false;
>   else
>   return true;
> @@ -933,7 +874,8 @@ handle_fatal_signal(int signo)
>   if (signo == SIGINT && !touchFlag) {
>   if ((interrupt_node->type & OP_DUMMY) == 0) {
>   ignoreErrors = false;
> -
> + extra_job->next = availableJobs;
> + availableJobs = extra_job;
>   Job_Make(interrupt_node);
>   }
>   }
> @@ -950,40 +892,6 @@ handle_fatal_signal(int signo)
>   exit(1);
>  }
>
> -/*
> - *-----------------------------------------------------------------------
> - * Job_Finish --
> - * Do final processing such as the running of the commands
> - * attached to the .END target.
> - *
> - * return true if fatal errors have happened.
> - *-----------------------------------------------------------------------
> - */
> -bool
> -Job_Finish(void)
> -{
> - bool problem = errorJobs != NULL;
> -
> - if ((end_node->type & OP_DUMMY) == 0) {
> - if (problem) {
> - Error("Errors reported so .END ignored");
> - } else {
> - Job_Make(end_node);
> - loop_handle_running_jobs();
> - }
> - }
> - return problem;
> -}
> -
> -void
> -Job_Begin(void)
> -{
> - if ((begin_node->type & OP_DUMMY) == 0) {
> - Job_Make(begin_node);
> - loop_handle_running_jobs();
> - }
> -}
> -
>  /*-
>   *-----------------------------------------------------------------------
>   * Job_Wait --
> diff --git a/job.h b/job.h
> index e8152d1..5356ee8 100644
> --- a/job.h
> +++ b/job.h
> @@ -65,16 +65,6 @@ extern bool can_start_job(void);
>   */
>  extern bool Job_Empty(void);
>
> -/* errors = Job_Finish();
> - * final processing including running .END target if no errors.
> - */
> -extern bool Job_Finish(void);
> -
> -/* Job_Begin();
> - * similarly, run .BEGIN job at start of job.
> - */
> -extern void Job_Begin(void);
> -
>  extern void Job_Wait(void);
>  extern void Job_AbortAll(void);
>  extern void print_errors(void);
> @@ -84,6 +74,10 @@ extern void print_errors(void);
>   * or a signal coming in.
>   */
>  extern void handle_running_jobs(void);
> +/* loop_handle_running_jobs();
> + * handle running jobs until they're finished.
> + */
> +extern void loop_handle_running_jobs(void);
>
>  /* handle_all_signals();
>   * if a signal was received, react accordingly.
> @@ -93,11 +87,13 @@ extern void handle_running_jobs(void);
>  extern void handle_all_signals(void);
>
>  extern void determine_expensive_job(Job *);
> -extern Job *runningJobs, *errorJobs;
> +extern Job *runningJobs, *errorJobs, *availableJobs;
>  extern void debug_job_printf(const char *, ...);
>  extern void handle_one_job(Job *);
>  extern int check_dying_signal(void);
>
>  extern const char *basedirectory;
>
> +extern bool sequential; /* True if we are running one single-job */
> +
>  #endif /* _JOB_H_ */
> diff --git a/main.c b/main.c
> index cb79cb1..ea0947c 100644
> --- a/main.c
> +++ b/main.c
> @@ -57,15 +57,14 @@
>  #include "pathnames.h"
>  #include "init.h"
>  #include "job.h"
> -#include "compat.h"
>  #include "targ.h"
>  #include "suff.h"
>  #include "str.h"
>  #include "main.h"
>  #include "lst.h"
>  #include "memory.h"
> -#include "make.h"
>  #include "dump.h"
> +#include "enginechoice.h"
>
>  #define MAKEFLAGS ".MAKEFLAGS"
>
> @@ -73,12 +72,12 @@ static LIST to_create; /* Targets to be made */
>  Lst create = &to_create;
>  bool allPrecious; /* .PRECIOUS given on line by itself */
>
> -static bool noBuiltins; /* -r flag */
> -static LIST makefiles; /* ordered list of makefiles to read */
> -static LIST varstoprint; /* list of variables to print */
> -int maxJobs; /* -j argument */
> -bool compatMake; /* -B argument */
> -static bool forceJobs = false;
> +static bool noBuiltins; /* -r flag */
> +static LIST makefiles; /* ordered list of makefiles to read */
> +static LIST varstoprint; /* list of variables to print */
> +static int optj; /* -j argument */
> +static bool compatMake; /* -B argument */
> +static bool forceJobs = false;
>  int debug; /* -d flag */
>  bool noExecute; /* -n flag */
>  bool keepgoing; /* -k flag */
> @@ -126,6 +125,12 @@ record_option(int c, const char *arg)
>   Var_Append(MAKEFLAGS, arg);
>  }
>
> +void
> +set_notparallel()
> +{
> + compatMake = true;
> +}
> +
>  static void
>  posixParseOptLetter(int c)
>  {
> @@ -313,7 +318,7 @@ MainParseArgs(int argc, char **argv)
>   const char *errstr;
>
>   forceJobs = true;
> - maxJobs = strtonum(optarg, 1, INT_MAX, &errstr);
> + optj = strtonum(optarg, 1, INT_MAX, &errstr);
>   if (errstr != NULL) {
>   fprintf(stderr,
>      "make: illegal argument to -j option"
> @@ -623,30 +628,25 @@ read_all_make_rules(bool noBuiltins, bool read_depend,
>   Parse_End();
>  }
>
> +static void
> +run_node(GNode *gn, bool *has_errors, bool *out_of_date)
> +{
> + LIST l;
> +
> + Lst_Init(&l);
> + Lst_AtEnd(&l, gn);
> + engine_run_list(&l, has_errors, out_of_date);
> + Lst_Destroy(&l, NOFREE);
> +}
>
>  int main(int, char **);
> -/*-
> - * main --
> - * The main function, for obvious reasons. Initializes variables
> - * and a few modules, then parses the arguments give it in the
> - * environment and on the command line. Reads the system makefile
> - * followed by either Makefile, makefile or the file given by the
> - * -f argument. Sets the .MAKEFLAGS PMake variable based on all the
> - * flags it has received by then uses either the Make or the Compat
> - * module to create the initial list of targets.
> - *
> - * Results:
> - * If -q was given, exits -1 if anything was out-of-date. Else it exits
> - * 0.
> - *
> - * Side Effects:
> - * The program exits when done. Targets are created. etc. etc. etc.
> - */
> +
>  int
>  main(int argc, char **argv)
>  {
>   static LIST targs; /* target nodes to create */
> - bool outOfDate = true; /* false if all targets up to date */
> + bool outOfDate = false; /* false if all targets up to date */
> + bool errored = false; /* true if errors occurred */
>   char *machine = figure_out_MACHINE();
>   char *machine_arch = figure_out_MACHINE_ARCH();
>   char *machine_cpu = figure_out_MACHINE_CPU();
> @@ -665,6 +665,7 @@ main(int argc, char **argv)
>   Static_Lst_Init(&makefiles);
>   Static_Lst_Init(&varstoprint);
>   Static_Lst_Init(&targs);
> + Static_Lst_Init(&special);
>
>   beSilent = false; /* Print commands as executed */
>   ignoreErrors = false; /* Pay attention to non-zero returns */
> @@ -676,7 +677,7 @@ main(int argc, char **argv)
>   touchFlag = false; /* Actually update targets */
>   debug = 0; /* No debug verbosity, please. */
>
> - maxJobs = DEFMAXJOBS;
> + optj = DEFMAXJOBS;
>   compatMake = false; /* No compat mode */
>
>
> @@ -757,6 +758,9 @@ main(int argc, char **argv)
>
>   read_all_make_rules(noBuiltins, read_depend, &makefiles, &d);
>
> + if (compatMake)
> + optj = 1;
> +
>   Var_Append("MFLAGS", Var_Value(MAKEFLAGS));
>
>   /* Install all the flags into the MAKEFLAGS env variable. */
> @@ -796,25 +800,27 @@ main(int argc, char **argv)
>   else
>   Targ_FindList(&targs, create);
>
> - Job_Init(maxJobs);
> - /* If the user has defined a .BEGIN target, execute the commands
> - * attached to it.  */
> - if (!queryFlag)
> - Job_Begin();
> - if (compatMake)
> - /* Compat_Init will take care of creating all the
> - * targets as well as initializing the module.  */
> - outOfDate = Compat_Run(&targs);
> - else {
> - /* Traverse the graph, checking on all the targets.  */
> - outOfDate = Make_Run(&targs);
> - }
> + choose_engine(compatMake);
> + Job_Init(optj);
> + if (!queryFlag && node_is_real(begin_node))
> + run_node(begin_node, &errored, &outOfDate);
> +
> + if (!errored)
> + engine_run_list(&targs, &errored, &outOfDate);
> +
> + if (!queryFlag && !errored && node_is_real(end_node))
> + run_node(end_node, &errored, &outOfDate);
>   }
>
>   /* print the graph now it's been processed if the user requested it */
>   if (DEBUG(GRAPH2))
>   post_mortem();
>
> + /* Note that we only hit this code if -k is used, otherwise we
> + * exited early in case of errors. */
> + if (errored)
> + Fatal("Errors while building");
> +
>   if (queryFlag && outOfDate)
>   return 1;
>   else
> diff --git a/main.h b/main.h
> index 469487e..3f5ce13 100644
> --- a/main.h
> +++ b/main.h
> @@ -38,4 +38,7 @@ extern void Main_ParseArgLine(const char *);
>   * .if make(...) statements. */
>  extern Lst create;
>
> +/* set_notparallel(): used to influence running mode from parse.c */
> +extern void set_notparallel(void);
> +
>  #endif
> diff --git a/make.c b/make.c
> index 4ffdda6..1eff38e 100644
> --- a/make.c
> +++ b/make.c
> @@ -71,6 +71,7 @@
>  #include "suff.h"
>  #include "var.h"
>  #include "error.h"
> +#include "expandchildren.h"
>  #include "make.h"
>  #include "gnode.h"
>  #include "extern.h"
> @@ -118,7 +119,7 @@ static bool randomize_queue;
>  long random_delay = 0;
>
>  bool
> -no_jobs_left()
> +nothing_left_to_build()
>  {
>   return Array_IsEmpty(&to_build);
>  }
> @@ -376,7 +377,13 @@ try_to_make_node(GNode *gn)
>   return true;
>   /* SIB: this is where commands should get prepared */
>   Make_DoAllVar(gn);
> - Job_Make(gn);
> + if (node_find_valid_commands(gn)) {
> + if (touchFlag)
> + Job_Touch(gn);
> + else
> + Job_Make(gn);
> + } else
> + node_failure(gn);
>   } else {
>   if (DEBUG(MAKE))
>   printf("up-to-date\n");
> @@ -504,6 +511,16 @@ add_targets_to_make(Lst todo)
>   randomize_garray(&to_build);
>  }
>
> +void
> +Make_Init()
> +{
> + /* wild guess at initial sizes */
> + Array_Init(&to_build, 500);
> + Array_Init(&examine, 150);
> + Array_Init(&heldBack, 100);
> + ohash_init(&targets, 10, &gnode_info);
> +}
> +
>  /*-
>   *-----------------------------------------------------------------------
>   * Make_Run --
> @@ -516,25 +533,15 @@ add_targets_to_make(Lst todo)
>   * calling on MakeStartJobs to keep the job table as full as
>   * possible.
>   *
> - * Results:
> - * true if work was done. false otherwise.
> - *
>   * Side Effects:
>   * The must_make field of all nodes involved in the creation of the given
>   * targets is set to 1. The to_build list is set to contain all the
>   * 'leaves' of these subgraphs.
>   *-----------------------------------------------------------------------
>   */
> -bool
> -Make_Run(Lst targs) /* the initial list of targets */
> +void
> +Make_Run(Lst targs, bool *has_errors, bool *out_of_date)
>  {
> - bool problem; /* errors occurred */
> -
> - /* wild guess at initial sizes */
> - Array_Init(&to_build, 500);
> - Array_Init(&examine, 150);
> - Array_Init(&heldBack, 100);
> - ohash_init(&targets, 10, &gnode_info);
>   if (DEBUG(PARALLEL))
>   random_setup();
>
> @@ -545,7 +552,8 @@ Make_Run(Lst targs) /* the initial list of targets */
>   * the next loop... (we won't actually start any, of course,
>   * this is just to see if any of the targets was out of date)
>   */
> - return MakeStartJobs();
> + if (MakeStartJobs())
> + *out_of_date = true;
>   } else {
>   /*
>   * Initialization. At the moment, no jobs are running and until
> @@ -572,8 +580,8 @@ Make_Run(Lst targs) /* the initial list of targets */
>   (void)MakeStartJobs();
>   }
>
> - if (!queryFlag)
> - problem = Job_Finish();
> + if (errorJobs != NULL)
> + *has_errors = true;
>
>   /*
>   * Print the final status of each target. E.g. if it wasn't made
> @@ -581,13 +589,9 @@ Make_Run(Lst targs) /* the initial list of targets */
>   */
>   if (targets_contain_cycles()) {
>   break_and_print_cycles(targs);
> - problem = true;
> + *has_errors = true;
>   }
>   Lst_Every(targs, MakePrintStatus);
> - if (problem)
> - Fatal("Errors while building");
> -
> - return true;
>  }
>
>  /* round-about detection: assume make is bug-free, if there are targets
> diff --git a/make.h b/make.h
> index 91839fb..cb8e09e 100644
> --- a/make.h
> +++ b/make.h
> @@ -41,8 +41,9 @@
>   */
>
>  extern void Make_Update(GNode *);
> -extern bool Make_Run(Lst);
> +extern void Make_Run(Lst, bool *, bool *);
> +extern void Make_Init(void);
>  extern long random_delay;
> -extern bool no_jobs_left(void);
> +extern bool nothing_left_to_build(void);
>
>  #endif /* _MAKE_H_ */
> diff --git a/parse.c b/parse.c
> index 4c90d18..dfc2abc 100644
> --- a/parse.c
> +++ b/parse.c
> @@ -181,40 +181,40 @@ static struct {
>   const char *keyword;
>   size_t sz;
>   uint32_t hv;
> - unsigned int type;
> + unsigned int special;
>   unsigned int special_op;
>  } specials[] = {
> -    { P(NODE_EXEC), SPECIAL_EXEC | SPECIAL_TARGETSOURCE, OP_EXEC, },
> -    { P(NODE_IGNORE), SPECIAL_IGNORE | SPECIAL_TARGETSOURCE, OP_IGNORE, },
> -    { P(NODE_INCLUDES), SPECIAL_NOTHING | SPECIAL_TARGET, 0, },
> -    { P(NODE_INVISIBLE),SPECIAL_INVISIBLE | SPECIAL_TARGETSOURCE,OP_INVISIBLE, },
> -    { P(NODE_JOIN), SPECIAL_JOIN | SPECIAL_TARGETSOURCE, OP_JOIN, },
> -    { P(NODE_LIBS), SPECIAL_NOTHING | SPECIAL_TARGET, 0, },
> -    { P(NODE_MADE), SPECIAL_MADE | SPECIAL_TARGETSOURCE, OP_MADE, },
> -    { P(NODE_MAIN), SPECIAL_MAIN | SPECIAL_TARGET, 0, },
> -    { P(NODE_MAKE), SPECIAL_MAKE | SPECIAL_TARGETSOURCE, OP_MAKE, },
> -    { P(NODE_MAKEFLAGS), SPECIAL_MFLAGS | SPECIAL_TARGET, 0, },
> -    { P(NODE_MFLAGS), SPECIAL_MFLAGS | SPECIAL_TARGET, 0, },
> -    { P(NODE_NOTMAIN), SPECIAL_NOTMAIN | SPECIAL_TARGETSOURCE, OP_NOTMAIN, },
> -    { P(NODE_NOTPARALLEL),SPECIAL_NOTPARALLEL | SPECIAL_TARGET, 0, },
> -    { P(NODE_NO_PARALLEL),SPECIAL_NOTPARALLEL | SPECIAL_TARGET, 0, },
> -    { P(NODE_NULL), SPECIAL_NOTHING | SPECIAL_TARGET, 0, },
> -    { P(NODE_OPTIONAL), SPECIAL_OPTIONAL | SPECIAL_TARGETSOURCE,OP_OPTIONAL, },
> -    { P(NODE_ORDER), SPECIAL_ORDER | SPECIAL_TARGET, 0, },
> -    { P(NODE_PARALLEL), SPECIAL_PARALLEL | SPECIAL_TARGET, 0, },
> -    { P(NODE_PATH), SPECIAL_PATH | SPECIAL_TARGET, 0, },
> -    { P(NODE_PHONY), SPECIAL_PHONY | SPECIAL_TARGETSOURCE, OP_PHONY, },
> -    { P(NODE_PRECIOUS), SPECIAL_PRECIOUS | SPECIAL_TARGETSOURCE,OP_PRECIOUS, },
> -    { P(NODE_RECURSIVE),SPECIAL_MAKE | SPECIAL_TARGETSOURCE, OP_MAKE, },
> -    { P(NODE_SILENT), SPECIAL_SILENT | SPECIAL_TARGETSOURCE, OP_SILENT, },
> -    { P(NODE_SINGLESHELL),SPECIAL_NOTHING | SPECIAL_TARGET, 0, },
> -    { P(NODE_SUFFIXES), SPECIAL_SUFFIXES | SPECIAL_TARGET, 0, },
> -    { P(NODE_USE), SPECIAL_USE | SPECIAL_TARGETSOURCE, OP_USE, },
> -    { P(NODE_WAIT), SPECIAL_WAIT | SPECIAL_TARGETSOURCE, 0 },
> -    { P(NODE_CHEAP), SPECIAL_CHEAP | SPECIAL_TARGETSOURCE, OP_CHEAP, },
> -    { P(NODE_EXPENSIVE),SPECIAL_EXPENSIVE | SPECIAL_TARGETSOURCE,OP_EXPENSIVE, },
> -    { P(NODE_POSIX), SPECIAL_NOTHING | SPECIAL_TARGET, 0 },
> -    { P(NODE_SCCS_GET), SPECIAL_NOTHING | SPECIAL_TARGET, 0 },
> +    { P(NODE_EXEC), SPECIAL_EXEC, OP_EXEC },
> +    { P(NODE_IGNORE), SPECIAL_IGNORE, OP_IGNORE },
> +    { P(NODE_INCLUDES), SPECIAL_NOTHING, 0 },
> +    { P(NODE_INVISIBLE), SPECIAL_INVISIBLE, OP_INVISIBLE },
> +    { P(NODE_JOIN), SPECIAL_JOIN, OP_JOIN },
> +    { P(NODE_LIBS), SPECIAL_NOTHING, 0 },
> +    { P(NODE_MADE), SPECIAL_MADE, OP_MADE },
> +    { P(NODE_MAIN), SPECIAL_MAIN, 0 },
> +    { P(NODE_MAKE), SPECIAL_MAKE, OP_MAKE },
> +    { P(NODE_MAKEFLAGS), SPECIAL_MFLAGS, 0 },
> +    { P(NODE_MFLAGS), SPECIAL_MFLAGS, 0 },
> +    { P(NODE_NOTMAIN), SPECIAL_NOTMAIN, OP_NOTMAIN },
> +    { P(NODE_NOTPARALLEL), SPECIAL_NOTPARALLEL, 0 },
> +    { P(NODE_NO_PARALLEL), SPECIAL_NOTPARALLEL, 0 },
> +    { P(NODE_NULL), SPECIAL_NOTHING, 0 },
> +    { P(NODE_OPTIONAL), SPECIAL_OPTIONAL, OP_OPTIONAL },
> +    { P(NODE_ORDER), SPECIAL_ORDER, 0 },
> +    { P(NODE_PARALLEL), SPECIAL_PARALLEL, 0 },
> +    { P(NODE_PATH), SPECIAL_PATH, 0 },
> +    { P(NODE_PHONY), SPECIAL_PHONY, OP_PHONY },
> +    { P(NODE_PRECIOUS), SPECIAL_PRECIOUS, OP_PRECIOUS },
> +    { P(NODE_RECURSIVE), SPECIAL_MAKE, OP_MAKE },
> +    { P(NODE_SILENT), SPECIAL_SILENT, OP_SILENT },
> +    { P(NODE_SINGLESHELL), SPECIAL_NOTHING, 0 },
> +    { P(NODE_SUFFIXES), SPECIAL_SUFFIXES, 0 },
> +    { P(NODE_USE), SPECIAL_USE, OP_USE },
> +    { P(NODE_WAIT), SPECIAL_WAIT, 0 },
> +    { P(NODE_CHEAP), SPECIAL_CHEAP, OP_CHEAP },
> +    { P(NODE_EXPENSIVE), SPECIAL_EXPENSIVE, OP_EXPENSIVE },
> +    { P(NODE_POSIX), SPECIAL_NOTHING, 0 },
> +    { P(NODE_SCCS_GET), SPECIAL_NOTHING, 0 },
>  };
>
>  #undef P
> @@ -225,10 +225,9 @@ create_special_nodes()
>   unsigned int i;
>
>   for (i = 0; i < sizeof(specials)/sizeof(specials[0]); i++) {
> - GNode *gn = Targ_FindNodeh(specials[i].keyword,
> -    specials[i].sz, specials[i].hv, TARG_CREATE);
> - gn->special = specials[i].type;
> - gn->special_op = specials[i].special_op;
> + (void)Targ_mk_special_node(specials[i].keyword,
> +    specials[i].sz, specials[i].hv,
> +    OP_ZERO, specials[i].special, specials[i].special_op);
>   }
>  }
>
> @@ -419,15 +418,13 @@ ParseDoSrc(
>      const char *esrc)
>  {
>   GNode *gn = Targ_FindNodei(src, esrc, TARG_CREATE);
> - if ((gn->special & SPECIAL_SOURCE) != 0) {
> - if (gn->special_op) {
> - Array_ForEach(targets, ParseDoSpecial, gn->special_op);
> - return;
> - } else {
> - assert((gn->special & SPECIAL_MASK) == SPECIAL_WAIT);
> - waiting++;
> - return;
> - }
> + if (gn->special_op) {
> + Array_ForEach(targets, ParseDoSpecial, gn->special_op);
> + return;
> + }
> + if (gn->special == SPECIAL_WAIT) {
> + waiting++;
> + return;
>   }
>
>   switch (specType) {
> @@ -705,10 +702,10 @@ handle_special_targets(Lst paths)
>
>   for (i = 0; i < gtargets.n; i++) {
>   type = gtargets.a[i]->special;
> - if ((type & SPECIAL_MASK) == SPECIAL_PATH) {
> + if (type == SPECIAL_PATH) {
>   seen_path++;
>   Lst_AtEnd(paths, find_suffix_path(gtargets.a[i]));
> - } else if ((type & SPECIAL_TARGET) != 0)
> + } else if (type != 0)
>   seen_special++;
>   else
>   seen_normal++;
> @@ -734,7 +731,7 @@ handle_special_targets(Lst paths)
>   dump_targets();
>   return 0;
>   } else if (seen_special == 1) {
> - specType = gtargets.a[0]->special & SPECIAL_MASK;
> + specType = gtargets.a[0]->special;
>   switch (specType) {
>   case SPECIAL_MAIN:
>   if (!Lst_IsEmpty(create)) {
> @@ -742,13 +739,8 @@ handle_special_targets(Lst paths)
>   }
>   break;
>   case SPECIAL_NOTPARALLEL:
> - {
> - extern int  maxJobs;
> -
> - maxJobs = 1;
> - compatMake = 1;
> + set_notparallel();
>   break;
> - }
>   case SPECIAL_ORDER:
>   predecessor = NULL;
>   break;
> @@ -838,6 +830,7 @@ ParseDoDependency(const char *line) /* the line to parse */
>   Array_Reset(&gsources);
>
>   cp = parse_do_targets(&paths, &tOp, line);
> + assert(specType == SPECIAL_PATH || Lst_IsEmpty(&paths));
>   if (cp == NULL || specType == SPECIAL_ERROR) {
>   /* invalidate targets for further processing */
>   Array_Reset(&gtargets);
> @@ -856,19 +849,15 @@ ParseDoDependency(const char *line) /* the line to parse */
>
>   line = cp;
>
> - /*
> - * Several special targets take different actions if present with no
> - * sources:
> - * a .SUFFIXES line with no sources clears out all old suffixes
> - * a .PRECIOUS line makes all targets precious
> - * a .IGNORE line ignores errors for all targets
> - * a .SILENT line creates silence when making all targets
> - * a .PATH removes all directories from the search path(s).
> - */
> + /* Several special targets have specific semantics with no source:
> + * .SUFFIXES clears out all old suffixes
> + * .PRECIOUS/.IGNORE/.SILENT
> + * apply to all target
> + * .PATH clears out all search paths.  */
>   if (!*line) {
>   switch (specType) {
>   case SPECIAL_SUFFIXES:
> - Suff_ClearSuffixes();
> + Suff_DisableAllSuffixes();
>   break;
>   case SPECIAL_PRECIOUS:
>   allPrecious = true;
> @@ -886,42 +875,26 @@ ParseDoDependency(const char *line) /* the line to parse */
>   break;
>   }
>   } else if (specType == SPECIAL_MFLAGS) {
> - /* Call on functions in main.c to deal with these arguments */
>   Main_ParseArgLine(line);
>   return;
>   } else if (specType == SPECIAL_NOTPARALLEL) {
>   return;
>   }
>
> - /*
> - * NOW GO FOR THE SOURCES
> - */
> + /* NOW GO FOR THE SOURCES */
>   if (specType == SPECIAL_SUFFIXES || specType == SPECIAL_PATH ||
>      specType == SPECIAL_NOTHING) {
>   while (*line) {
> -    /*
> -     * If the target was one that doesn't take files as its
> -     * sources but takes something like suffixes, we take each
> -     * space-separated word on the line as a something and deal
> -     * with it accordingly.
> -     *
> -     * If the target was .SUFFIXES, we take each source as a
> -     * suffix and add it to the list of suffixes maintained by
> -     * the Suff module.
> -     *
> -     * If the target was a .PATH, we add the source as a
> -     * directory to search on the search path.
> +    /* Some special targets take a list of space-separated
> +     * words.  For each word,
>       *
> -     * If it was .INCLUDES, the source is taken to be the
> -     * suffix of files which will be #included and whose search
> -     * path should be present in the .INCLUDES variable.
> +     * if .SUFFIXES, add it to the list of suffixes maintained
> +     * by suff.c.
>       *
> -     * If it was .LIBS, the source is taken to be the suffix of
> -     * files which are considered libraries and whose search
> -     * path should be present in the .LIBS variable.
> +     * if .PATHS, add it as a directory on the main search path.
>       *
> -     * If it was .NULL, the source is the suffix to use when a
> -     * file has no valid suffix.
> +     * if .LIBS/.INCLUDE/.NULL... this has been deprecated,
> +     * ignore
>       */
>      while (*cp && !ISSPACE(*cp))
>      cp++;
> @@ -937,6 +910,7 @@ ParseDoDependency(const char *line) /* the line to parse */
>       ln = Lst_Adv(ln))
>      Dir_AddDiri(Lst_Datum(ln), line, cp);
>      break;
> +    Lst_Destroy(&paths, NOFREE);
>      }
>      default:
>      break;
> @@ -947,7 +921,6 @@ ParseDoDependency(const char *line) /* the line to parse */
>   cp++;
>      line = cp;
>   }
> - Lst_Destroy(&paths, NOFREE);
>   } else {
>   while (*line) {
>   /*
> @@ -1642,12 +1615,12 @@ Parse_File(const char *filename, FILE *stream)
>   bool expectingCommands = false;
>   bool commands_seen = false;
>
> - /* somewhat permanent spaces to shave time */
> - BUFFER buf;
> - BUFFER copy;
> + /* permanent spaces to shave time */
> + static BUFFER buf;
> + static BUFFER copy;
>
> - Buf_Init(&buf, MAKE_BSIZE);
> - Buf_Init(&copy, MAKE_BSIZE);
> + Buf_Reinit(&buf, MAKE_BSIZE);
> + Buf_Reinit(&copy, MAKE_BSIZE);
>
>   Parse_FromFile(filename, stream);
>   do {
> @@ -1687,8 +1660,6 @@ Parse_File(const char *filename, FILE *stream)
>   Cond_End();
>
>   Parse_ReportErrors();
> - Buf_Destroy(&buf);
> - Buf_Destroy(&copy);
>  }
>
>  void
> diff --git a/suff.c b/suff.c
> index 94f0160..e03b745 100644
> --- a/suff.c
> +++ b/suff.c
> @@ -39,28 +39,9 @@
>   * suff.c --
>   * Functions to maintain suffix lists and find implicit dependents
>   * using suffix transformation rules
> - *
> - * Interface:
> - * Suff_Init Initialize all things to do with suffixes.
> - *
> - * Suff_ClearSuffixes Clear out all the suffixes.
> - *
> - * Suff_AddSuffix Add the passed string as another known suffix.
> - *
> - * Suff_ParseAsTransform Line might be a suffix line, check it.
> - * If it's not, return NULL. Otherwise, add
> - * another transformation to the suffix graph.
> - * Returns GNode suitable for framing, I mean,
> - * tacking commands, attributes, etc. on.
> - *
> - * Suff_FindDeps Find implicit sources for and the location of
> - * a target based on its suffix. Returns the
> - * bottom-most node added to the graph or NULL
> - * if the target had no implicit sources.
>   */
>
>  #include <ctype.h>
> -#include <signal.h>
>  #include <stddef.h>
>  #include <stdint.h>
>  #include <stdio.h>
> @@ -70,9 +51,7 @@
>  #include "config.h"
>  #include "defines.h"
>  #include "dir.h"
> -#include "direxpand.h"
>  #include "engine.h"
> -#include "arch.h"
>  #include "suff.h"
>  #include "var.h"
>  #include "targ.h"
> @@ -81,9 +60,9 @@
>  #include "lst.h"
>  #include "memory.h"
>  #include "gnode.h"
> -#include "make.h"
>  #include "stats.h"
>  #include "dump.h"
> +#include "expandchildren.h"
>
>  /* XXX the suffixes hash is stored using a specific hash function, suitable
>   * for looking up suffixes in reverse.
> @@ -91,7 +70,7 @@
>  static struct ohash suffixes;
>
>  /* We remember the longest suffix, so we don't need to look beyond that.  */
> -size_t maxLen;
> +size_t maxLen = 0U;
>  static LIST srclist;
>
>  /* Transforms (.c.o) are stored in another hash, independently from suffixes.
> @@ -172,7 +151,6 @@ static Suff *new_suffixi(const char *, const char *);
>  static void reverse_hash_add_char(uint32_t *, const char *);
>  static uint32_t reverse_hashi(const char *, const char **);
>  static unsigned int reverse_slot(struct ohash *, const char *, const char **);
> -static void clear_suffixes(void);
>  static void record_possible_suffix(Suff *, GNode *, char *, Lst, Lst);
>  static void record_possible_suffixes(GNode *, Lst, Lst);
>  static Suff *find_suffix_as_suffix(Lst, const char *, const char *);
> @@ -184,9 +162,6 @@ static bool SuffRemoveSrc(Lst);
>  static void SuffAddLevel(Lst, Src *);
>  static Src *SuffFindThem(Lst, Lst);
>  static Src *SuffFindCmds(Src *, Lst);
> -static void SuffExpandChildren(LstNode, GNode *);
> -static void SuffExpandVarChildren(LstNode, GNode *, GNode *);
> -static void SuffExpandWildChildren(LstNode, GNode *, GNode *);
>  static bool SuffApplyTransform(GNode *, GNode *, Suff *, Suff *);
>  static void SuffFindDeps(GNode *, Lst);
>  static void SuffFindArchiveDeps(GNode *, Lst);
> @@ -348,18 +323,11 @@ SuffInsert(Lst l, Suff *s)
>   }
>  }
>
> -/*-
> - *-----------------------------------------------------------------------
> - * Suff_ClearSuffixes --
> - * Nuke the list of suffixes but keep all transformation
> - * rules around.
> - *
> - * Side Effects:
> - * Current suffixes are reset
> - *-----------------------------------------------------------------------
> - */
> -static void
> -clear_suffixes(void)
> +/* Suff_DisableAllSuffixes
> + * mark all current suffixes as inactive, and reset precedence
> + * computation.  */
> +void
> +Suff_DisableAllSuffixes(void)
>  {
>   unsigned int i;
>   Suff *s;
> @@ -372,12 +340,6 @@ clear_suffixes(void)
>   maxLen = 0;
>  }
>
> -void
> -Suff_ClearSuffixes(void)
> -{
> - clear_suffixes();
> -}
> -
>
>  /* okay = parse_transform(str, &src, &targ);
>   * try parsing a string as a transformation rule, returns true if
> @@ -488,6 +450,18 @@ find_best_suffix(const char *s, const char *e)
>   return best;
>  }
>
> +Lst
> +find_best_path(const char *name)
> +{
> + Suff *s = find_best_suffix(name, name + strlen(name));
> + if (s != NULL) {
> + if (DEBUG(SUFF))
> + printf("suffix is \"%s\"...", s->name);
> + return &s->searchPath;
> + } else
> + return defaultPath;
> +}
> +
>  /*-
>   *-----------------------------------------------------------------------
>   * Suff_ParseAsTransform --
> @@ -523,7 +497,7 @@ Suff_ParseAsTransform(const char *line, const char *end)
>
>   gn->type = OP_TRANSFORM;
>   if (s->flags & SUFF_PATH) {
> - gn->special = SPECIAL_PATH | SPECIAL_TARGET;
> + gn->special = SPECIAL_PATH;
>   gn->suffix = t;
>   }
>
> @@ -612,7 +586,7 @@ build_suffixes_graph(void)
>      gn = ohash_next(&transforms, &i)) {
>       if (Lst_IsEmpty(&gn->commands) && Lst_IsEmpty(&gn->children))
>   continue;
> - if ((gn->special & SPECIAL_MASK) == SPECIAL_PATH)
> + if (gn->special == SPECIAL_PATH)
>   continue;
>       if (parse_transform(gn->name, &s, &s2)) {
>   SuffInsert(&s2->children, s);
> @@ -629,11 +603,7 @@ build_suffixes_graph(void)
>   *
>   * Side Effects:
>   * The searchPath field of all the suffixes is extended by the
> - * directories in defaultPath. If paths were specified for the
> - * ".h" suffix, the directories are stuffed into a global variable
> - * called ".INCLUDES" with each directory preceded by a -I. The same
> - * is done for the ".a" suffix, except the variable is called
> - * ".LIBS" and the flag is -L.
> + * directories in defaultPath.
>   *-----------------------------------------------------------------------
>   */
>  static void
> @@ -912,203 +882,6 @@ SuffFindCmds(Src *targ, Lst slst)
>   return NULL;
>  }
>
> -static void
> -SuffLinkParent(GNode *cgn, GNode *pgn)
> -{
> - Lst_AtEnd(&cgn->parents, pgn);
> - if (!has_been_built(cgn))
> - pgn->children_left++;
> - else if ( ! (cgn->type & (OP_EXEC|OP_USE))) {
> - if (cgn->built_status == REBUILT)
> - pgn->child_rebuilt = true;
> - (void)Make_TimeStamp(pgn, cgn);
> - }
> -}
> -
> -static void
> -SuffExpandVarChildren(LstNode after, GNode *cgn, GNode *pgn)
> -{
> - GNode *gn; /* New source 8) */
> - char *cp; /* Expanded value */
> - LIST members;
> -
> -
> - if (DEBUG(SUFF))
> - printf("Expanding \"%s\"...", cgn->name);
> -
> - cp = Var_Subst(cgn->name, &pgn->localvars, true);
> - if (cp == NULL) {
> - printf("Problem substituting in %s", cgn->name);
> - printf("\n");
> - return;
> - }
> -
> - Lst_Init(&members);
> -
> - if (cgn->type & OP_ARCHV) {
> - /*
> - * Node was an archive(member) target, so we want to call
> - * on the Arch module to find the nodes for us, expanding
> - * variables in the parent's context.
> - */
> - const char *sacrifice = (const char *)cp;
> -
> - (void)Arch_ParseArchive(&sacrifice, &members, &pgn->localvars);
> - } else {
> - /* Break the result into a vector of strings whose nodes
> - * we can find, then add those nodes to the members list.
> - * Unfortunately, we can't use brk_string because it
> - * doesn't understand about variable specifications with
> - * spaces in them...  */
> - const char *start, *cp2;
> -
> - for (start = cp; *start == ' ' || *start == '\t'; start++)
> - continue;
> - for (cp2 = start; *cp2 != '\0';) {
> - if (ISSPACE(*cp2)) {
> - /* White-space -- terminate element, find the
> - * node, add it, skip any further spaces.  */
> - gn = Targ_FindNodei(start, cp2, TARG_CREATE);
> - cp2++;
> - Lst_AtEnd(&members, gn);
> - while (ISSPACE(*cp2))
> - cp2++;
> - /* Adjust cp2 for increment at start of loop,
> - * but set start to first non-space.  */
> - start = cp2;
> - } else if (*cp2 == '$')
> - /* Start of a variable spec -- contact variable
> - * module to find the end so we can skip over
> - * it.  */
> - Var_ParseSkip(&cp2, &pgn->localvars);
> - else if (*cp2 == '\\' && cp2[1] != '\0')
> - /* Escaped something -- skip over it.  */
> - cp2+=2;
> - else
> - cp2++;
> -    }
> -
> -    if (cp2 != start) {
> -    /* Stuff left over -- add it to the list too.  */
> -    gn = Targ_FindNodei(start, cp2, TARG_CREATE);
> -    Lst_AtEnd(&members, gn);
> -    }
> - }
> - /* Add all elements of the members list to the parent node.  */
> - while ((gn = Lst_DeQueue(&members)) != NULL) {
> - if (DEBUG(SUFF))
> - printf("%s...", gn->name);
> - if (Lst_Member(&pgn->children, gn) == NULL) {
> - Lst_Append(&pgn->children, after, gn);
> - after = Lst_Adv(after);
> - SuffLinkParent(gn, pgn);
> - }
> - }
> - /* Free the result.  */
> - free(cp);
> - if (DEBUG(SUFF))
> - printf("\n");
> -}
> -
> -static void
> -SuffExpandWildChildren(LstNode after, GNode *cgn, GNode *pgn)
> -{
> - Suff *s;
> - char *cp; /* Expanded value */
> -
> - LIST exp; /* List of expansions */
> - Lst path; /* Search path along which to expand */
> -
> - if (DEBUG(SUFF))
> - printf("Wildcard expanding \"%s\"...", cgn->name);
> -
> - /* Find a path along which to expand the word.
> - *
> - * If the word has a known suffix, use that path.
> - * If it has no known suffix and we're allowed to use the null
> - * suffix, use its path.
> - * Else use the default system search path.  */
> - s = find_best_suffix(cgn->name, cgn->name + strlen(cgn->name));
> -
> - if (s != NULL) {
> - if (DEBUG(SUFF))
> - printf("suffix is \"%s\"...", s->name);
> - path = &s->searchPath;
> - } else
> - /* Use default search path.  */
> - path = defaultPath;
> -
> - /* Expand the word along the chosen path. */
> - Lst_Init(&exp);
> - Dir_Expand(cgn->name, path, &exp);
> -
> - /* Fetch next expansion off the list and find its GNode.  */
> - while ((cp = Lst_DeQueue(&exp)) != NULL) {
> - GNode *gn; /* New source 8) */
> - if (DEBUG(SUFF))
> - printf("%s...", cp);
> - gn = Targ_FindNode(cp, TARG_CREATE);
> -
> - /* If gn isn't already a child of the parent, make it so and
> - * up the parent's count of children to build.  */
> - if (Lst_Member(&pgn->children, gn) == NULL) {
> - Lst_Append(&pgn->children, after, gn);
> - after = Lst_Adv(after);
> - SuffLinkParent(gn, pgn);
> - }
> - }
> -
> - if (DEBUG(SUFF))
> - printf("\n");
> -}
> -
> -/*-
> - *-----------------------------------------------------------------------
> - * SuffExpandChildren --
> - * Expand the names of any children of a given node that contain
> - * variable invocations or file wildcards into actual targets.
> - *
> - * Side Effects:
> - * The expanded node is removed from the parent's list of children,
> - * and the parent's children to build counter is decremented,
> - *      but other nodes may be added.
> - *-----------------------------------------------------------------------
> - */
> -static void
> -SuffExpandChildren(LstNode ln, /* LstNode of child, so we can replace it */
> -    GNode *pgn)
> -{
> - GNode *cgn = Lst_Datum(ln);
> -
> - /* First do variable expansion -- this takes precedence over wildcard
> - * expansion. If the result contains wildcards, they'll be gotten to
> - * later since the resulting words are tacked on to the end of the
> - * children list.  */
> - if (strchr(cgn->name, '$') != NULL)
> - SuffExpandVarChildren(ln, cgn, pgn);
> - else if (Dir_HasWildcards(cgn->name))
> - SuffExpandWildChildren(ln, cgn, pgn);
> - else
> -    /* Third case: nothing to expand.  */
> - return;
> -
> - /* Since the source was expanded, remove it from the list of children to
> - * keep it from being processed.  */
> - pgn->children_left--;
> - Lst_Remove(&pgn->children, ln);
> -}
> -
> -void
> -expand_children_from(GNode *parent, LstNode from)
> -{
> - LstNode np, ln;
> -
> - for (ln = from; ln != NULL; ln = np) {
> - np = Lst_Adv(ln);
> - SuffExpandChildren(ln, parent);
> - }
> -}
> -
>  /*-
>   *-----------------------------------------------------------------------
>   * SuffApplyTransform --
> @@ -1140,7 +913,7 @@ SuffApplyTransform(
>   if (Lst_AddNew(&tGn->children, sGn)) {
>   /* Not already linked, so form the proper links between the
>   * target and source.  */
> - SuffLinkParent(sGn, tGn);
> + LinkParent(sGn, tGn);
>   }
>
>   if ((sGn->type & OP_OPMASK) == OP_DOUBLEDEP) {
> @@ -1154,7 +927,7 @@ SuffApplyTransform(
>   if (Lst_AddNew(&tGn->children, gn)) {
>   /* Not already linked, so form the proper links
>   * between the target and source.  */
> - SuffLinkParent(gn, tGn);
> + LinkParent(gn, tGn);
>   }
>   }
>   }
> @@ -1247,7 +1020,7 @@ SuffFindArchiveDeps(
>
>   /* Create the link between the two nodes right off. */
>   if (Lst_AddNew(&gn->children, mem))
> - SuffLinkParent(mem, gn);
> + LinkParent(mem, gn);
>
>   /* Copy variables from member node to this one.  */
>   Var(TARGET_INDEX, gn) = Var(TARGET_INDEX, mem);
> @@ -1685,19 +1458,14 @@ Suff_Init(void)
>   Static_Lst_Init(&srclist);
>   ohash_init(&transforms, 4, &gnode_info);
>
> - /*
> - * Create null suffix for single-suffix rules (POSIX). The thing doesn't
> - * actually go on the suffix list or everyone will think that's its
> - * suffix.
> - */
> + /* Create null suffix for single-suffix rules (POSIX). The thing doesn't
> + * actually go on the suffix list as it matches everything.  */
>   emptySuff = new_suffix("");
> - make_suffix_known(emptySuff);
> + emptySuff->flags = SUFF_ACTIVE;
> + emptySuff->order = 0;
>   Dir_Concat(&emptySuff->searchPath, defaultPath);
>   ohash_init(&suffixes, 4, &suff_info);
> - order = 0;
> - clear_suffixes();
>   special_path_hack();
> -
>  }
>
>
> diff --git a/suff.h b/suff.h
> index 2fe7923..79e65d2 100644
> --- a/suff.h
> +++ b/suff.h
> @@ -3,7 +3,7 @@
>  /* $OpenBSD: suff.h,v 1.10 2012/12/06 14:30:35 espie Exp $ */
>
>  /*
> - * Copyright (c) 2001 Marc Espie.
> + * Copyright (c) 2001-2019 Marc Espie.
>   *
>   * Redistribution and use in source and binary forms, with or without
>   * modification, are permitted provided that the following conditions
> @@ -27,16 +27,40 @@
>   * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>   */
>
> -extern void Suff_ClearSuffixes(void);
> +extern void Suff_Init(void);
> +
> +/* Suff_DisableAllSuffixes():
> + * disable current suffixes and the corresponding rules.
> + * They may be re-activated by adding a suffix anew.  */
> +extern void Suff_DisableAllSuffixes(void);
> +/* gn = Suff_ParseAsTransform(line, eline):
> + * Try parsing a [line,eline[ as a suffix transformation
> + * (.a.b or .a). If successful, returns a gn we can add
> + * commands to (this is actually a transform kept on a
> + * separate hash from normal targets).  Otherwise returns NULL. */
>  extern GNode *Suff_ParseAsTransform(const char *, const char *);
> +/* Suff_AddSuffixi(name, ename):
> + * add the passed string interval [name,ename[ as a known
> + * suffix. */
>  extern void Suff_AddSuffixi(const char *, const char *);
> -extern void Suff_FindDeps(GNode *);
> -extern void Suff_Init(void);
> +/* process_suffixes_after_makefile_is_read():
> + * finish setting up the transformation graph for Suff_FindDep
> + * and the .PATH.sfx paths get the default path appended for
> + * find_suffix_path().  */
>  extern void process_suffixes_after_makefile_is_read(void);
> +/* Suff_FindDeps(gn):
> + * find implicit dependencies for gn and fill out corresponding
> + * fields. */
> +extern void Suff_FindDeps(GNode *);
> +/* l = find_suffix_path(gn):
> + * returns the path associated with a gn, either because of its
> + * suffix, or the default path.  */
>  extern Lst find_suffix_path(GNode *);
> +/* Suff_PrintAll():
> + * displays all suffix information. */
>  extern void Suff_PrintAll(void);
> -extern void expand_children_from(GNode *, LstNode);
> -#define expand_all_children(gn) \
> -    expand_children_from(gn, Lst_First(&(gn)->children))
> -
> +/* path = find_best_path(name):
> + * find the best path for the name, according to known suffixes.
> + */
> +extern Lst find_best_path(const char *name);
>  #endif
> diff --git a/targ.c b/targ.c
> index b2d2489..bba0192 100644
> --- a/targ.c
> +++ b/targ.c
> @@ -122,8 +122,11 @@ struct ohash_info gnode_info = {
>   offsetof(GNode, name), NULL, hash_calloc, hash_free, element_alloc
>  };
>
> -#define Targ_FindConstantNode(n, f) Targ_FindNodeh(n, sizeof(n), K_##n, f)
> +static GNode *Targ_mk_node(const char *, const char *, unsigned int,
> +    unsigned char, unsigned int);
>
> +#define Targ_mk_constant(n, type) \
> +    Targ_mk_special_node(n, sizeof(n), K_##n, type, SPECIAL_NONE, 0)
>
>  GNode *begin_node, *end_node, *interrupt_node, *DEFAULT;
>
> @@ -132,27 +135,28 @@ Targ_Init(void)
>  {
>   /* A small make file already creates 200 targets.  */
>   ohash_init(&targets, 10, &gnode_info);
> - begin_node = Targ_FindConstantNode(NODE_BEGIN, TARG_CREATE);
> - begin_node->type |= OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT;
> - end_node = Targ_FindConstantNode(NODE_END, TARG_CREATE);
> - end_node->type |= OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT;
> - interrupt_node = Targ_FindConstantNode(NODE_INTERRUPT, TARG_CREATE);
> - interrupt_node->type |= OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT;
> - DEFAULT = Targ_FindConstantNode(NODE_DEFAULT, TARG_CREATE);
> - DEFAULT->type |= OP_DUMMY | OP_NOTMAIN| OP_TRANSFORM | OP_NODEFAULT;
> + begin_node = Targ_mk_constant(NODE_BEGIN,
> +    OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT);
> + end_node = Targ_mk_constant(NODE_END,
> +    OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT);
> + interrupt_node = Targ_mk_constant(NODE_INTERRUPT,
> +    OP_DUMMY | OP_NOTMAIN | OP_NODEFAULT);
> + DEFAULT = Targ_mk_constant(NODE_DEFAULT,
> +    OP_DUMMY | OP_NOTMAIN| OP_TRANSFORM | OP_NODEFAULT);
>
>  }
>
> -GNode *
> -Targ_NewGNi(const char *name, const char *ename)
> +static GNode *
> +Targ_mk_node(const char *name, const char *ename,
> +    unsigned int type, unsigned char special, unsigned int special_op)
>  {
>   GNode *gn;
>
>   gn = ohash_create_entry(&gnode_info, name, &ename);
>   gn->path = NULL;
> - gn->type = OP_ZERO;
> - gn->special = SPECIAL_NONE;
> - gn->special_op = 0;
> + gn->type = type;
> + gn->special = special;
> + gn->special_op = special_op;
>   gn->children_left = 0;
>   gn->must_make = false;
>   gn->built_status = UNKNOWN;
> @@ -183,20 +187,20 @@ Targ_NewGNi(const char *name, const char *ename)
>  }
>
>  GNode *
> -Targ_FindNodei(const char *name, const char *ename, int flags)
> +Targ_NewGNi(const char *name, const char *ename)
>  {
> - uint32_t hv;
> -
> - hv = ohash_interval(name, &ename);
> - return Targ_FindNodeih(name, ename, hv, flags);
> + return Targ_mk_node(name, ename, OP_ZERO, SPECIAL_NONE, 0);
>  }
>
>  GNode *
> -Targ_FindNodeih(const char *name, const char *ename, uint32_t hv, int flags)
> +Targ_FindNodei(const char *name, const char *ename, int flags)
>  {
> + uint32_t hv;
>   GNode *gn;
>   unsigned int slot;
>
> + hv = ohash_interval(name, &ename);
> +
>   slot = ohash_lookup_interval(&targets, name, ename, hv);
>
>   gn = ohash_find(&targets, slot);
> @@ -209,6 +213,24 @@ Targ_FindNodeih(const char *name, const char *ename, uint32_t hv, int flags)
>   return gn;
>  }
>
> +GNode *
> +Targ_mk_special_node(const char *name, size_t n, uint32_t hv,
> +    unsigned int type, unsigned char special, unsigned int special_op)
> +{
> + GNode *gn;
> + unsigned int slot;
> + const char *ename = name + n - 1;
> +
> + slot = ohash_lookup_interval(&targets, name, ename, hv);
> +
> + assert(ohash_find(&targets, slot) == NULL);
> +
> + gn = Targ_mk_node(name, ename, type, special, special_op);
> + ohash_insert(&targets, slot, gn);
> +
> + return gn;
> +}
> +
>  void
>  Targ_FindList(Lst nodes, Lst names)
>  {
> @@ -255,6 +277,12 @@ Targ_Precious(GNode *gn)
>   return false;
>  }
>
> +bool
> +node_is_real(GNode *gn)
> +{
> + return (gn->type & OP_DUMMY) == 0;
> +}
> +
>  void
>  Targ_PrintCmd(void *p)
>  {
> @@ -321,9 +349,3 @@ targets_hash()
>  {
>   return &targets;
>  }
> -
> -GNode *
> -Targ_FindNodeh(const char *name, size_t n, uint32_t hv, int flags)
> -{
> - return Targ_FindNodeih(name, name + n - 1, hv, flags);
> -}
> diff --git a/targ.h b/targ.h
> index 78517e3..69b527c 100644
> --- a/targ.h
> +++ b/targ.h
> @@ -46,17 +46,10 @@ extern GNode *Targ_FindNodei(const char *, const char *, int);
>
>
>
> -/* set of helpers for constant nodes */
> -extern GNode *Targ_FindNodeih(const char *, const char *, uint32_t, int);
> +/* helper for constant nodes */
> +extern GNode *Targ_mk_special_node(const char *, size_t, uint32_t,
> +    unsigned int, unsigned char, unsigned int);
>
> -__only_inline GNode *
> -Targ_FindNodeh(const char *, size_t, uint32_t, int);
> -
> -__only_inline GNode *
> -Targ_FindNodeh(const char *name, size_t n, uint32_t hv, int flags)
> -{
> - return Targ_FindNodeih(name, name + n - 1, hv, flags);
> -}
>  extern void Targ_FindList(Lst, Lst);
>  extern bool Targ_Ignore(GNode *);
>  extern bool Targ_Silent(GNode *);
> @@ -64,6 +57,7 @@ extern bool Targ_Precious(GNode *);
>  extern void Targ_PrintCmd(void *);
>  extern void Targ_PrintType(int);
>  extern void Targ_PrintGraph(int);
> +extern bool node_is_real(GNode *);
>
>  extern GNode *begin_node, *end_node, *interrupt_node, *DEFAULT;
>  struct ohash_info;

Reply | Threaded
Open this post in threaded view
|

Re: MAKE: redux patch

Marc Espie-2
On Fri, Jan 10, 2020 at 10:52:03PM +0100, Alexander Bluhm wrote:

> On Fri, Jan 10, 2020 at 01:58:47PM +0100, Marc Espie wrote:
> > Bleh, I forgot to synch two patches I already committed. Here's a patch
> > that applies cleanly.
>
> I did run this make through a full regress.  It seems that make
> regress in /usr/src/regress/usr.sbin/ldapd/ exits with an error.
>
> http://bluhm.genua.de/regress/results/2020-01-10T14%3A11%3A58Z/logs/usr.sbin/ldapd/make.log
>
> *** Error 1 in /usr/src/regress/usr.sbin/ldapd (Makefile:44 '.END': @[ -f /usr/src/regress/usr.sbin/ldapd/obj/ldapd.pid ] &&  kill $(cat /us...)
>
> I have never seen this problem before.
>
> bluhm

Oh, the test is wrong, and it's now enough to have make complain about it.

Before the patch, errors in .END and .BEGIN were not properly looked at.

See, that -f ... &&   will be *false* if the file doesn't exist.
The way to this kind of test without failing is

if [ -f ... ]; then ... fi


Index: Makefile
===================================================================
RCS file: /cvs/src/regress/usr.sbin/ldapd/Makefile,v
retrieving revision 1.9
diff -u -p -r1.9 Makefile
--- Makefile 28 Jun 2018 02:47:55 -0000 1.9
+++ Makefile 11 Jan 2020 11:29:35 -0000
@@ -39,9 +39,10 @@ bootstrap:
 
 .if ! (make(clean) || make(cleandir) || make(obj))
 .END:
- @[ -f ${.OBJDIR}/ldapd.pid ] &&\
+ @if [ -f ${.OBJDIR}/ldapd.pid ]; then \
     ${SUDO} kill $$(cat ${.OBJDIR}/ldapd.pid) &&\
-    rm ${.OBJDIR}/ldapd.pid
+    rm ${.OBJDIR}/ldapd.pid; \
+ fi
 .endif
 
 connect: bootstrap

Reply | Threaded
Open this post in threaded view
|

Re: MAKE: redux patch

Todd C. Miller-3
On Sat, 11 Jan 2020 12:34:23 +0100, Marc Espie wrote:

> Oh, the test is wrong, and it's now enough to have make complain about it.
>
> Before the patch, errors in .END and .BEGIN were not properly looked at.
>
> See, that -f ... &&   will be *false* if the file doesn't exist.
> The way to this kind of test without failing is
>
> if [ -f ... ]; then ... fi

Right, this is a common Makefile issue.  OK millert@

 - todd