Building tree without making obj symlinks

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Building tree without making obj symlinks

Masao Uebayashi-3
With these changes applied, and by specifying MAKESRCDIRPREFIX/MAKEOBJDIRPREFIX,
I can build tree without creating obj symlinks under source.

If src directory is /a/src, and obj is /b/obj, use
        MAKESRCDIRPREFIX=/a/src
        MAKEOBJDIRPREFIX=/b/obj
then .OBJDIR under /a/src/bin/ls becomes /b/obj/bin/ls (if object directories
are already generated by "make obj").

I'm not 100% sure about the change in bsd.obj.mk (i.e. interaction with
bsd.subdir.mk).

0001-Teach-make-1-handling-of-BSDSRCDIRPREFIX-BSDOBJDIRPR.patch (2K) Download Attachment
0002-Create-obj-directories-for-BSDSRCDIRPREFIX-BSDOBJDIR.patch (750 bytes) Download Attachment
0003-Pass-MAKESRCDIRPREFIX-MAKEOBJDIRPREFIX-for-cross-bui.patch (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Building tree without making obj symlinks

Theo de Raadt
Why?

I've been here for a while.  My experiences:

Lots of variations of build environment -> lots of failure
conditions happen -> many people wasting their time.

Please supply a good justification why that is good.

If you can't, throw your diff away.  It Simple as that.

> With these changes applied, and by specifying MAKESRCDIRPREFIX/MAKEOBJDIRPREFIX,
> I can build tree without creating obj symlinks under source.
>
> If src directory is /a/src, and obj is /b/obj, use
> MAKESRCDIRPREFIX=/a/src
> MAKEOBJDIRPREFIX=/b/obj
> then .OBJDIR under /a/src/bin/ls becomes /b/obj/bin/ls (if object directories
> are already generated by "make obj").
>
> I'm not 100% sure about the change in bsd.obj.mk (i.e. interaction with
> bsd.subdir.mk).
>
> --y0ulUmNC+osPPQO6
> Content-Type: text/plain; charset=us-ascii
> Content-Disposition: attachment; filename="0001-Teach-make-1-handling-of-BSDSRCDIRPREFIX-BSDOBJDIRPR.patch"
>
> >From d81931ee7fea3e1f30c7d05ef95bbc01a489f73f Mon Sep 17 00:00:00 2001
> From: Masao Uebayashi <[hidden email]>
> Date: Fri, 24 Jun 2016 02:35:35 +0900
> Subject: [PATCH 1/3] Teach make(1) handling of BSDSRCDIRPREFIX/BSDOBJDIRPREFIX
>
> ---
>  usr.bin/make/main.c | 47 ++++++++++++++++++++++++++++++++++++++++-------
>  1 file changed, 40 insertions(+), 7 deletions(-)
>
> diff --git a/usr.bin/make/main.c b/usr.bin/make/main.c
> index ac2402c..6cb4191 100644
> --- a/usr.bin/make/main.c
> +++ b/usr.bin/make/main.c
> @@ -546,22 +546,55 @@ chdir_verify_path(const char *path, struct dirs *d)
>  static void
>  setup_CURDIR_OBJDIR(struct dirs *d, const char *machine)
>  {
> - char *path;
> + char *srcpfx, *objpfx;
> + char objdirbuf[PATH_MAX], *objdir;
>  
>   d->current = figure_out_CURDIR();
>   /*
> - * If the MAKEOBJDIR (or by default, the _PATH_OBJDIR) directory
> - * exists, change into it and build there.  
> + * 1. If both MAKESRCDIRPREFIX and MAKEOBJDIRPREFIX are defined,
> + *    the object directory is:
> + * ${MAKEOBJDIRPREFIX}/<sub-directory under ${MAKESRCDIRPREFIX}>
> + * 2. If MAKEOBJDIR is defined, the object directory is:
> + * ${.CURDIR}/${MAKEOBJDIR}
> + * 3. Or by default:
> + * ${.CURDIR}/<_PATH_OBJDIR>
> + *
> + * If the object directory exists, change into it and build there.
> + * The object directory has to be created in prior by "make obj";
> + * otherwise chdir(2) fails and build is done in the current directory.
>   *
>   * Once things are initted,
>   * have to add the original directory to the search path,
>   * and modify the paths for the Makefiles appropriately.  The
>   * current directory is also placed as a variable for make scripts.
>   */
> - if ((path = getenv("MAKEOBJDIR")) == NULL) {
> - path = _PATH_OBJDIR;
> - }
> - d->object = chdir_verify_path(path, d);
> +
> + srcpfx = getenv("MAKESRCDIRPREFIX");
> + objpfx = getenv("MAKEOBJDIRPREFIX");
> + if (srcpfx != NULL && objpfx != NULL) {
> + const size_t srcpfxlen = strlen(srcpfx);
> + const ssize_t srcsfxlen = strlen(d->current) - srcpfxlen;
> + const size_t objpfxlen = strlen(objpfx);
> +
> + if (srcsfxlen <= 0 ||
> +    memcmp(d->current, srcpfx, srcpfxlen) != 0 ||
> +    d->current[srcpfxlen] != '/') {
> + Fatal("make: .CURDIR (%s) not under "
> +    "MAKESRCDIRPREFIX (%s).", d->current, srcpfx);
> + }
> + if (objpfxlen + srcsfxlen + 1 > PATH_MAX) {
> + Fatal("make: .OBJDIR too long (%s%s).", objpfx,
> +    d->current + srcpfxlen);
> + }
> + objdir = objdirbuf;
> + strncpy(objdir, objpfx, objpfxlen);
> + strncpy(objdir + objpfxlen, d->current + srcpfxlen, srcsfxlen);
> + } else {
> + if ((objdir = getenv("MAKEOBJDIR")) == NULL) {
> + objdir = _PATH_OBJDIR;
> + }
> + }
> + d->object = chdir_verify_path(objdir, d);
>   if (d->object == NULL)
>   d->object = d->current;
>  }
> --
> 2.8.4
>
>
> --y0ulUmNC+osPPQO6
> Content-Type: text/plain; charset=us-ascii
> Content-Disposition: attachment; filename="0002-Create-obj-directories-for-BSDSRCDIRPREFIX-BSDOBJDIR.patch"
>
> >From 317a1522aac0464d87d6ed37559db7e73bf5d91c Mon Sep 17 00:00:00 2001
> From: Masao Uebayashi <[hidden email]>
> Date: Fri, 24 Jun 2016 02:36:19 +0900
> Subject: [PATCH 2/3] Create obj directories for
>  BSDSRCDIRPREFIX/BSDOBJDIRPREFIX
>
> ---
>  share/mk/bsd.obj.mk | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/share/mk/bsd.obj.mk b/share/mk/bsd.obj.mk
> index 0c2a67a..d8c21ca 100644
> --- a/share/mk/bsd.obj.mk
> +++ b/share/mk/bsd.obj.mk
> @@ -4,6 +4,12 @@
>  .if !target(obj)
>  .  if defined(NOOBJ)
>  obj:
> +.  elif defined(MAKEOBJDIRPREFIX) && defined(MAKESRCDIRPREFIX)
> +
> +_SUBDIRUSE:
> +obj! _SUBDIRUSE
> + mkdir -p ${MAKEOBJDIRPREFIX}${.CURDIR:C|^${MAKESRCDIRPREFIX}||}
> +
>  .  else
>  
>  .  if defined(MAKEOBJDIR)
> --
> 2.8.4
>
>
> --y0ulUmNC+osPPQO6
> Content-Type: text/plain; charset=us-ascii
> Content-Disposition: attachment; filename="0003-Pass-MAKESRCDIRPREFIX-MAKEOBJDIRPREFIX-for-cross-bui.patch"
>
> >From 4741aa198b2dd411b2a7baed848263561d30734a Mon Sep 17 00:00:00 2001
> From: Masao Uebayashi <[hidden email]>
> Date: Fri, 24 Jun 2016 10:52:18 +0900
> Subject: [PATCH 3/3] Pass MAKESRCDIRPREFIX/MAKEOBJDIRPREFIX for cross build
>
> ---
>  Makefile.cross | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 52 insertions(+)
>
> diff --git a/Makefile.cross b/Makefile.cross
> index 8e2afa1..90d8e25 100644
> --- a/Makefile.cross
> +++ b/Makefile.cross
> @@ -119,6 +119,8 @@ cross-env:
>   @echo ${CROSSENV} MACHINE=${TARGET} \
>      MACHINE_ARCH=`cat ${CROSSDIR}/TARGET_ARCH` \
>      MACHINE_CPU=`cat ${CROSSDIR}/TARGET_CPU` \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      BSDOBJDIR=${CROSSDIR}/usr/obj \
>      BSDSRCDIR=${.CURDIR} MAKEOBJDIR=obj.${MACHINE}.${TARGET}
>  
> @@ -154,6 +156,8 @@ ${CROSSOBJ}: ${CROSSDIRS}
>      COMPILER_VERSION=${COMPILER_VERSION} \
>      MACHINE=${TARGET} \
>      MACHINE_ARCH=${TARGET_ARCH} MACHINE_CPU=${TARGET_CPU} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      BSDSRCDIR=${.CURDIR} MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
>      ${MAKE} obj)
>   @touch ${CROSSOBJ}
> @@ -164,12 +168,16 @@ ${CROSSINCLUDES}: ${CROSSOBJ}
>      COMPILER_VERSION=${COMPILER_VERSION} \
>      MACHINE=${TARGET} \
>      MACHINE_ARCH=${TARGET_ARCH} MACHINE_CPU=${TARGET_CPU} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
>      TARGET_ARCH=${TARGET_ARCH} TARGET_CPU=${TARGET_CPU} \
>      ${MAKE} prereq && \
>      COMPILER_VERSION=${COMPILER_VERSION} \
>      MACHINE=${TARGET} \
>      MACHINE_ARCH=${TARGET_ARCH} MACHINE_CPU=${TARGET_CPU} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
>      TARGET_ARCH=${TARGET_ARCH} TARGET_CPU=${TARGET_CPU} \
>      ${MAKE} DESTDIR=${CROSSDIR} includes)
> @@ -177,16 +185,22 @@ ${CROSSINCLUDES}: ${CROSSOBJ}
>  
>  ${CROSSBINUTILS}: ${CROSSINCLUDES}
>   (cd ${.CURDIR}/${BINUTILS_DIR}; \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
>      PIE_DEFAULT=${PIE_DEFAULT} \
>      TARGET_ARCH=${TARGET_ARCH} TARGET_CPU=${TARGET_CPU} \
>      PATH=${CROSSPATH} \
>      ${MAKE} -f Makefile.bsd-wrapper depend && \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
>      PIE_DEFAULT=${PIE_DEFAULT} \
>      TARGET_ARCH=${TARGET_ARCH} TARGET_CPU=${TARGET_CPU} \
>      ${MAKE} -f Makefile.bsd-wrapper all && \
>      DESTDIR=${CROSSDIR} MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      COMPILER_VERSION=${COMPILER_VERSION} \
>      PIE_DEFAULT=${PIE_DEFAULT} \
>      MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> @@ -217,14 +231,20 @@ ${CROSSGCC}: ${CROSSBINUTILS}
>  .if ${COMPILER_VERSION:L} == "gcc3"
>   (cd ${.CURDIR}/gnu/usr.bin/gcc; \
>      MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      TARGET_ARCH=${TARGET_ARCH} TARGET_CPU=${TARGET_CPU} \
>      CROSSDIR=${CROSSDIR} \
>      PATH=${CROSSPATH} ${MAKE} -f Makefile.bsd-wrapper depend && \
>      MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      TARGET_ARCH=${TARGET_ARCH} TARGET_CPU=${TARGET_CPU} \
>      CROSSDIR=${CROSSDIR} \
>      ${MAKE} -f Makefile.bsd-wrapper all && \
>      env CROSSDIR=${CROSSDIR} DESTDIR=${CROSSDIR} MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      PATH=${CROSSPATH} ${MAKE} -f Makefile.bsd-wrapper install)
>   cp -f ${CROSSDIR}/usr/bin/${TARGET_CANON}-g++ ${CROSSDIR}/usr/${TARGET_CANON}/bin/${TARGET_CANON}-c++
>   cp -f ${CROSSDIR}/usr/bin/gcc ${CROSSDIR}/usr/${TARGET_CANON}/bin/${TARGET_CANON}-cc
> @@ -245,6 +265,8 @@ ${CROSSGCC}: ${CROSSBINUTILS}
>   (cd ${.CURDIR}/gnu/usr.bin/cc; \
>      PIE_DEFAULT=${PIE_DEFAULT} \
>      MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      TARGET_ARCH=${TARGET_ARCH} TARGET_CPU=${TARGET_CPU} \
>      CROSSDIR=${CROSSDIR} \
>      MACHINE=${TARGET} \
> @@ -252,6 +274,8 @@ ${CROSSGCC}: ${CROSSBINUTILS}
>   (cd ${.CURDIR}/gnu/usr.bin/cc; \
>      PIE_DEFAULT=${PIE_DEFAULT} \
>      MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      TARGET_ARCH=${TARGET_ARCH} TARGET_CPU=${TARGET_CPU} \
>      CROSSDIR=${CROSSDIR} \
>      MACHINE=${TARGET} \
> @@ -260,6 +284,8 @@ ${CROSSGCC}: ${CROSSBINUTILS}
>      env CROSSDIR=${CROSSDIR} DESTDIR=${CROSSDIR} \
>      PIE_DEFAULT=${PIE_DEFAULT} \
>      MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      TARGET_ARCH=${TARGET_ARCH} TARGET_CPU=${TARGET_CPU} \
>      MACHINE=${TARGET} \
>      PATH=${CROSSPATH} ${MAKE} install)
> @@ -294,14 +320,22 @@ cross-lib: ${CROSSGCC}
>      for lib in csu libc; do \
>      (cd $$lib; \
>          eval ${CROSSENV} MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      ${MAKE} depend all ;\
>          eval ${CROSSENV} MAKEOBJDIR=obj.${MACHINE}.${TARGET} DESTDIR=${CROSSDIR} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      ${MAKE} install); \
>      done; \
>      eval ${CROSSENV} MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> + MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> + MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>          SKIPDIR=\"${NO_CROSS}\" \
>          ${MAKE} depend all ; \
>      eval ${CROSSENV} MAKEOBJDIR=obj.${MACHINE}.${TARGET} DESTDIR=${CROSSDIR} \
> + MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> + MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>          SKIPDIR=\"${NO_CROSS}\" \
>          ${MAKE} install)
>  
> @@ -312,9 +346,13 @@ cross-bin: ${CROSSOBJ}
>   for i in libexec bin sbin usr.bin usr.sbin; do \
>   (cd ${.CURDIR}/$$i; \
>      eval ${CROSSENV} MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> + MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> + MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>          SKIPDIR=\"${BINUTILS} ${NO_CROSS}\" \
>          ${MAKE} depend all ; \
>      eval ${CROSSENV} MAKEOBJDIR=obj.${MACHINE}.${TARGET} DESTDIR=${CROSSDIR} \
> + MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> + MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>          SKIPDIR=\"${BINUTILS} ${NO_CROSS}\" \
>          ${MAKE} install); \
>   done
> @@ -326,9 +364,13 @@ cross-gnu: ${CROSSOBJ}
>   for i in gnu/lib gnu/usr.sbin gnu/usr.bin ; do \
>   (cd ${.CURDIR}/$$i; \
>      eval ${CROSSENV} MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> + MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> + MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>          SKIPDIR=\"${NO_CROSS}\" \
>          ${MAKE} depend all ; \
>      eval ${CROSSENV} MAKEOBJDIR=obj.${MACHINE}.${TARGET} DESTDIR=${CROSSDIR} \
> + MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> + MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>          SKIPDIR=\"${NO_CROSS}\" \
>          ${MAKE} install); \
>   done
> @@ -340,6 +382,8 @@ cross-share:   ${CROSSOBJ}
>   for i in share; do \
>      (cd ${.CURDIR}/$$i; \
>          eval ${CROSSENV} MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> + MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> + MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>   SKIPDIR=\"${NO_CROSS}\" \
>   DESTDIR=${CROSSDIR} \
>   ${MAKE} depend all install); \
> @@ -352,6 +396,8 @@ cross-sys:   ${CROSSOBJ}
>   for i in sys; do \
>      (cd ${.CURDIR}/$$i; \
>          eval ${CROSSENV} MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> + MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> + MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>   SKIPDIR=\"${NO_CROSS}\" \
>   DESTDIR=${CROSSDIR} \
>   ${MAKE} depend all install); \
> @@ -368,6 +414,8 @@ cross-depend:
>   @(cd ${.CURDIR} && \
>      BSDOBJDIR=${CROSSDIR}/usr/obj \
>      BSDSRCDIR=${.CURDIR} MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      SKIPDIR="${NO_CROSS}" \
>      ${MAKE} depend)
>  
> @@ -375,6 +423,8 @@ cross-clean:
>   @(cd ${.CURDIR} && \
>      BSDOBJDIR=${CROSSDIR}/usr/obj \
>      BSDSRCDIR=${.CURDIR} MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      SKIPDIR="${NO_CROSS}" \
>      ${MAKE} clean)
>  
> @@ -382,5 +432,7 @@ cross-cleandir:
>   @(cd ${.CURDIR} && \
>      BSDOBJDIR=${CROSSDIR}/usr/obj \
>      BSDSRCDIR=${.CURDIR} MAKEOBJDIR=obj.${MACHINE}.${TARGET} \
> +    MAKESRCDIRPREFIX=${MAKESRCDIRPREFIX} \
> +    MAKEOBJDIRPREFIX=${MAKEOBJDIRPREFIX} \
>      SKIPDIR="${NO_CROSS}" \
>      ${MAKE} cleandir)
> --
> 2.8.4
>
>
> --y0ulUmNC+osPPQO6--
>

Reply | Threaded
Open this post in threaded view
|

Re: Building tree without making obj symlinks

Ted Unangst-6
Theo de Raadt wrote:
> Why?
>
> I've been here for a while.  My experiences:
>
> Lots of variations of build environment -> lots of failure
> conditions happen -> many people wasting their time.
>
> Please supply a good justification why that is good.

Actually, it's kind of interesting. Taking the obj/ links out of the src/ tree
is a natural extension of taking the objects themselves out.

Two users could build from the same src tree into different obj directories.
Right now that doesn't work because there can only be one symlink.
src and obj on nfs mostly works, but only if one is careful about the mount
points.

As a third or fifth or whatever configuration, it may be too much variety, but
it sounds like it could replace the current obj system. Same amount of
variation.