Installing a tree of files

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Installing a tree of files

Ingo Feinerer-2
Hi,

to all experienced porters: is there a best practice for installing a whole
tree of files in do-install?

I have encountered following patterns:

cp -Rp
tar -czf - -C ... | tar xzf - -C ...
pax -rw
INSTALL_DATA in a for loop (works only if you list all subdirectories (with their subdirs etc.))

FreeBSD has the COPYTREE* macros for this purpose
(https://www.freebsd.org/doc/en/books/porters-handbook/book.html#install-copytree).

If there is such a recommended procedure would it make sense to provide
a similar macro in bsd.port.mk?

Best regards,
Ingo

Reply | Threaded
Open this post in threaded view
|

Re: Installing a tree of files

Brian Callahan-5


On 8/11/19 1:52 AM, Ingo Feinerer wrote:

> Hi,
>
> to all experienced porters: is there a best practice for installing a whole
> tree of files in do-install?
>
> I have encountered following patterns:
>
> cp -Rp
> tar -czf - -C ... | tar xzf - -C ...
> pax -rw
> INSTALL_DATA in a for loop (works only if you list all subdirectories (with their subdirs etc.))
>
> FreeBSD has the COPYTREE* macros for this purpose
> (https://www.freebsd.org/doc/en/books/porters-handbook/book.html#install-copytree).
>
> If there is such a recommended procedure would it make sense to provide
> a similar macro in bsd.port.mk?
>
> Best regards,
> Ingo
>

There is this, found for example in games/polymorphable:
${INSTALL_DATA_DIR} ${PREFIX}/share/flare/mods/polymorphable
         cd ${WRKSRC} && find * -type d -exec ${INSTALL_DATA_DIR} \
                 "${PREFIX}/share/flare/mods/polymorphable/{}" \;
         cd ${WRKSRC} && find * -type f -exec ${INSTALL_DATA} \
                 "{}" "${PREFIX}/share/flare/mods/polymorphable/{}" \;

~Brian

Reply | Threaded
Open this post in threaded view
|

Re: Installing a tree of files

Klemens Nanni-2
On Sun, Aug 11, 2019 at 09:56:23AM -0400, Brian Callahan wrote:
> There is this, found for example in games/polymorphable:
> ${INSTALL_DATA_DIR} ${PREFIX}/share/flare/mods/polymorphable
>         cd ${WRKSRC} && find * -type d -exec ${INSTALL_DATA_DIR} \
>                 "${PREFIX}/share/flare/mods/polymorphable/{}" \;
>         cd ${WRKSRC} && find * -type f -exec ${INSTALL_DATA} \
>                 "{}" "${PREFIX}/share/flare/mods/polymorphable/{}" \;
Which can be greatly simplified...

I have no intention to hijack this thread, but maybe the diff can show
one can install multiple directories and/or files under differnt
conditions while traversing the source tree only once.

No PLIST change.
OK?


Index: Makefile
===================================================================
RCS file: /cvs/ports/games/polymorphable/Makefile,v
retrieving revision 1.7
diff -u -p -r1.7 Makefile
--- Makefile 12 Jul 2019 20:46:23 -0000 1.7
+++ Makefile 11 Aug 2019 16:10:08 -0000
@@ -17,15 +17,10 @@ RUN_DEPENDS = games/flare
 NO_BUILD = Yes
 NO_TEST = Yes
 
-post-extract:
- rm ${WRKSRC}/maps/{demo,test}.txt~
- rm ${WRKSRC}/tilesetdefs/{lpc_ortho,tileset_lpc}.txt~
+MODDIR = ${PREFIX}/share/flare/mods/polymorphable
 
 do-install:
- ${INSTALL_DATA_DIR} ${PREFIX}/share/flare/mods/polymorphable
- cd ${WRKSRC} && find * -type d -exec ${INSTALL_DATA_DIR} \
- "${PREFIX}/share/flare/mods/polymorphable/{}" \;
- cd ${WRKSRC} && find * -type f -exec ${INSTALL_DATA} \
- "{}" "${PREFIX}/share/flare/mods/polymorphable/{}" \;
+ cd ${WRKSRC} && find . -type d -exec ${INSTALL_DATA_DIR} ${MODDIR}/{} \; \
+    -o -type f ! -name \*.txt~ -exec ${INSTALL_DATA} {} ${MODDIR}/{} \;
 
 .include <bsd.port.mk>

Reply | Threaded
Open this post in threaded view
|

Re: Installing a tree of files

Brian Callahan-5


On 8/11/19 12:50 PM, Klemens Nanni wrote:

> On Sun, Aug 11, 2019 at 09:56:23AM -0400, Brian Callahan wrote:
>> There is this, found for example in games/polymorphable:
>> ${INSTALL_DATA_DIR} ${PREFIX}/share/flare/mods/polymorphable
>>          cd ${WRKSRC} && find * -type d -exec ${INSTALL_DATA_DIR} \
>>                  "${PREFIX}/share/flare/mods/polymorphable/{}" \;
>>          cd ${WRKSRC} && find * -type f -exec ${INSTALL_DATA} \
>>                  "{}" "${PREFIX}/share/flare/mods/polymorphable/{}" \;
> Which can be greatly simplified...
>
> I have no intention to hijack this thread, but maybe the diff can show
> one can install multiple directories and/or files under differnt
> conditions while traversing the source tree only once.
>
> No PLIST change.
> OK?

Sure, but do you mind sweeping the tree for this do-install idiom? It's
probably not in too many places these days, but it's certainly in more
than just this one port.

~Brian

>
> Index: Makefile
> ===================================================================
> RCS file: /cvs/ports/games/polymorphable/Makefile,v
> retrieving revision 1.7
> diff -u -p -r1.7 Makefile
> --- Makefile 12 Jul 2019 20:46:23 -0000 1.7
> +++ Makefile 11 Aug 2019 16:10:08 -0000
> @@ -17,15 +17,10 @@ RUN_DEPENDS = games/flare
>   NO_BUILD = Yes
>   NO_TEST = Yes
>  
> -post-extract:
> - rm ${WRKSRC}/maps/{demo,test}.txt~
> - rm ${WRKSRC}/tilesetdefs/{lpc_ortho,tileset_lpc}.txt~
> +MODDIR = ${PREFIX}/share/flare/mods/polymorphable
>  
>   do-install:
> - ${INSTALL_DATA_DIR} ${PREFIX}/share/flare/mods/polymorphable
> - cd ${WRKSRC} && find * -type d -exec ${INSTALL_DATA_DIR} \
> - "${PREFIX}/share/flare/mods/polymorphable/{}" \;
> - cd ${WRKSRC} && find * -type f -exec ${INSTALL_DATA} \
> - "{}" "${PREFIX}/share/flare/mods/polymorphable/{}" \;
> + cd ${WRKSRC} && find . -type d -exec ${INSTALL_DATA_DIR} ${MODDIR}/{} \; \
> +    -o -type f ! -name \*.txt~ -exec ${INSTALL_DATA} {} ${MODDIR}/{} \;
>  
>   .include <bsd.port.mk>

Reply | Threaded
Open this post in threaded view
|

Re: Installing a tree of files

Klemens Nanni-2
On Sun, Aug 11, 2019 at 01:33:18PM -0400, Brian Callahan wrote:
> Sure, but do you mind sweeping the tree for this do-install idiom? It's
> probably not in too many places these days, but it's certainly in more than
> just this one port.
I'll do these as I go since they're quite low priority, but yes.

Reply | Threaded
Open this post in threaded view
|

Re: Installing a tree of files

Stuart Henderson
In reply to this post by Brian Callahan-5
On 2019/08/11 13:33, Brian Callahan wrote:
>
> Sure, but do you mind sweeping the tree for this do-install idiom? It's
> probably not in too many places these days, but it's certainly in more
> than just this one port.

If other ports are being touched can we have something that avoids a
fork for every file please? Some of these have tens of thousands of
files.

Since ownership isn't important any more (now we build ports as
non-root) it might be simplest just to cp nowadays.

Reply | Threaded
Open this post in threaded view
|

Re: Installing a tree of files

Ted Unangst-6
Stuart Henderson wrote:

> On 2019/08/11 13:33, Brian Callahan wrote:
> >
> > Sure, but do you mind sweeping the tree for this do-install idiom? It's
> > probably not in too many places these days, but it's certainly in more
> > than just this one port.
>
> If other ports are being touched can we have something that avoids a
> fork for every file please? Some of these have tens of thousands of
> files.
>
> Since ownership isn't important any more (now we build ports as
> non-root) it might be simplest just to cp nowadays.

Inconveniently, find -exec {} + requires that {} go last, not followed by
destination. Maybe some way to sh around that.

Reply | Threaded
Open this post in threaded view
|

Re: Installing a tree of files

Klemens Nanni-2
On Tue, Aug 13, 2019 at 12:08:58AM -0400, Ted Unangst wrote:
> Inconveniently, find -exec {} + requires that {} go last, not followed by
> destination. Maybe some way to sh around that.
Yup, I could work with xargs here, but before doing that I think sthen's
point of simply copying trees as a whole (using pax for example) is the
better approach here.

Reply | Threaded
Open this post in threaded view
|

Re: Installing a tree of files

Jan Stary
In reply to this post by Ingo Feinerer-2
On Aug 11 07:52:34, [hidden email] wrote:

> Hi,
>
> to all experienced porters: is there a best practice for installing a whole
> tree of files in do-install?
>
> I have encountered following patterns:
>
> cp -Rp
> tar -czf - -C ... | tar xzf - -C ...
> pax -rw
> INSTALL_DATA in a for loop (works only if you list all subdirectories (with their subdirs etc.))

find /src | cpio -p -d -u /dst

> FreeBSD has the COPYTREE* macros for this purpose
> (https://www.freebsd.org/doc/en/books/porters-handbook/book.html#install-copytree).
>
> If there is such a recommended procedure would it make sense to provide
> a similar macro in bsd.port.mk?
>
> Best regards,
> Ingo
>