web pages describing individual packages (was Re: [OT] securely sharing documents on OpenBSD?)

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

web pages describing individual packages (was Re: [OT] securely sharing documents on OpenBSD?)

Lars Nooden-2
On Tue, 17 Aug 2010, ropers wrote:
>  NB: Whatever happened to the web pages describing individual packages?
>  They used to exist up to 4.6 at e.g.
>    http://www.openbsd.org/4.6_packages/i386/mod_dav-1.0.3p7.tgz-long.html
>  and I at least found them helpful.

I found them helpful, too.

They were pulled from CVS around the beginning of June.

I'd expect that the pages would be easier to work with if generated
automatically.  Most of the descriptive metadata needed to create the
pages *should* in principle be available in the Makefile, DESCR-*, and
PLIST-* for each package:

    + name of the package -- DISTNAME
    + version
    + URI for original source or project
    + URI for port tarball
    + contact mail address for maintainer(s)
    + application category
    + short description
    + long description -- DESCR-*
    + list of files added by the package -- PLIST-*

For that to work automatically, each port needs to contain a common core
of metadata and have agreement on the descriptors.  See bsd.port.mk(5)

Below is a half-baked sketch (not a real script) to illustrate the general
concept involved to find the metadata for an individual package.  find()
can be used to walk the ports tree and find packages to run a real script
on. Wrapping output up as an XHTML document is mostly a matter of printing
a here document.

/Lars

---

#!/usr/bin/perl

use strict;
use integer;
use Getopt::Std;

##
###
##

my %metadata = ();
my $path     = undef;

our $opt_f    = undef; # files installed by package
our $opt_h    = undef; # help

#   + name of the package -- DISTNAME
#   + version
#   + URI for original source or project
#   + URI for port tarball
#   + contact mail address for maintainer(s)
#   + application category
#   + short description
#   + long description
#   + list of files added by the package

sub Usage {
   my ( $err ) = ( @_ );

   print qq(Usage: $0 [fh] {path}\n);

   exit (0);
}

##
##

sub Help {
    print qq(


);
   &Usage;

}


sub New_Metadata {
   # core metadata fields
   $metadata{'OpenBSD_App'}= {
     PACKAGE     => undef, #
     VERSION     => undef, #
     HOMEPAGE    => undef, #
     TARBALL     => undef, #
     MAINTAINER  => undef, #
     SUMMARY     => undef, #
     DESCRIPTION => undef, #
     CATEGORIES  => []
   };

   $metadata{'Dublin_Core'}= {
     'DC.TITLE'       => undef,
     'DC.SUBJECT'     => undef,
     'DC.DESCRIPTION' => undef,
     'DC.TYPE'        => undef,
     'DC.SOURCE'      => undef,
     'DC.RELATION'    => undef,
     'DC.COVERAGE'    => undef,
     'DC.CREATOR'     => undef,
     'DC.PUBLISHER'   => undef,
     'DC.CONTRIBUTOR' => undef,
     'DC.RIGHTS'      => undef,
     'DC.DATE'        => undef,
     'DC.FORMAT'      => undef,
     'DC.IDENTIFIER'  => undef,
     'DC.LANGUAGE'    => undef
   };

   $metadata{'links'} = ();

   return ( 1 );

}

sub dublin_core {
   ## http://dublincore.org/documents/usageguide/
}

sub stylesheets {
   print qq(<style type="text/css" media="screen">\n);
   print qq(</style>\n);
}

sub xhtml_head {
   my ( %metadata ) = ( @_ );
   print qq(<head>\n);
   print qq( <title>),$metadata{OpenBSD_App}->{'PACKAGE'},qq(-),$metadata{OpenBSD_App}->{'PACKAGE'},qq(</title>\n);

   print qq( <!--[if IE]>\n \n <![endif]-->\n);

   # <link rel="next"  href="" />
   # <link rel="subsection" href="" />

   &stylesheets;

   print qq(</head>\n\n);

   return ( 1 );
}

sub Read_Makefile {
   my ( $dir ) = ( @_ );

   return ( 1 );
}

sub GetKeywords {
   my ( $path ) = ( @_ );
   my %variables = ();
   my @variable_names = ( 'DISTNAME', 'HOMEPAGE', 'CATEGORIES', 'MAINTAINER', 'COMMENT', 'COMMENT-main' );


#  + name of the package -- DISTNAME
#  + version -- DISTNAME || VERSION
#  + URI for original source or project -- HOMEPAGE
#  + URI for port tarball
#  + contact mail address for maintainer(s) -- MAINTAINER
#  + application category -- CATEGORIES
#  + short description -- COMMENT
#  + long description
#  + list of files added by the package

   chdir( $path ) or die ("Can't cd to '$path': $!");
   for my $var ( @variable_names ) {

     my $make = qq(/usr/bin/make show=$var |); # this is dog slow but more accurate

     open ( MAKEFILE, $make ) or die ("Can't open '$make': $!\n");

     while ( my $line = <MAKEFILE> ) {
       chomp $line;
       next unless ( $line =~ /\w/ );
       unless ( $var eq 'CATEGORIES' ) {
         print qq($var\t$line\n);
         $var = 'COMMENT' if ( $var eq 'COMMENT-main' );
         $variables{"$var"}=$line;
       } else {
         $variables{"$var"}=split(/\s+/,$line);
       }
     }

     close ( MAKEFILE ) or die ("Can't close MAKEFILE : $!\n");

   }


   return ( %variables );
}

sub FileList {
   my ( $path ) = ( @_ );

   my $find = qq(/usr/bin/find $path -name 'PLIST*' -print |);

   my %install = (
     'bin'   => '/usr/bin/', #
     'man'   => '/usr/share/man/', #
     'share' => '/usr/share' #
   );

   my @files = ();

   open  ( PLIST, $find )  or die ("Can't open '$find': $!\n");
   while ( my $plist = <PLIST> ) {
     chomp $plist;

     open  ( FILES, $plist )  or die ("Can't open '$plist': $!\n");
     # print qq(\tPLIST=$plist\n);

    # --- this substitution should be optional ----
     while ( my $file = <FILES> ) {
       chomp $file;

       next unless ( $file =~ /^\@??([a-z]+)\s*/ );

       my $type = $1;
       # print qq(\nTYPE=$type\n \t$file\n) if ( exists ( $install{$type} ) );

       if ( exists ( $install{$type} ) ) {
          $file =~ s/^\@?([a-z]+)\s*/$install{$type}/
       } else {
          next;
       }

       print qq(\t$file\n) if ( exists ( $install{$type} ) );

     }

     close ( FILES )          or die ("Can't close FILES : $!\n");

   }
   close ( PLIST )         or die ("Can't close PLIST : $!\n");

   return ( 1 );
}

##
###
##

$path = pop( @ARGV ) if ( $#ARGV >= 0 );

getopts('fh');

&FileList( $path ) if ( $opt_f );
&New_Metadata ;
&Help     if ( $opt_h );
&GetKeywords( $path );

&Usage($!) unless ( $path );
-d $path or die ("Must provide a real directory : '$path'\n");

exit (0);

Reply | Threaded
Open this post in threaded view
|

Re: web pages describing individual packages (was Re: [OT] securely sharing documents on OpenBSD?)

Stuart Henderson
On 2010/08/17 23:14, Lars Nooden wrote:

> On Tue, 17 Aug 2010, ropers wrote:
> > NB: Whatever happened to the web pages describing individual packages?
> > They used to exist up to 4.6 at e.g.
> >   http://www.openbsd.org/4.6_packages/i386/mod_dav-1.0.3p7.tgz-long.html
> > and I at least found them helpful.
>
> I found them helpful, too.
>
> They were pulled from CVS around the beginning of June.
>
> I'd expect that the pages would be easier to work with if generated
> automatically.

unsurprisingly, they are generated by a program...
/usr/ports/infrastructure/package/gen-package-pages

Reply | Threaded
Open this post in threaded view
|

Re: web pages describing individual packages (was Re: [OT] securely sharing documents on OpenBSD?)

Lars Nooden-2
On Tue, 17 Aug 2010, Stuart Henderson wrote:
> unsurprisingly, they are generated by a program...
> /usr/ports/infrastructure/package/gen-package-pages

Good to know.  I've made a quick survey of what's in ports today:
http://www-personal.umich.edu/~lars/OpenBSD/Ports/ports-makefile-variables.txt

There does not appear to be a common core of Makefile variables.

Can someone expand on the comments in cvs for faq15.html regarding the
difficulties?

Reply | Threaded
Open this post in threaded view
|

Re: web pages describing individual packages (was Re: [OT] securely sharing documents on OpenBSD?)

Joachim Schipper-2
On Wed, Aug 18, 2010 at 03:43:38PM +0300, Lars Nooden wrote:
> On Tue, 17 Aug 2010, Stuart Henderson wrote:
> >unsurprisingly, they are generated by a program...
> >/usr/ports/infrastructure/package/gen-package-pages
>
> Good to know.  I've made a quick survey of what's in ports today:
> http://www-personal.umich.edu/~lars/OpenBSD/Ports/ports-makefile-variables.txt
>
> There does not appear to be a common core of Makefile variables.

That's just grepping - make SHOW=... will give you all of the "derived"
variables as well, e.g. PKGNAME.

                Joachim

--
TFMotD: atu (4) - Atmel AT76C50x USB IEEE 802.11b wireless network
device

Reply | Threaded
Open this post in threaded view
|

Re: web pages describing individual packages (was Re: [OT] securely sharing documents on OpenBSD?)

Stuart Henderson
On 2010/08/18 15:17, Joachim Schipper wrote:

> On Wed, Aug 18, 2010 at 03:43:38PM +0300, Lars Nooden wrote:
> > On Tue, 17 Aug 2010, Stuart Henderson wrote:
> > >unsurprisingly, they are generated by a program...
> > >/usr/ports/infrastructure/package/gen-package-pages
> >
> > Good to know.  I've made a quick survey of what's in ports today:
> > http://www-personal.umich.edu/~lars/OpenBSD/Ports/ports-makefile-variables.txt
> >
> > There does not appear to be a common core of Makefile variables.
>
> That's just grepping - make SHOW=... will give you all of the "derived"
> variables as well, e.g. PKGNAME.

dump-vars

Reply | Threaded
Open this post in threaded view
|

Re: web pages describing individual packages (was Re: [OT] securely sharing documents on OpenBSD?)

Marc Espie-2
In reply to this post by Lars Nooden-2
On Wed, Aug 18, 2010 at 03:43:38PM +0300, Lars Nooden wrote:
> On Tue, 17 Aug 2010, Stuart Henderson wrote:
> >unsurprisingly, they are generated by a program...
> >/usr/ports/infrastructure/package/gen-package-pages
>
> Good to know.  I've made a quick survey of what's in ports today:
> http://www-personal.umich.edu/~lars/OpenBSD/Ports/ports-makefile-variables.txt

> There does not appear to be a common core of Makefile variables.

Looks like you're reinventing either bsd.port.mk(5), or dump-vars, or
sqlports... or maybe all of them. ;-)

Of course there is a common core of variables, that's what documented
in bsd.port.mk(5). They have default values which are adequate. And don't
forget that make *is* a scripting language, so it's quite permissible
to use other variable names in ports' Makefile (yes, we could edict rules
about naming, but the inconvenience FAR offsets the benefits).