I have a program I wish to submit for the base

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

I have a program I wish to submit for the base

Luke Small
pkg_ping  [-s timeout]
[-n maximum_mirrors_written]

It scrapes each mirror's location and URL from  openbsd.org/ftp.html and
tests the package repository with the  version and architecture of the
machine. It kills the ftp() and sed() functions it calls from C if it takes
too long by using kqueue. It calls uname as well and I put kqueue on it
too, in case there is a chance uname can be called and stall like ftp.

After install, it can write download mirrors to /etc/pkg.conf. I want to
enable the user to write down one or many mirrors as has been calculated by
timing the download of the nearly 700 KB SHA256 file from each mirror.

I think that if pkg_add can't find a suitable mirror, pkg_ping could be
called to find the fastest available mirror(s), especially if their mirror
of choice goes down, or they put off upgrading so long that their mirror of
choice deletes their system's repository.

I think I'm done with it. It is absolutely a coincidence that it is 666
lines. I'm not changing it.

-Luke N Small

pkg_ping.c (20K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Luke Small
I think I fixed all your suggestions. I don't strictly adhere to kernel
normal in the use of comments and I parse command-line arguments without
using getopt(3), but the method is robust.

-Luke

<A few quick comments from glancing over this:

 o I definitely don't think camel case will be accepted

 o I'm pretty sure strtonum(3) is strongly preferred over strtod(3) et al.

pkg_ping.c (20K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Nicholas Marriott-2
Firstly, I don't think we need this in base and I think there is little
to no chance of it being taken, even if the code is improved.

Secondly:

- The code is still miles off style(9) and isn't really a consistent
  style within itself either.

- Forking uname(1)? What? No offence, but that is hilarious :-). Why
  fork uname(1) for uname(3) but not date(1) for gettimeofday(2)?

- Why would you fork sed either?

I think C is the wrong tool for this. Why not write a shell, perl, or
python script?

Then if people start to use it you could make a port.



On Fri, Jan 29, 2016 at 01:34:30AM -0600, Luke Small wrote:

> I think I fixed all your suggestions. I don't strictly adhere to kernel
> normal in the use of comments and I parse command-line arguments without
> using getopt(3), but the method is robust.
>
> -Luke
>
> <A few quick comments from glancing over this:
>
>  o I definitely don't think camel case will be accepted
>
>  o I'm pretty sure strtonum(3) is strongly preferred over strtod(3) et al.

> /*
>  * Copyright (c) 2016 Luke N. Small
>  *
>  * Permission to use, copy, modify, and distribute this software for any
>  * purpose with or without fee is hereby granted, provided that the above
>  * copyright notice and this permission notice appear in all copies.
>  *
>  * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
>  * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
>  * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
>  * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
>  * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
>  * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
>  * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
>  */
>  
>
> /* Special thanks to Dan Mclaughlin for the ftp to sed idea
>  *
>  * ftp -o - http://www.openbsd.org/ftp.html | \
>  * sed -n \
>  *  -e 's:</a>$::' \
>  * -e 's: <strong>\([^<]*\)<.*:\1:p' \
>  * -e 's:^\( [hfr].*\):\1:p'
>  */
>  
>  
> #define EVENT_NOPOLL
> #define EVENT_NOSELECT
>
> #include <stdio.h>
> #include <unistd.h>
> #include <stdlib.h>
> #include <err.h>
> #include <errno.h>
> #include <sys/types.h>
> #include <sys/event.h>
> #include <signal.h>
> #include <string.h>
>
> struct mirror_st
> {
> char * country_title;
> char * mirror;
> char * install_path;
> double diff;
> struct mirror_st * next;
> };
>
> int ftp_cmp (const void * a, const void * b)
> {
> struct mirror_st ** one = (struct mirror_st **)a;
> struct mirror_st ** two = (struct mirror_st **)b;
>
> if ( (*one)->diff < (*two)->diff )
> return -1;
> if ( (*one)->diff > (*two)->diff )
> return 1;
> return 0;
> }
>
> int country_cmp (const void * a, const void * b)
> {
> struct mirror_st ** one = (struct mirror_st **)a;
> struct mirror_st ** two = (struct mirror_st **)b;
>
> // list the USA mirrors first, it will subsort correctly
> int8_t temp = !strncmp("USA", (*one)->country_title, 3);
> if (temp != !strncmp("USA", (*two)->country_title, 3))
> {
> if (temp)
> return -1;
> return 1;
> }
>
> return strcmp( (*one)->country_title, (*two)->country_title ) ;
> }
>
>
> double get_time_diff(struct timeval a, struct timeval b)
> {
>     long sec;
>     long usec;
>     sec = b.tv_sec - a.tv_sec;
>     usec = b.tv_usec - a.tv_usec;
>     if (usec < 0)
>     {
> --sec;
> usec += 1000000;
>     }
>     return sec + ((double)usec / 1000000.0);
> }
>
> void manpage(char * a)
> {
> errx(1, "%s [-s timeout] [-n maximum_mirrors_written]", a);
> }
>
>
> int main(int argc, char *argv[])
> {
> pid_t ftp_pid, sed_pid, uname_pid;
> int ftp_to_sed[2];
> int sed_to_parent[2];
> int uname_to_parent[2];
> char uname_r[5], uname_m[20], character;
> int i;
> double s = 7;
> int position, num, c, n = 5000;
> FILE *input;
>
> if (argc > 1)
> {
> if (argc % 2 == 0)
> manpage(argv[0]);
>
> position = 0;
> while (++position < argc)
> {
> if (strlen(argv[position]) != 2)
> manpage(argv[0]);
>
> if (!strcmp(argv[position], "-s"))
> {
> ++position;
> c = -1;
> i = 0;
> while ((character = argv[position][++c]) != '\0')
> {
> if (character == '.')
> ++i;
>
> if ( ((character < '0' || character > '9') && character != '.') || i > 1 )
> {
> if (character == '-')
> errx(1, "No negative numbers.");
> errx(1, "Incorrect floating point format.");
> }
> }
> errno = 0;
> strtod(argv[position], NULL);
> if (errno == ERANGE)
> err(1, NULL);
> if ((s = strtod(argv[position], NULL)) > 100000000.0)
> errx(1, "The argument should less than or equal to 100000000");
> }
> else if (!strcmp(argv[position], "-n"))
> {
> ++position;
> if (strlen(argv[position]) > 3)
> errx(1, "Integer should be less than or equal to 3 digits long.");
> c = -1;
> n = 0;
> while ((character = argv[position][++c]) != '\0')
> {
> if ( character < '0' || character > '9' )
> {
> if (character == '.')
> errx(1, "No decimal points.");
> if (character == '-')
> errx(1, "No negative numbers.");
> errx(1, "Incorrect integer format.");
> }
> n = n * 10 + (int)(character - '0');
> }
> }
> else
> manpage(argv[0]);
> }
> }
>
> struct kevent ke[1];
> struct timespec timeout;
>
> timeout.tv_sec = (int)s;
> timeout.tv_nsec = (int)(   (s - (double)timeout.tv_sec) * 1000000000  );
>
> int kq = kqueue();
> if (kq == -1)
> errx(1, "kq!");
>
> if (pipe(uname_to_parent) == -1)
> err(1, NULL);
>
>
> // "uname -rm" returns version and architecture like: "5.8 amd64\n" to standard out
> uname_pid = fork();
> if (uname_pid == (pid_t) 0)
> { /* uname child */
> close(uname_to_parent[0]);
> dup2(uname_to_parent[1], STDOUT_FILENO);         /*attaching to pipe(s)*/
> execl("/usr/bin/uname","/usr/bin/uname", "-rm", NULL);
>         errx(1, "uname execl() failed.");
> }
> if (uname_pid == -1)
> err(1, NULL);
>
> close(uname_to_parent[1]);
>
>
> EV_SET(ke, uname_to_parent[0], EVFILT_READ, EV_ADD | EV_ONESHOT, 0, 0, NULL);
> if (kevent(kq, ke, 1, NULL, 0, NULL) == -1)
> {
> kill(uname_pid, SIGKILL);
> errx(1, "kevent register fail.");
> }
>
> i = kevent(kq, NULL, 0, ke, 1, &timeout);
> if (i == -1)
> {
> kill(uname_pid, SIGKILL);
> errx(1, "kevent uname");
> }
>
> if (i == 0)
> {
> kill(uname_pid, SIGKILL);
> errx(1, "uname timed out.");
> }
>
> input = fdopen (uname_to_parent[0], "r");
> if (input == NULL)
> err(1, NULL);
>
> num = 0;
> position = -1;
> while ((c = getc(input)) != EOF)
> {
> if (num == 0)
> {
> if (++position >= 5)
> errx(1, "uname_r[] got too long!");
> if (c != ' ')
> uname_r[position] = c;
> else
> {
> uname_r[position] = '\0';
> num = 1;
> position = -1;
> }
> }
> else
> {
> if (++position >= 20)
> errx(1, "uname_m[] got too long!");
> if (c != '\n')
> uname_m[position] = c;
> else
> {
> uname_m[position] = '\0';
> break;
> }
> }
> }
> fclose (input);
> close(uname_to_parent[0]);
>
>
> if (pipe(ftp_to_sed) == -1)
> err(1, NULL);
>
>
> ftp_pid = fork();
> if (ftp_pid == (pid_t) 0)
> { /*ftp child*/
> close(ftp_to_sed[0]);
> dup2(ftp_to_sed[1], STDOUT_FILENO);         /*attaching to pipe(s)*/
>         execl("/usr/bin/ftp","ftp", "-Vo", "-", "http://www.openbsd.org/ftp.html", NULL);
>         errx(1, "ftp execl() failed.");
> }
> if (ftp_pid == -1)
> err(1, NULL);
>
> close(ftp_to_sed[1]);
>
> if (pipe(sed_to_parent) == -1)
> err(1, NULL);
>
>
> sed_pid = fork();
> if (sed_pid == (pid_t) 0)
> { /* sed child */
> close(sed_to_parent[0]);
> dup2(ftp_to_sed[0], STDIN_FILENO);         /*attaching to pipe(s)*/
> dup2(sed_to_parent[1], STDOUT_FILENO);
> execl("/usr/bin/sed","sed","-n",
>   "-e", "s:</a>$::",
>   "-e", "s:\t<strong>\\([^<]*\\)<.*:\\1:p",
>   "-e", "s:^\\(\t[hfr].*\\):\\1:p", NULL);
> kill(ftp_pid, SIGKILL);
>         errx(1, "sed execl() failed.");
> }
> if (sed_pid == -1)
> {
> kill(ftp_pid, SIGKILL);
> err(1, NULL);
> }
>
> EV_SET(ke, sed_to_parent[0], EVFILT_READ, EV_ADD | EV_ONESHOT, 0, 0, NULL);
> if (kevent(kq, ke, 1, NULL, 0, NULL) == -1)
> {
> kill(ftp_pid, SIGKILL);
> kill(sed_pid, SIGKILL);
> errx(1, "sed_to_parent kevent register fail.");
> }
>
> close(ftp_to_sed[0]);         /*close pipe(s) child attached to*/
> close(sed_to_parent[1]);
>
> i = kevent(kq, NULL, 0, ke, 1, &timeout);
> if (i == -1)
> {
> kill(ftp_pid, SIGKILL);
> kill(sed_pid, SIGKILL);
> errx(1, "kevent, timeout may be too large");
> }
>
> if (i == 0)
> {
> kill(ftp_pid, SIGKILL);
> kill(sed_pid, SIGKILL);
> errx(1, "timed out fetching openbsd.org/ftp.html.");
> }
>    
> input = fdopen (sed_to_parent[0], "r");
> if (input == NULL)
> err(1, NULL);
>
> char line[300];
> num = 0;
> position = 0;
> struct mirror_st *start, *end, *mTemp1, *mTemp2, *mTemp3;
> start = end = malloc(sizeof(struct mirror_st));
> if (start == NULL)
> errx(1, "malloc failed.");
> start->next = NULL;
>
> while ((c = getc (input)) != EOF)
> {
> if (position >= 300)
> {
> kill(ftp_pid, SIGKILL);
> kill(sed_pid, SIGKILL);
> errx(1, "line[] got too long!");
> }
> if (num == 0)
> {
> if ( c != '\n' )
> line[position++] = c;
> else
> {
> // there is a space before the newline that is eliminated
> line[position++] = '\0';
>
> end->country_title = malloc(position);
> if (end->country_title == NULL)
> {
> kill(ftp_pid, SIGKILL);
> kill(sed_pid, SIGKILL);
> errx(1, "malloc failed.");
> }
>
> strlcpy(end->country_title, line, position);
>
> position = 0;
> num = 1;
> }
> }
> else
> {
> if (position == 0)
> {
> if ( (c != 'h') && (c != 'f') && (c != 'r') )
> continue;
> else if (c == 'r')
> break;
> else if (c == 'f')
> {
> // changes ftp listings to http. ftp.html says they can be either
> line[position++] = 'h';
> line[position++] = 't';
> continue;
> }
> }
> if ( c != '\n' )
> line[position++] = c;
> else
> {
> line[position++] = '\0';
>
> position += num = strlen(uname_r) + 10 + strlen(uname_m) + 7;
> end->mirror = malloc(position);
> if (end->mirror == NULL)
> {
> kill(ftp_pid, SIGKILL);
> kill(sed_pid, SIGKILL);
> errx(1, "malloc failed.");
> }
> c =  strlcpy(end->mirror,             line, position);
> c += strlcpy(end->mirror + c,       uname_r, position - c);
> c += strlcpy(end->mirror + c, "/packages/", position - c);
> c += strlcpy(end->mirror + c,       uname_m, position - c);
>     strlcpy(end->mirror + c,    "/SHA256", position - c);
>
> position += 15 - num;
> end->install_path = malloc(position);
> if (end->install_path == NULL)
> {
> kill(ftp_pid, SIGKILL);
> kill(sed_pid, SIGKILL);
> errx(1, "malloc failed.");
> }
> c = strlcpy(end->install_path,                  line, position);
>    strlcpy(end->install_path + c, "%c/packages/%a/", position - c);
>
>
> end = end->next = malloc(sizeof(struct mirror_st));
> if (end == NULL)
> {
> kill(ftp_pid, SIGKILL);
> kill(sed_pid, SIGKILL);
> errx(1, "malloc failed.");
> }
> end->next = NULL;
>
> position = 0;
> num = 0;
> }
> }
> }
>
> fclose (input);
>
> // this is to prevent possible segfault in the next sequence.
> if (start == end)
> errx(1, "Too few entries found.");
>
> mTemp1 = start;
> while (mTemp1->next != end)
> mTemp1 = mTemp1->next;
> free(end->country_title);
> free(end);
> mTemp1->next = NULL;
>
> // Eliminate redundant mirrors (no longer need the 'end' value)
> mTemp1 = start;
> while (mTemp1->next != NULL)
> {
> mTemp2 = mTemp1;
> do
> {
> if (!strcmp(mTemp1->mirror, mTemp2->next->mirror))
> {
> mTemp3 = mTemp2->next;
> mTemp2 = mTemp2->next = mTemp3->next;
>
> free(mTemp3->country_title);
> free(mTemp3->install_path);
> free(mTemp3->mirror);
> free(mTemp3);
>
> if (mTemp2 == NULL)
> break;
> }
> else
> mTemp2 = mTemp2->next;
>
> } while ( mTemp2->next != NULL );
> mTemp1 = mTemp1->next;
> }
>
> int mirror_num = 0;
> mTemp1 = start;
> while (mTemp1 != NULL)
> {
> ++mirror_num;
> mTemp1 = mTemp1->next;
> }
>
> struct mirror_st ** array;
>
> if ( (array = calloc(mirror_num, sizeof(struct mirror_st *))) == NULL)
> errx(1, "calloc failed.");
>
> mTemp1 = start;
> for (c = 0; c < mirror_num; ++c)
> {
> array[c] = mTemp1;
>
> mTemp1 = mTemp1->next;
> }
>
> qsort(array, mirror_num, sizeof(struct mirror_st *), country_cmp);
>
> close(sed_to_parent[0]);
>
> for (c = 0; c < mirror_num; ++c)
> {
> mTemp1 = array[c];
>
> printf("\n%d : %s  :  %s\n", (mirror_num - c) - 1, mTemp1->country_title, mTemp1->mirror);
>
> ftp_pid = fork();
> if (ftp_pid == (pid_t) 0)
> { /* ping child */
> execl("/usr/bin/ftp", "ftp", "-Vmo", "/dev/null", mTemp1->mirror, NULL);
> errx(1, "ftp execl() failed.");
> }
> if (ftp_pid == -1)
> err(1, NULL);
> EV_SET(ke, ftp_pid, EVFILT_PROC, EV_ADD | EV_ONESHOT, NOTE_EXIT, 0, NULL);
> if (kevent(kq, ke, 1, NULL, 0, NULL) == -1)
> {
> kill(ftp_pid, SIGKILL);
> errx(1, "kevent register fail.");
> }
>
> mTemp1->diff = 0;
> struct timeval tv_start, tv_end;
> gettimeofday(&tv_start, NULL);
>
> skip:
>
> i = kevent(kq, NULL, 0, ke, 1, &timeout);
> if (i == -1)
> {
> kill(ftp_pid, SIGKILL);
> errx(1, "kevent");
> }
> if (i == 0)
> {
> printf("\n");
> kill(ftp_pid, SIGKILL);
> mTemp1->diff = timeout.tv_sec + (double)timeout.tv_nsec / 1000000000.0;
>
> // Loop until ftp() is dead.
> goto skip;
> }
>
> if ( ke->data == 0 )
> {
> gettimeofday(&tv_end, NULL);
>
> mTemp1->diff = get_time_diff(tv_start, tv_end);
> }
> else if ( mTemp1->diff == 0 )
> mTemp1->diff  =  timeout.tv_sec + (double)timeout.tv_nsec / 1000000000.0 + 1;
> }
>
> qsort(array, mirror_num, sizeof(struct mirror_st *), ftp_cmp);
>
> printf("\n\n");
>
> for (c = mirror_num - 1; c >= 0; --c)
> {
> printf("%d : %s:\n\t%s : ", c, array[c]->country_title, array[c]->install_path);
>
> if (array[c]->diff < timeout.tv_sec + (double)timeout.tv_nsec / 1000000000.0)
> printf("%f\n\n", array[c]->diff);
> else if (array[c]->diff == timeout.tv_sec + (double)timeout.tv_nsec / 1000000000.0)
> printf("Timeout\n\n");
> else
> printf("Download Error\n\n");
> }
>
> if (array[0]->diff >= timeout.tv_sec + (double)timeout.tv_nsec / 1000000000.0)
> errx(1, "No mirrors found within timeout period.");
>
> char *buf;
> int lines;
> int total_length = 0;
> int copy = 0;
>
> for (position = 0; position < mirror_num; ++position)
> total_length += strlen("installpath += \n") + strlen(array[position]->install_path);
>
> if ( (input = fopen("/etc/pkg.conf", "r")) == NULL )
> {
> buf = (char*)malloc(total_length + 1);
> if (buf == NULL)
> errx(1, "malloc failed.");
> }
> else
> {
> fseek(input, 0, SEEK_END);
> num = ftell(input);
> fseek(input, 0, SEEK_SET);
> lines = 0;
> buf = (char*)malloc(num + total_length + 11 + 1);
> if (buf == NULL)
> errx(1, "malloc failed.");
> fread(buf, 1, num, input);
> fclose(input);
> for (c = 0; c < num; ++c)
> {
> if (buf[c] == '\n')
> ++lines;
> }
>
> position = 0;
> for (c = 0; c < num; ++c)
> {
> if (buf[c] == '\n')
> {
> if (--lines == 0 && position == 0)
> break; // get rid of the lonely newline at the end, if it exists
> position = 0;
> }
> else if (position++ == 0)
> {
> if ( !strncmp(buf + c, "installpath", 11) )
> {
> if (lines)
> {
> while (buf[++c] != '\n');
> --lines;
> position = 0;
> continue;
> }
> else
> break;
> }
> }
> buf[copy++] = buf[c];
> }
> total_length += num;
> }
>
> copy += strlcpy(buf + copy,      "installpath = ", total_length - copy);
> copy += strlcpy(buf + copy, array[0]->install_path, total_length - copy);
>
> if (n < mirror_num)
> mirror_num = n;
>
> position = 1;
> while (position < mirror_num)
> {
> // Eliminates dowload error and timeout mirrors
> if (array[position]->diff >= timeout.tv_sec + (double)timeout.tv_nsec / 1000000000.0)
> break;
> copy += strlcpy(buf + copy,            "\ninstallpath += ", total_length - copy);
> copy += strlcpy(buf + copy, array[position++]->install_path, total_length - copy);
> }
> copy += strlcpy(buf + copy, "\n", total_length - copy);
>
>
> input = fopen("/etc/pkg.conf", "w");
> if (input != NULL)
> {
> fwrite(buf, 1, copy, input);
> fclose(input);
> printf("\n\nEdit out all PKG_PATH environment variable exports");
> printf(" and run \"unset PKG_PATH\".\n\n/etc/pkg.conf:\n%s\n", buf);
> }
> else
> {
> printf("\n\nThis could have been the contents of /etc/pkg.conf (run as superuser):\n");
> printf("%s\n", buf);
> }
>
> if (argc == 1)
> printf("%s [-s timeout] [-n maximum_mirrors_written]\n", argv[0]);
>
> return 0;
> }

Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Peter J. Philipp-3
Luke, don't feel bad.  Very little code that is "offered" gets taken by
the OpenBSD project.  OpenBSD really only takes when they see benefit
for the project.  An example for that is openssh.  What you really want
to do is focus on your own projects and make them available somewhere so
that when OpenBSD gets wind of it they'll take it.

Cheers,

-peter

On 01/29/16 09:19, Nicholas Marriott wrote:

> Firstly, I don't think we need this in base and I think there is little
> to no chance of it being taken, even if the code is improved.
>
> Secondly:
>
> - The code is still miles off style(9) and isn't really a consistent
>   style within itself either.
>
> - Forking uname(1)? What? No offence, but that is hilarious :-). Why
>   fork uname(1) for uname(3) but not date(1) for gettimeofday(2)?
>
> - Why would you fork sed either?
>
> I think C is the wrong tool for this. Why not write a shell, perl, or
> python script?
>
> Then if people start to use it you could make a port.
>
>
>
> On Fri, Jan 29, 2016 at 01:34:30AM -0600, Luke Small wrote:
>> I think I fixed all your suggestions. I don't strictly adhere to kernel
>> normal in the use of comments and I parse command-line arguments without
>> using getopt(3), but the method is robust.
>>
>> -Luke
>>
>> <A few quick comments from glancing over this:
>>
>>  o I definitely don't think camel case will be accepted
>>
>>  o I'm pretty sure strtonum(3) is strongly preferred over strtod(3) et al.
>>

Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

lists-2
In reply to this post by Nicholas Marriott-2
Fri, 29 Jan 2016 08:19:14 +0000 Nicholas Marriott
> Firstly, I don't think we need this in base and I think there is little
> to no chance of it being taken, even if the code is improved.

Many folks tried this part (advising Luke), he takes none and keeps
repeating wrong concepts, his assignment looks misaligned somewhat.

> Secondly:
>
> - The code is still miles off style(9) and isn't really a consistent
>   style within itself either.

This comes from an apprentice wannabe, probably best to recommend him
further self help.

> - Forking uname(1)? What? No offence, but that is hilarious :-). Why
>   fork uname(1) for uname(3) but not date(1) for gettimeofday(2)?

The kid knows nothing of UNIX, ask him book reading comprehension
questions in private please.

> - Why would you fork sed either?

Hint: suggest another list@

> I think C is the wrong tool for this. Why not write a shell, perl, or
> python script?

C is the wrong tool for that person, knows nothing of shell too.  So
best pick learning shell first.  Typical, but never hopeless (still)?

> Then if people start to use it you could make a port.

Without thought at design stage, barely usable for private learning
projects homework.  The result is reiterations on misc@ where ideas
spark in developer heads after some kid starts asking noisily without
listening or prior knowledge.

Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Luke Small
I wanted to use kqueue. Name another script or programming language that
offers it from the base install. NONE!

Why should I write it in another language. I already did it in C. Is there
another way other than kqueue that you can wait for the ftp call to quit,
while being able to kill it if it takes too long?

-Luke

On Fri, Jan 29, 2016 at 3:42 AM, <[hidden email]> wrote:

> Fri, 29 Jan 2016 08:19:14 +0000 Nicholas Marriott
> > Firstly, I don't think we need this in base and I think there is little
> > to no chance of it being taken, even if the code is improved.
>
> Many folks tried this part (advising Luke), he takes none and keeps
> repeating wrong concepts, his assignment looks misaligned somewhat.
>
> > Secondly:
> >
> > - The code is still miles off style(9) and isn't really a consistent
> >   style within itself either.
>
> This comes from an apprentice wannabe, probably best to recommend him
> further self help.
>
> > - Forking uname(1)? What? No offence, but that is hilarious :-). Why
> >   fork uname(1) for uname(3) but not date(1) for gettimeofday(2)?
>
> The kid knows nothing of UNIX, ask him book reading comprehension
> questions in private please.
>
> > - Why would you fork sed either?
>
> Hint: suggest another list@
>
> > I think C is the wrong tool for this. Why not write a shell, perl, or
> > python script?
>
> C is the wrong tool for that person, knows nothing of shell too.  So
> best pick learning shell first.  Typical, but never hopeless (still)?
>
> > Then if people start to use it you could make a port.
>
> Without thought at design stage, barely usable for private learning
> projects homework.  The result is reiterations on misc@ where ideas
> spark in developer heads after some kid starts asking noisily without
> listening or prior knowledge.
>
Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Jeremie Courreges-Anglas-2
Luke Small <[hidden email]> writes:

> I wanted to use kqueue. Name another script or programming language that
> offers it from the base install. NONE!

If you want to discover how to use kqueue, fine, but that's not how
design decisions are done in OpenBSD land.

> Why should I write it in another language. I already did it in C. Is there
> another way other than kqueue that you can wait for the ftp call to quit,
> while being able to kill it if it takes too long?

Yes, there are other ways. There are also ways that don't involve
ftp(1), sed(1) and uname(1).

Luke, sorry if it sounds blunt but your code is just not good enough to
be accepted into base.  You've probably learned some things when writing
this program, and maybe it fits your use case, but that's all.

Aside from that I've never felt the need for such kind of program, and
I don't feel like there's much demand from others.

Cheers,
--
jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE

Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Loganaden Velvindron-2
On Fri, Jan 29, 2016 at 12:44 PM, Jérémie Courrèges-Anglas <[hidden email]>
wrote:

> Luke Small <[hidden email]> writes:
>
> > I wanted to use kqueue. Name another script or programming language that
> > offers it from the base install. NONE!
>
>
>
Hi Luke,

I understand your perspective. If you use OpenBSD already, then I would
suggest you start by fixing bugs in documentation as you encounter them or
small fixes, as a start.

Look at what OpenBSD is currently heading: tame, improvements to OpenBGPd,
and the crazy W^X stuff.

(url: http://www.openbsd.org/papers/).

If you are looking to work for something interesting, here is a good start.
Run -current, and if you encounter a bug, try fixing it, and think about
ways to improve your small fixes.
Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Luke Small
In reply to this post by Jeremie Courreges-Anglas-2
I'm not merely experimenting with kqueue because I like the shiny bells and
whistles. I want to know how fast a mirror will download the same file from
different mirrors. ftp() is shitty for expediency. It does one of three
things it fails fast, succeeds fast, or it could take FOREVERRRRRR!!! I
want to detect all three of these scenarios and stop it if it takes
forever. So I call kqueue to time how long it takes ftp to run. If it takes
too long, I kill it. I don't know of any other calls that can do this other
than kqueue. And in a fresh install with absolutely no packages, I think
the only way to do it is by using C.

-Luke

On Fri, Jan 29, 2016 at 6:44 AM, Jérémie Courrèges-Anglas <[hidden email]>
wrote:

> Luke Small <[hidden email]> writes:
>
> > I wanted to use kqueue. Name another script or programming language that
> > offers it from the base install. NONE!
>
> If you want to discover how to use kqueue, fine, but that's not how
> design decisions are done in OpenBSD land.
>
> > Why should I write it in another language. I already did it in C. Is
> there
> > another way other than kqueue that you can wait for the ftp call to quit,
> > while being able to kill it if it takes too long?
>
> Yes, there are other ways. There are also ways that don't involve
> ftp(1), sed(1) and uname(1).
>
> Luke, sorry if it sounds blunt but your code is just not good enough to
> be accepted into base.  You've probably learned some things when writing
> this program, and maybe it fits your use case, but that's all.
>
> Aside from that I've never felt the need for such kind of program, and
> I don't feel like there's much demand from others.
>
> Cheers,
> --
> jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE
>
Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Luke Small
In reply to this post by Nicholas Marriott-2
I fixed the uname(1) call and replaced it with uname(3) I read the style
man page. ran the program through indent.

I ran it through sed because it reduces code complexity. Why re-engineer
the wheel?

I use C because I can use kqueue from a fresh install. You have to use
unaudited packages to use perl or python kqueue. I want the program to be
safe to run as root.

I use kqueue because I like it, but also because the mirror ftp calls need
to have a wait() call that can collect the status and can enforce a timeout
period. ftp can be a bitch that runs without stopping if you let it. I'm
not willing to let it run for hours, unless the user specifically lets the
timeout period be hours, where I've written it to allow that.

-Luke

On Fri, Jan 29, 2016 at 2:19 AM, Nicholas Marriott <
[hidden email]> wrote:

> Firstly, I don't think we need this in base and I think there is little
> to no chance of it being taken, even if the code is improved.
>
> Secondly:
>
> - The code is still miles off style(9) and isn't really a consistent
>   style within itself either.
>
> - Forking uname(1)? What? No offence, but that is hilarious :-). Why
>   fork uname(1) for uname(3) but not date(1) for gettimeofday(2)?
>
> - Why would you fork sed either?
>
> I think C is the wrong tool for this. Why not write a shell, perl, or
> python script?
>
> Then if people start to use it you could make a port.
>
>

pkg_ping.c (24K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Loganaden Velvindron-2
On Mon, Feb 1, 2016 at 6:18 AM, Luke Small <[hidden email]> wrote:

> I fixed the uname(1) call and replaced it with uname(3) I read the style
> man page. ran the program through indent.
>
>
2 seasoned OpenBSD developers have taken time to reply to you, and they do
not like the general idea. No seasoned OpenBSD developer has shown any
interest. I suggest you drop the discussion.
Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Luke Small
In reply to this post by Luke Small
Whoops, got rid of putting in a null character when I should have left it
in.

-Luke

pkg_ping.c (24K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Jorge Castillo
I can't comment on code quality since I suck at programming but you
yourself said your program does not follow style(9) as much as it could, I
think this is not a good start. Why not make it a port? If this becomes
useful to a lot of people then maybe it can be in base later, but not
before it shows to be wildly popular. To be frank a file with all available
mirrors is as good as it gets for me, I've never felt the need to see which
mirror is faster, the only though that has come to me concerning mirrors,
while using OpenBSD all this years is "damn it sure would be nice to know
which mirrors there are without connecting to the internet first". To
anyone reading this have a nice day.
Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Stuart Henderson-6
On 2016/02/01 08:43, Jorge Castillo wrote:
>                   the only though that has come to me concerning mirrors,
> while using OpenBSD all this years is "damn it sure would be nice to know
> which mirrors there are without connecting to the internet first".

Fixed in -current, see /etc/examples/pkg.conf.

> To anyone reading this have a nice day.

thanks :)

Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Dmitrij D. Czarkoff-2
In reply to this post by Jorge Castillo
Jorge Castillo said:
> Why not make it a port?

Making port for figuring out PKGPATH doesn't sound right.

See, there are four problems with the program:

1.  It is not good enough in doing its job.  Which is funny, because
    picking right mirror is trivially done without any program.
2.  It uses external tools for tasks that could be trivially implemented
    in C.
3.  It doesn't filter out obviously bad choices, eg. users in Europe
    will test mirrors in North America.
4.  It doesn't meet OpenBSD's standards for code in base.

Problems #2, #3 and #4 can be fixed, but problem #1 makes this
discussion completely pointless.  Provided that all OpenBSD developers
who cared to participate in this discussion pointed out this issue, I'd
suggest to stop wasting time and bandwidth right here.

Luke, if you disagree with my assessment, please publish your program on
github and convince tech media to mention it.  And move to next thing.
Thank you in advance.

--
Dmitrij D. Czarkoff

Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Luke Small
1. You can pick a mirror relatively trivially, but since I've run the
program, the fastest one isn't the one I chose manually. Also, it can
choose multiple mirrors at once, so presumably if there is a failure, it
will choose the next mirror(s) that it wrote down in pkg.conf

2. You are saying that the ftp protocol can be implemented trivially? You
are ridiculous sir.

3. How do you suggest I filter out obviously bad choices. Add on a perl
geolocation package that isn't available in a base install. How about I
just ftp download a smaller file to discover the latency.

4. How doesn't it meet standards. I wrote it according to the style man
page as far as I can tell. And I ran it through indent. Even though I think
kernel normal form is less readable.

I think that there is some unwritten policy that nobody can get something
like this into the system. Why on earth hasn't this happened yet?
On Feb 1, 2016 10:48, "Dmitrij D. Czarkoff" <[hidden email]> wrote:

> Jorge Castillo said:
> > Why not make it a port?
>
> Making port for figuring out PKGPATH doesn't sound right.
>
> See, there are four problems with the program:
>
> 1.  It is not good enough in doing its job.  Which is funny, because
>     picking right mirror is trivially done without any program.
> 2.  It uses external tools for tasks that could be trivially implemented
>     in C.
> 3.  It doesn't filter out obviously bad choices, eg. users in Europe
>     will test mirrors in North America.
> 4.  It doesn't meet OpenBSD's standards for code in base.
>
> Problems #2, #3 and #4 can be fixed, but problem #1 makes this
> discussion completely pointless.  Provided that all OpenBSD developers
> who cared to participate in this discussion pointed out this issue, I'd
> suggest to stop wasting time and bandwidth right here.
>
> Luke, if you disagree with my assessment, please publish your program on
> github and convince tech media to mention it.  And move to next thing.
> Thank you in advance.
>
> --
> Dmitrij D. Czarkoff
>
Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

Janne Johansson-3
2016-02-01 22:31 GMT+01:00 Luke Small <[hidden email]>:

>
> I think that there is some unwritten policy that nobody can get something
> like this into the system. Why on earth hasn't this happened yet?
>
>
Yeah, I think that if anyone has something they want into base it should go
in,
even if the developers don't like it. Couldn't end in a bad way, could it?

--
May the most significant bit of your life be positive.
Reply | Threaded
Open this post in threaded view
|

Re: I have a program I wish to submit for the base

phessler
In reply to this post by Luke Small
On 2016 Feb 01 (Mon) at 15:31:25 -0600 (-0600), Luke Small wrote:
:I think that there is some unwritten policy that nobody can get something
:like this into the system. Why on earth hasn't this happened yet?

Your proposed addition has been rejected, and will continue to be rejected.

1) The code is not up to the quality standards of OpenBSD.

2) The feature is not desired.

3) We are not interested in "debating" this with you, please drop the subject.