A suggestion for snapshots

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

A suggestion for snapshots

Lars Engblom
Quite often the snapshot of the packages and the base system are out of
sync, because naturally, the base has to be built before packages.

For example in this moment, as I write this, Firefox can not be
installed in a new system installed from snapshots, as the packages are
compiled against an older snapshot (amd64)

If there are just space on the ftp servers, I would suggest keeping two
snapshots: one complete with both base and packages (always in sync) and
one with just the newest base. This would make life easier for people
following snapshots.

Regards,
  Lasse

Reply | Threaded
Open this post in threaded view
|

Re: A suggestion for snapshots

Marc Espie-2
On Fri, Sep 06, 2013 at 03:14:44PM +0300, Lars Engblom wrote:
> For example in this moment, as I write this, Firefox can not be
> installed in a new system installed from snapshots, as the packages
> are compiled against an older snapshot (amd64)
Known issue.

> If there are just space on the ftp servers, I would suggest keeping
> two snapshots:
That's the problem.  Requiring that would put a lot of mirrors out of the
game.

There are also bottlenecks in fanning out from the actual build machines.
Ports bulk builders are aware of the issues.  These take time to solve.

Reply | Threaded
Open this post in threaded view
|

Re: A suggestion for snapshots

Bryan Vyhmeister-3
On Fri, Sep 06, 2013 at 06:59:29PM +0200, Marc Espie wrote:
> There are also bottlenecks in fanning out from the actual build machines.
> Ports bulk builders are aware of the issues.  These take time to solve.

Would additional fast build machines help? Is that a large part of the
problem?

Bryan

Reply | Threaded
Open this post in threaded view
|

Re: A suggestion for snapshots

Marc Espie-2
On Fri, Sep 06, 2013 at 01:19:02PM -0700, Bryan Vyhmeister wrote:
> On Fri, Sep 06, 2013 at 06:59:29PM +0200, Marc Espie wrote:
> > There are also bottlenecks in fanning out from the actual build machines.
> > Ports bulk builders are aware of the issues.  These take time to solve.
>
> Would additional fast build machines help? Is that a large part of the
> problem?
>
> Bryan

Nope, I think that network bandwidth and topology is part of the current
problem, and lack of time from some of the people involved to fix it.

Reply | Threaded
Open this post in threaded view
|

Re: A suggestion for snapshots

Amit Kulkarni-5
In reply to this post by Lars Engblom
On Fri, Sep 6, 2013 at 7:14 AM, Lars Engblom
<[hidden email]>wrote:

> Quite often the snapshot of the packages and the base system are out of
> sync, because naturally, the base has to be built before packages.
>
> For example in this moment, as I write this, Firefox can not be installed
> in a new system installed from snapshots, as the packages are compiled
> against an older snapshot (amd64)
>
> If there are just space on the ftp servers, I would suggest keeping two
> snapshots: one complete with both base and packages (always in sync) and
> one with just the newest base. This would make life easier for people
> following snapshots.
>
> Regards,
>  Lasse
>
>
The problem with ports is that even with a build farm, the ports guy has to
make sure dpb runs to the end. In the best case, a dpb run WITHOUT problems
to the end takes atleast a day with a fast quad core machine. gcc4, JDK 1.6
+ 1.7, GTK+2, GTK+3, Qt4, Webkit, Firefox are some of the worst ports in
terms of build time and the largest offender Libreoffice which alone takes
4-6 hrs of all quad cores (Xeon E3-1230v2 3.3GHz). I might have missed some
offenders, I just built a subset, experienced porters who handle the whole
tree know better than me which ones are also worthy candidates.

Finding and fixing port problems takes a minimum of 2 and I am guessing
typically 4 days to pump out a wholly built ports tree, on a extremely fast
arch like amd64. By which time the userland, kernel and xenocara have
changed a lot underneath. Hence, you get these mismatches from time to
time. It is not catastrophic, solution is to wait for the next snap. Even
if the ports build machine untars userland, kernel, xenocara, running dpb
again may force rebuilds or sometimes not.

Reply | Threaded
Open this post in threaded view
|

Re: A suggestion for snapshots

Theo de Raadt
In reply to this post by Lars Engblom
> On Fri, Sep 6, 2013 at 7:14 AM, Lars Engblom
> <[hidden email]>wrote:
>
> > Quite often the snapshot of the packages and the base system are out of
> > sync, because naturally, the base has to be built before packages.
> >
> > For example in this moment, as I write this, Firefox can not be installed
> > in a new system installed from snapshots, as the packages are compiled
> > against an older snapshot (amd64)
> >
> > If there are just space on the ftp servers, I would suggest keeping two
> > snapshots: one complete with both base and packages (always in sync) and
> > one with just the newest base. This would make life easier for people
> > following snapshots.
> >
> > Regards,
> >  Lasse
> >
> >
> The problem with ports is that even with a build farm, the ports guy has to
> make sure dpb runs to the end. In the best case, a dpb run WITHOUT problems
> to the end takes atleast a day with a fast quad core machine. gcc4, JDK 1.6
> + 1.7, GTK+2, GTK+3, Qt4, Webkit, Firefox are some of the worst ports in
> terms of build time and the largest offender Libreoffice which alone takes
> 4-6 hrs of all quad cores (Xeon E3-1230v2 3.3GHz). I might have missed some
> offenders, I just built a subset, experienced porters who handle the whole
> tree know better than me which ones are also worthy candidates.
>
> Finding and fixing port problems takes a minimum of 2 and I am guessing
> typically 4 days to pump out a wholly built ports tree, on a extremely fast
> arch like amd64. By which time the userland, kernel and xenocara have
> changed a lot underneath. Hence, you get these mismatches from time to
> time. It is not catastrophic, solution is to wait for the next snap. Even
> if the ports build machine untars userland, kernel, xenocara, running dpb
> again may force rebuilds or sometimes not.

Anyone want to pay for a faster network link?

Step up -- then we can solve this problem easily.

Reply | Threaded
Open this post in threaded view
|

Re: A suggestion for snapshots

Lars Engblom
In reply to this post by Lars Engblom
I think the issue is more about space than bandwidth. It's after all only the mirrors that would downloads both base snapshots.

A normal snapshot user would only use one of the bases.

On the opposite, it would save some bandwidth, as I'm probably not the only one that has been sometimes downloading the whole package and base folder during package freeze to have something in sync for new installs during the time when base might be out of sync for longer times. 

-------- Original message --------
From: Theo de Raadt <[hidden email]>
Date: 07/09/2013  06:13  (GMT+02:00)
To: Amit Kulkarni <[hidden email]>
Cc: Lars Engblom <[hidden email]>,misc <[hidden email]>
Subject: Re: A suggestion for snapshots
 

> On Fri, Sep 6, 2013 at 7:14 AM, Lars Engblom
> <[hidden email]>wrote:
>
> > Quite often the snapshot of the packages and the base system are out of
> > sync, because naturally, the base has to be built before packages.
> >
> > For example in this moment, as I write this, Firefox can not be installed
> > in a new system installed from snapshots, as the packages are compiled
> > against an older snapshot (amd64)
> >
> > If there are just space on the ftp servers, I would suggest keeping two
> > snapshots: one complete with both base and packages (always in sync) and
> > one with just the newest base. This would make life easier for people
> > following snapshots.
> >
> > Regards,
> >  Lasse
> >
> >
> The problem with ports is that even with a build farm, the ports guy has to
> make sure dpb runs to the end. In the best case, a dpb run WITHOUT problems
> to the end takes atleast a day with a fast quad core machine. gcc4, JDK 1.6
> + 1.7, GTK+2, GTK+3, Qt4, Webkit, Firefox are some of the worst ports in
> terms of build time and the largest offender Libreoffice which alone takes
> 4-6 hrs of all quad cores (Xeon E3-1230v2 3.3GHz). I might have missed some
> offenders, I just built a subset, experienced porters who handle the whole
> tree know better than me which ones are also worthy candidates.
>
> Finding and fixing port problems takes a minimum of 2 and I am guessing
> typically 4 days to pump out a wholly built ports tree, on a extremely fast
> arch like amd64. By which time the userland, kernel and xenocara have
> changed a lot underneath. Hence, you get these mismatches from time to
> time. It is not catastrophic, solution is to wait for the next snap. Even
> if the ports build machine untars userland, kernel, xenocara, running dpb
> again may force rebuilds or sometimes not.

Anyone want to pay for a faster network link?

Step up -- then we can solve this problem easily.

Reply | Threaded
Open this post in threaded view
|

Re: A suggestion for snapshots

STeve Andre'
In reply to this post by Theo de Raadt
On 09/06/13 23:13, Theo de Raadt wrote:

>> On Fri, Sep 6, 2013 at 7:14 AM, Lars Engblom
>> <[hidden email]>wrote:
>>
>>> Quite often the snapshot of the packages and the base system are out of
>>> sync, because naturally, the base has to be built before packages.
>>>
>>> For example in this moment, as I write this, Firefox can not be installed
>>> in a new system installed from snapshots, as the packages are compiled
>>> against an older snapshot (amd64)
>>>
>>> If there are just space on the ftp servers, I would suggest keeping two
>>> snapshots: one complete with both base and packages (always in sync) and
>>> one with just the newest base. This would make life easier for people
>>> following snapshots.
>>>
>>> Regards,
>>>   Lasse
>>>
>>>
>> The problem with ports is that even with a build farm, the ports guy has to
>> make sure dpb runs to the end. In the best case, a dpb run WITHOUT problems
>> to the end takes atleast a day with a fast quad core machine. gcc4, JDK 1.6
>> + 1.7, GTK+2, GTK+3, Qt4, Webkit, Firefox are some of the worst ports in
>> terms of build time and the largest offender Libreoffice which alone takes
>> 4-6 hrs of all quad cores (Xeon E3-1230v2 3.3GHz). I might have missed some
>> offenders, I just built a subset, experienced porters who handle the whole
>> tree know better than me which ones are also worthy candidates.
>>
>> Finding and fixing port problems takes a minimum of 2 and I am guessing
>> typically 4 days to pump out a wholly built ports tree, on a extremely fast
>> arch like amd64. By which time the userland, kernel and xenocara have
>> changed a lot underneath. Hence, you get these mismatches from time to
>> time. It is not catastrophic, solution is to wait for the next snap. Even
>> if the ports build machine untars userland, kernel, xenocara, running dpb
>> again may force rebuilds or sometimes not.
> Anyone want to pay for a faster network link?
>
> Step up -- then we can solve this problem easily.
>
>
OK.  How much would it cost per month for faster access?  Do you have
several options for increased speeds?

I smell a fundraiser here--paying for a year's costs in advance. Perhaps
then others would come up with larger chunks for future costs.  It
would certainly be bad to not be able to come up with the funds for
the future net costs.  I think it should be thought of as another cost,
just like new hardware.

--STeve Andre'

Reply | Threaded
Open this post in threaded view
|

Re: A suggestion for snapshots

Stuart Henderson
In reply to this post by Lars Engblom
On 2013-09-07, Lars Engblom <[hidden email]> wrote:
> I think the issue is more about space than bandwidth.

No, it's about bandwidth, first from the build machines to the main
distribution site (fanout), and then from the fanout to mirrors.
Between them, amd64 and i386 packages take ~40GB, then there are
the other slower arch which are pushed out a bit less often.
Feeding this out across the mirroring infrastructure takes time.

> From: Theo de Raadt <[hidden email]>
> Date: 07/09/2013  06:13  (GMT+02:00)
>
> Anyone want to pay for a faster network link?
>
> Step up -- then we can solve this problem easily.

Of course this is only easy now that Theo and a few others have
recently put significant effort into improving the internet
infrastructure in Alberta.