Alternative way to get sources, CVS slow

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Alternative way to get sources, CVS slow

Henrique Lengler-2
Hi,

I'm trying to keep and maintain a copy of -stable source code in my system.
The problem is that I can't work with CVS.
It is too slow to download. My internet can download things at 500kb/s - 1MB/s,
but when I am doing a checkout, the download stay too slow, I already tried with a
bunch of mirrors, and it is almost the same thing. I don't know if the problem is that
all CVS mirrors are slow.
I let my computer all the last day doing a checkout for src ports and xenocara. After
10 hours it was in ports yet.

Also, the worst problem is that my internet connections sometimes drop, and so in an
attempt to continue the interrupted checkout I run the command again, the problem is
that it takes about a half hour to start to continue getting the code, this when it
works, because sometimes after all this time I get:
Write failed: Broken pipe
Also sometimes looks like it is getting everything again, because it says
cvs server: Updating foo
to every file already in the folder.

I'm looking to a way to get this sources, there is a long time I'm trying and it
ever happens some problem like this, and I can't continue to get the sources, so I
need to delete everything a start again.

CVS looks too complicated and confuse.
I would like to be able to get from http, git, ftp or anything sipler and faster.
Is there any alternative?
--
Regards

Henrique Lengler

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Kenneth Gober
the OpenBSD CD-ROM set includes the -RELEASE sources.

using those as a starting point, updating to -STABLE should be very fast.

-ken

On Sat, Feb 21, 2015 at 1:06 PM, Henrique Lengler <[hidden email]>
wrote:

> Hi,
>
> I'm trying to keep and maintain a copy of -stable source code in my system.
> The problem is that I can't work with CVS.
> It is too slow to download. My internet can download things at 500kb/s -
> 1MB/s,
> but when I am doing a checkout, the download stay too slow, I already
> tried with a
> bunch of mirrors, and it is almost the same thing. I don't know if the
> problem is that
> all CVS mirrors are slow.
> I let my computer all the last day doing a checkout for src ports and
> xenocara. After
> 10 hours it was in ports yet.
>
> Also, the worst problem is that my internet connections sometimes drop,
> and so in an
> attempt to continue the interrupted checkout I run the command again, the
> problem is
> that it takes about a half hour to start to continue getting the code,
> this when it
> works, because sometimes after all this time I get:
> Write failed: Broken pipe
> Also sometimes looks like it is getting everything again, because it says
> cvs server: Updating foo
> to every file already in the folder.
>
> I'm looking to a way to get this sources, there is a long time I'm trying
> and it
> ever happens some problem like this, and I can't continue to get the
> sources, so I
> need to delete everything a start again.
>
> CVS looks too complicated and confuse.
> I would like to be able to get from http, git, ftp or anything sipler and
> faster.
> Is there any alternative?
> --
> Regards
>
> Henrique Lengler

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Alexander Hall
In reply to this post by Henrique Lengler-2
I believe the source tarballs avaliable on the CDs (and mirrors) come with CVS directories, so they might be a good start. The FAQ might shed some light on this.

/Alexander


On February 21, 2015 7:06:44 PM CET, Henrique Lengler <[hidden email]> wrote:

>Hi,
>
>I'm trying to keep and maintain a copy of -stable source code in my
>system.
>The problem is that I can't work with CVS.
>It is too slow to download. My internet can download things at 500kb/s
>- 1MB/s,
>but when I am doing a checkout, the download stay too slow, I already
>tried with a
>bunch of mirrors, and it is almost the same thing. I don't know if the
>problem is that
>all CVS mirrors are slow.
>I let my computer all the last day doing a checkout for src ports and
>xenocara. After
>10 hours it was in ports yet.
>
>Also, the worst problem is that my internet connections sometimes drop,
>and so in an
>attempt to continue the interrupted checkout I run the command again,
>the problem is
>that it takes about a half hour to start to continue getting the code,
>this when it
>works, because sometimes after all this time I get:
>Write failed: Broken pipe
>Also sometimes looks like it is getting everything again, because it
>says
>cvs server: Updating foo
>to every file already in the folder.
>
>I'm looking to a way to get this sources, there is a long time I'm
>trying and it
>ever happens some problem like this, and I can't continue to get the
>sources, so I
>need to delete everything a start again.
>
>CVS looks too complicated and confuse.
>I would like to be able to get from http, git, ftp or anything sipler
>and faster.
>Is there any alternative?
>--
>Regards
>
>Henrique Lengler

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Raf Czlonka-2
In reply to this post by Henrique Lengler-2
On Sat, Feb 21, 2015 at 06:06:44PM GMT, Henrique Lengler wrote:

> Hi,

Hi Henrique,

> I'm trying to keep and maintain a copy of -stable source code in my
> system.
> The problem is that I can't work with CVS.
> It is too slow to download. My internet can download things at 500kb/s
> - 1MB/s, but when I am doing a checkout, the download stay too slow, I
> already tried with a bunch of mirrors, and it is almost the same
> thing. I don't know if the problem is that all CVS mirrors are slow.
>
> I let my computer all the last day doing a checkout for src ports and
> xenocara. After 10 hours it was in ports yet.
>
> Also, the worst problem is that my internet connections sometimes
> drop, and so in an attempt to continue the interrupted checkout I run
> the command again, the problem is that it takes about a half hour to
> start to continue getting the code, this when it works, because
> sometimes after all this time I get:
>
> Write failed: Broken pipe
> Also sometimes looks like it is getting everything again, because it says
> cvs server: Updating foo
> to every file already in the folder.
>
> I'm looking to a way to get this sources, there is a long time I'm
> trying and it ever happens some problem like this, and I can't
> continue to get the sources, so I need to delete everything a start
> again.

I've been there and can only sympathise.

> CVS looks too complicated and confuse.

Probably because you don't know it, in which case anything will look
complicated: git, hg, etc.

> I would like to be able to get from http, git, ftp or anything simpler
> and faster.
> Is there any alternative

Luckily -stable doesn't change too often so you can simply download
tarballs with the -release code and use patches.

Alternatively, you can use CVSync[0] to get the code.

[0] http://www.openbsd.org/cvsync.html

Cheers,

Raf

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Kevin Lyda
On Sat, Feb 21, 2015 at 6:35 PM, Raf Czlonka <[hidden email]> wrote:
> > CVS looks too complicated and confuse.
> Probably because you don't know it, in which case anything will look
> complicated: git, hg, etc.

Actually most younger developers I've worked with do know git and find
things like svn and cvs incredibly hard to use. It is a case of what
you're used to being the easier thing.

Kevin

--
Kevin Lyda
Galway, Ireland

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Raf Czlonka-2
On Sat, Feb 21, 2015 at 06:41:25PM GMT, Kevin Lyda wrote:
> On Sat, Feb 21, 2015 at 6:35 PM, Raf Czlonka <[hidden email]> wrote:
> > > CVS looks too complicated and confuse.
> > Probably because you don't know it, in which case anything will look
> > complicated: git, hg, etc.
>
> Actually most younger developers I've worked with do know git and find
> things like svn and cvs incredibly hard to use. It is a case of what
> you're used to being the easier thing.

Hi Kevin,

That's essentially what I have written above:

s/ don't know/'re not used to/

Raf

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Sebastian Reitenbach
In reply to this post by Henrique Lengler-2
On Saturday, February 21, 2015 19:06 CET, Henrique Lengler <[hidden email]> wrote:
 

> Hi,
>
> I'm trying to keep and maintain a copy of -stable source code in my system.
> The problem is that I can't work with CVS.
> It is too slow to download. My internet can download things at 500kb/s - 1MB/s,
> but when I am doing a checkout, the download stay too slow, I already tried with a
> bunch of mirrors, and it is almost the same thing. I don't know if the problem is that
> all CVS mirrors are slow.
> I let my computer all the last day doing a checkout for src ports and xenocara. After
> 10 hours it was in ports yet.
>
> Also, the worst problem is that my internet connections sometimes drop, and so in an
> attempt to continue the interrupted checkout I run the command again, the problem is
> that it takes about a half hour to start to continue getting the code, this when it
> works, because sometimes after all this time I get:
> Write failed: Broken pipe
> Also sometimes looks like it is getting everything again, because it says
> cvs server: Updating foo
> to every file already in the folder.
>
> I'm looking to a way to get this sources, there is a long time I'm trying and it
> ever happens some problem like this, and I can't continue to get the sources, so I
> need to delete everything a start again.
>
> CVS looks too complicated and confuse.
> I would like to be able to get from http, git, ftp or anything sipler and faster.
> Is there any alternative?

If you don't do it already, you may want to use compression on CVS checkout.
I.e.

cvs -z9 co src

Sebastian

> --
> Regards
>
> Henrique Lengler

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Henrique Lengler-2
In reply to this post by Alexander Hall
On Sat, Feb 21, 2015 at 07:23:16PM +0100, Alexander Hall wrote:
> I believe the source tarballs avaliable on the CDs (and mirrors) come with CVS directories, so they might be a good start. The FAQ might shed some light on this.
>
> /Alexander

This looks like the best way, so I will do this.

Thanks for helping.

--
Regards

Henrique Lengler

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Edgar Pettijohn III-2
On 02/21/15 14:38, Henrique Lengler wrote:
> On Sat, Feb 21, 2015 at 07:23:16PM +0100, Alexander Hall wrote:
>> I believe the source tarballs avaliable on the CDs (and mirrors) come with CVS directories, so they might be a good start. The FAQ might shed some light on this.
>>
>> /Alexander
> This looks like the best way, so I will do this.
>
> Thanks for helping.
>
http://www.openbsd.org/anoncvs.html

Speaking for myself updating is usually slow also, but not as slow as
"getting" an initial tree.  However, if -stable is just -release with
patches.  It should be real fast to just download the source tarballs
and apply the patches.  Then voila you have -stable.  Then again my
understanding of -stable may be off.

Edgar

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

James Hartley
In reply to this post by Henrique Lengler-2
On Saturday, February 21, 2015, Henrique Lengler <[hidden email]>
wrote:
>
> I'm trying to keep and maintain a copy of -stable source code in my system.
> The problem is that I can't work with CVS.
> It is too slow to download.


Another alternative is to set up a server which has gone through the
laborious initial download, but periodically does a CVS update set up via
cron.  Updates are relatively quick.

Access the box either via NFS, copy the tree(s) via scp, or transfer via
tar files.  It's your choice.

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Henrique Lengler-2
In reply to this post by Alexander Hall
On Sat, Feb 21, 2015 at 07:23:16PM +0100, Alexander Hall wrote:
> I believe the source tarballs avaliable on the CDs (and mirrors) come with CVS directories, so they might be a good start. The FAQ might shed some light on this.
>
> /Alexander

Just another question, once I get the tarballs and unpack them, to update the code, I should run cvs checkout or updtate?

--
Regards

Henrique Lengler

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

James Hartley
On Saturday, February 21, 2015, Henrique Lengler <[hidden email]>
wrote:

> once I get the tarballs and unpack them, to update the code, I should run
> cvs checkout or updtate?
>

Update.  Reading the cvs manpage will help other questions as well.

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Henrique Lengler-2
On Sat, Feb 21, 2015 at 03:51:37PM -0600, James Hartley wrote:
> On Saturday, February 21, 2015, Henrique Lengler <[hidden email]>
> wrote:
>
> > once I get the tarballs and unpack them, to update the code, I should run
> > cvs checkout or updtate?
> >
>
> Update.  Reading the cvs manpage will help other questions as well.

So I ran this command:
cvs -d [hidden email]:/cvs -q up -rOPENBSD_5_6 -Pd

And it is stoped, for more than a hour.

This is what top show about the process:

  PID USERNAME PRI NICE  SIZE   RES STATE     WAIT      TIME    CPU COMMAND
19822 root      -5    0 2416K 3708K idle      pipewr    0:01  0.00% cvs

--
Regards

Henrique Lengler

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

James Hartley
On Sat, Feb 21, 2015 at 6:14 PM, Henrique Lengler <[hidden email]>
wrote:

> So I ran this command:
> cvs -d [hidden email]:/cvs -q up -rOPENBSD_5_6 -Pd
>
> And it is stoped, for more than a hour.
>

Downloading from the main server is a poor choice as this is the motivation
for the entire mirror system.   Choose a server which is close(st) to you
geographically  from the list found at the following:

http://www.openbsd.org/anoncvs.html

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Stuart Henderson
In reply to this post by Raf Czlonka-2
On 2015-02-21, Raf Czlonka <[hidden email]> wrote:
> Alternatively, you can use CVSync[0] to get the code.

That is likely going to be worse on a slow/unstable link as you
are fetching the entire repository history (~2GB rather than ~750MB),
not just updating the checked-out tree.

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Stuart Henderson
In reply to this post by Edgar Pettijohn III-2
On 2015-02-21, Edgar Pettijohn <[hidden email]> wrote:

> On 02/21/15 14:38, Henrique Lengler wrote:
>> On Sat, Feb 21, 2015 at 07:23:16PM +0100, Alexander Hall wrote:
>>> I believe the source tarballs avaliable on the CDs (and mirrors) come with CVS directories, so they might be a good start. The FAQ might shed some light on this.
>>>
>>> /Alexander
>> This looks like the best way, so I will do this.
>>
>> Thanks for helping.
>>
> http://www.openbsd.org/anoncvs.html
>
> Speaking for myself updating is usually slow also, but not as slow as
> "getting" an initial tree.  However, if -stable is just -release with
> patches.  It should be real fast to just download the source tarballs
> and apply the patches.  Then voila you have -stable.  Then again my
> understanding of -stable may be off.

-stable is a bit different, sometimes things are applied there which
aren't considered serious enough to warrant issuing an errata.

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Raf Czlonka-2
In reply to this post by Stuart Henderson
On Sun, Feb 22, 2015 at 12:43:57PM GMT, Stuart Henderson wrote:
> On 2015-02-21, Raf Czlonka <[hidden email]> wrote:
> > Alternatively, you can use CVSync[0] to get the code.
>
> That is likely going to be worse on a slow/unstable link as you
> are fetching the entire repository history (~2GB rather than ~750MB),
> not just updating the checked-out tree.

One would think that. However, CVSync + local 'cvs up' is much faster
and more reliable than "remote" 'cvs up'.

I'm using it daily so this is not speculative - I'm writing from my own
experience. Also, I had very similar issues as the OP.

Cheers,

Raf

P.S. I'm use CVSync with your mirror :^)

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Stuart Henderson
On 2015-02-22, Raf Czlonka <[hidden email]> wrote:

> On Sun, Feb 22, 2015 at 12:43:57PM GMT, Stuart Henderson wrote:
>> On 2015-02-21, Raf Czlonka <[hidden email]> wrote:
>> > Alternatively, you can use CVSync[0] to get the code.
>>
>> That is likely going to be worse on a slow/unstable link as you
>> are fetching the entire repository history (~2GB rather than ~750MB),
>> not just updating the checked-out tree.
>
> One would think that. However, CVSync + local 'cvs up' is much faster
> and more reliable than "remote" 'cvs up'.
>
> I'm using it daily so this is not speculative - I'm writing from my own
> experience. Also, I had very similar issues as the OP.
>
> Cheers,
>
> Raf
>
> P.S. I'm use CVSync with your mirror :^)
>
>

Interesting.. I stopped running local cvsync to my server at home
a while ago, as 'cvs up' from my mirror (over adsl) was faster, not
even taking the extra cvsync time into account ;)

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Henrique Lengler-2
On Sun, Feb 22, 2015 at 05:18:24PM +0000, Stuart Henderson wrote:
> Interesting.. I stopped running local cvsync to my server at home
> a while ago, as 'cvs up' from my mirror (over adsl) was faster, not
> even taking the extra cvsync time into account ;)

So I successfully fetched the code, now I will compile, and see if I get any error,
as I get before. Because last time I checkout the code, things like xenocara failed to build.

I post here when I end.
--
Regards

Henrique Lengler

Reply | Threaded
Open this post in threaded view
|

Re: Alternative way to get sources, CVS slow

Henrique Lengler-2
On Sun, Feb 22, 2015 at 04:40:39PM -0300, Henrique Lengler wrote:
> On Sun, Feb 22, 2015 at 05:18:24PM +0000, Stuart Henderson wrote:
> > Interesting.. I stopped running local cvsync to my server at home
> > a while ago, as 'cvs up' from my mirror (over adsl) was faster, not
> > even taking the extra cvsync time into account ;)
>
> So I successfully fetched the code, now I will compile, and see if I get any error,
> as I get before. Because last time I checkout the code, things like xenocara failed to build.
>
Holy #@$$!!

I got a compiler error, the same I posted here:
http://marc.info/?l=openbsd-misc&m=142444902627315&w=2

I really need help, I'm following he FAQ, and I don't know why it don't work!
--
Regards

Henrique Lengler

12