Long delay updating xenocara source tree?

classic Classic list List threaded Threaded
37 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Long delay updating xenocara source tree?

Dave Anderson-4
I've run into this problem perhaps a dozen times over the past several
months while running amd64-current, most recently at 15:53 2012/1/26 EST
while running a system built from source updated at about 14:30
2012/1/21 EST: when trying to update the xenocara source tree there is a
very long (perhaps infinite) delay between issuing the 'cvs ...' command
and the start of any visible activity.  In this most recent case the
delay was about 9 hours.  Updating the src and ports source trees at
about the same time and using the same CVSROOT has always worked OK;
there's some delay but not a really long one.  And sometimes the
xenocara update has worked without any problem.  When it doesn't, 'rm
-rf /usr/xenocara' followed by reloading from the 5.0-release CD has
always allowed a subsequent cvs update to work.

The command I'm using is
  # cd /usr/xenocara && cvs -d$CVSROOT -q up -Pd
(except for the working directory, exactly the same as the command for
updating the src or ports tree).

This has happened with several different values for CVSROOT, currently
  # echo $CVSROOT
  [hidden email]:/cvs

I am very confused by this.  Any clues would be greatly appreciated.

        Dave

--
Dave Anderson
<[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Tomas Bodzar-4
On Fri, Jan 27, 2012 at 9:10 PM, Dave Anderson <[hidden email]> wrote:

> I've run into this problem perhaps a dozen times over the past several
> months while running amd64-current, most recently at 15:53 2012/1/26 EST
> while running a system built from source updated at about 14:30
> 2012/1/21 EST: when trying to update the xenocara source tree there is a
> very long (perhaps infinite) delay between issuing the 'cvs ...' command
> and the start of any visible activity. B In this most recent case the
> delay was about 9 hours. B Updating the src and ports source trees at
> about the same time and using the same CVSROOT has always worked OK;
> there's some delay but not a really long one. B And sometimes the
> xenocara update has worked without any problem. B When it doesn't, 'rm
> -rf /usr/xenocara' followed by reloading from the 5.0-release CD has
> always allowed a subsequent cvs update to work.
>
> The command I'm using is
> B # cd /usr/xenocara && cvs -d$CVSROOT -q up -Pd
> (except for the working directory, exactly the same as the command for
> updating the src or ports tree).

How about:

# cd /usr ; cvs -d$CVSROOT -q up -Pd xenocara

>
> This has happened with several different values for CVSROOT, currently
> B # echo $CVSROOT
> B [hidden email]:/cvs
>
> I am very confused by this. B Any clues would be greatly appreciated.
>
> B  B  B  B Dave
>
> --
> Dave Anderson
> <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Philip Guenther-2
In reply to this post by Dave Anderson-4
On Fri, Jan 27, 2012 at 12:10 PM, Dave Anderson <[hidden email]>
wrote:

> I've run into this problem perhaps a dozen times over the past several
> months while running amd64-current, most recently at 15:53 2012/1/26 EST
> while running a system built from source updated at about 14:30
> 2012/1/21 EST: when trying to update the xenocara source tree there is a
> very long (perhaps infinite) delay between issuing the 'cvs ...' command
> and the start of any visible activity.  In this most recent case the
> delay was about 9 hours.  Updating the src and ports source trees at
> about the same time and using the same CVSROOT has always worked OK;
> there's some delay but not a really long one.  And sometimes the
> xenocara update has worked without any problem.  When it doesn't, 'rm
> -rf /usr/xenocara' followed by reloading from the 5.0-release CD has
> always allowed a subsequent cvs update to work.
>
> The command I'm using is
>  # cd /usr/xenocara && cvs -d$CVSROOT -q up -Pd
> (except for the working directory, exactly the same as the command for
> updating the src or ports tree).

I bet it'll be faster if you don't use the -P or -d options.

The -d option to "cvs up" requires the cvs server to walk directories
that are present on the server but not present on the client.  That
includes directories which are now empty because all their files have
been removed (ala "cvs rm")...of which there are a bunch in the
xenocara tree.

The -P option requires extra work too, though it's not as bad as the
-d option, IIRC.

Personally, I use the rule of thumb of only using -d and -P when I
have reason to believe directories have been added or removed, either
from seeing the commit email or from a build failing because a
directory is missing...


Philip Guenther

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Stuart Henderson
On 2012-01-28, Philip Guenther <[hidden email]> wrote:

>> I've run into this problem perhaps a dozen times over the past several
>> months while running amd64-current, most recently at 15:53 2012/1/26 EST
>> while running a system built from source updated at about 14:30
>> 2012/1/21 EST: when trying to update the xenocara source tree there is a
>> very long (perhaps infinite) delay between issuing the 'cvs ...' command
>> and the start of any visible activity.  In this most recent case the
>> delay was about 9 hours.  Updating the src and ports source trees at
>> about the same time and using the same CVSROOT has always worked OK;
>> there's some delay but not a really long one.  And sometimes the
>> xenocara update has worked without any problem.  When it doesn't, 'rm
>> -rf /usr/xenocara' followed by reloading from the 5.0-release CD has
>> always allowed a subsequent cvs update to work.
>>
>> The command I'm using is
>>  # cd /usr/xenocara && cvs -d$CVSROOT -q up -Pd
>> (except for the working directory, exactly the same as the command for
>> updating the src or ports tree).
>
> I bet it'll be faster if you don't use the -P or -d options.

If people try this, please don't report any build problems unless
you've re-run with both of these options.

Also note it's not going to work well for ports..

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Dave Anderson-4
In reply to this post by Philip Guenther-2
On Fri, 27 Jan 2012, Philip Guenther wrote:

>On Fri, Jan 27, 2012 at 12:10 PM, Dave Anderson <[hidden email]>
>wrote:
>> I've run into this problem perhaps a dozen times over the past several
>> months while running amd64-current, most recently at 15:53 2012/1/26 EST
>> while running a system built from source updated at about 14:30
>> 2012/1/21 EST: when trying to update the xenocara source tree there is a
>> very long (perhaps infinite) delay between issuing the 'cvs ...' command
>> and the start of any visible activity.  In this most recent case the
>> delay was about 9 hours.  Updating the src and ports source trees at
>> about the same time and using the same CVSROOT has always worked OK;
>> there's some delay but not a really long one.  And sometimes the
>> xenocara update has worked without any problem.  When it doesn't, 'rm
>> -rf /usr/xenocara' followed by reloading from the 5.0-release CD has
>> always allowed a subsequent cvs update to work.
>>
>> The command I'm using is
>>  # cd /usr/xenocara && cvs -d$CVSROOT -q up -Pd
>> (except for the working directory, exactly the same as the command for
>> updating the src or ports tree).
>
>I bet it'll be faster if you don't use the -P or -d options.
>
>The -d option to "cvs up" requires the cvs server to walk directories
>that are present on the server but not present on the client.  That
>includes directories which are now empty because all their files have
>been removed (ala "cvs rm")...of which there are a bunch in the
>xenocara tree.
>
>The -P option requires extra work too, though it's not as bad as the
>-d option, IIRC.
>
>Personally, I use the rule of thumb of only using -d and -P when I
>have reason to believe directories have been added or removed, either
>from seeing the commit email or from a build failing because a
>directory is missing...

Thanks for the info.  I've been using -Pd because
<http://www.openbsd.org/anoncvs.html> says to use them; I haven't yet
had a chance to look into how cvs works beyond reading the man page,
faq, etc.

        Dave

--
Dave Anderson
<[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Steffen Daode Nurpmeso-2
Hey,

Dave Anderson wrote [2012-01-28 15:13+0100]:
[.]
> I haven't yet had a chance to look into how cvs works beyond
> reading the man page, faq, etc.

Also true for me.

> >> I've run into this problem perhaps a dozen times over the past several
> >> months
[.]

I've noted a lot of upload network traffic when doing 'cvs up'
on OpenBSD repos; i.e., before anything else happened about
~70 MB (www) and ~150 MB (src) *upstream* traffic were produced,
and it took more than an hour before the download of data began
(src).  I never had similar problems with anoncvs.mindrot.org, nor
cvs.savannah.gnu.org and *.cvs.sourceforge.net etc.

If you're doing your updates in such a regular manner, i think
your best bet is cvsync(1), even if that means additional local
storage - but it is *far* more efficient in both, time and
traffic.  (Not to talk about the possibility to do 'cvs log' and
the like locally, without internet connection; if that's an issue.)
Pears similar ciao,

--steffen

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Nick Holland
In reply to this post by Dave Anderson-4
On 01/28/12 09:12, Dave Anderson wrote:
> Thanks for the info.  I've been using -Pd because
> <http://www.openbsd.org/anoncvs.html> says to use them; I haven't yet
> had a chance to look into how cvs works beyond reading the man page,
> faq, etc.

and please continue to use them.
-Pd is the RIGHT way.

Apparently, Philip gets away with it, but he's a developer and he knows
this stuff pretty well, we don't expect ordinary users to clean up the
mess it can make.  I'll defer to his expertise on coding and probably
CVS, but there are many things in many parts of the tree where a lack of
a -Pd will hurt you in ways other than slow updates.  There are
thousands of ways to use cvs incorrectly, -Pd is the correct way to do
updates (or maybe -PAd under some circumstances).


And none of this has anything to do with your real problem.  I run far
slower hardware than most people, and xenocara updates don't take nine
hours (and if I understand you, that was nine hours then you gave up and
killed it).  This has NOTHING TO DO WITH COMMAND LINE OPTIONS.  I wrote
the FAQ you used, I use that FAQ, and I do it on hardware like mac68k
and sparc, and it works, it does not take nine hours to update xenocara
(it just feels like it...)

If you could...next time you see this, use a
  CVSROOT=[hidden email]:/cvs
and see if things run better.  NOTE: DO NOT GET USED TO USING THIS
MIRROR, IT IS BEING SHUT DOWN VERY SOON.  But, being it's been being
advertised as being shut down, it's pretty lightly loaded, and it
handles the CVS temp directory as an mfs, which really really helps
(this is on the server end. Nothing you can do about it on your side).
My hunch, as a soon-to-be former mirror operator is that you are having
a problem with your mirror of choice, not a problem on your end, and it
may be a problem with multiple mirrors.

I just checked out xenocara from that mirror, and then did an update on
my amd64 system, the update took less than one minute.  Your results
will vary, but not to nine hours, unless you are using dialup. :)

Nick.

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Henning Brauer
In reply to this post by Steffen Daode Nurpmeso-2
* Steffen Daode Nurpmeso <[hidden email]> [2012-01-28 15:44]:
> I've noted a lot of upload network traffic when doing 'cvs up'
> on OpenBSD repos; i.e., before anything else happened about
> ~70 MB (www) and ~150 MB (src) *upstream* traffic were produced,
> and it took more than an hour before the download of data began
> (src).  I never had similar problems with anoncvs.mindrot.org, nor
> cvs.savannah.gnu.org and *.cvs.sourceforge.net etc.

there aren't all that many repositories the size of ours out there.

> If you're doing your updates in such a regular manner, i think
> your best bet is cvsync(1), even if that means additional local
> storage - but it is *far* more efficient in both, time and
> traffic.  (Not to talk about the possibility to do 'cvs log' and
> the like locally, without internet connection; if that's an issue.)
> Pears similar ciao,

that's what many of us do - full repository, synced somehow (i
personally rsync from another machine which in turn speaks cvsync to
bob) locally. Especially convenient when you sit in an airplane or the
like and want to diff...

--
Henning Brauer, [hidden email], [hidden email]
BS Web Services, http://bsws.de, Full-Service ISP
Secure Hosting, Mail and DNS Services. Dedicated Servers, Root to Fully Managed
Henning Brauer Consulting, http://henningbrauer.com/

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Dave Anderson-4
In reply to this post by Nick Holland
On Sat, 28 Jan 2012, Nick Holland wrote:

>On 01/28/12 09:12, Dave Anderson wrote:
>> Thanks for the info.  I've been using -Pd because
>> <http://www.openbsd.org/anoncvs.html> says to use them; I haven't yet
>> had a chance to look into how cvs works beyond reading the man page,
>> faq, etc.
>
>and please continue to use them.
>-Pd is the RIGHT way.

I plan to.  Dropping them felt kind of iffy; thanks for the confirmation
that it isn't the way to go.

>Apparently, Philip gets away with it, but he's a developer and he knows
>this stuff pretty well, we don't expect ordinary users to clean up the
>mess it can make.  I'll defer to his expertise on coding and probably
>CVS, but there are many things in many parts of the tree where a lack of
>a -Pd will hurt you in ways other than slow updates.  There are
>thousands of ways to use cvs incorrectly, -Pd is the correct way to do
>updates (or maybe -PAd under some circumstances).
>
>
>And none of this has anything to do with your real problem.  I run far
>slower hardware than most people, and xenocara updates don't take nine
>hours (and if I understand you, that was nine hours then you gave up and
>killed it).  This has NOTHING TO DO WITH COMMAND LINE OPTIONS.  I wrote
>the FAQ you used, I use that FAQ, and I do it on hardware like mac68k
>and sparc, and it works, it does not take nine hours to update xenocara
>(it just feels like it...)

No, it was about 9 hours from issuing the cvs update command until there
was any visible action; the update ran to completion in a total of about
11 hours.  I've killed some other update attempts which ran even longer
without any visible action.

>If you could...next time you see this, use a
>  CVSROOT=[hidden email]:/cvs
>and see if things run better.  NOTE: DO NOT GET USED TO USING THIS
>MIRROR, IT IS BEING SHUT DOWN VERY SOON.  But, being it's been being
>advertised as being shut down, it's pretty lightly loaded, and it
>handles the CVS temp directory as an mfs, which really really helps
>(this is on the server end. Nothing you can do about it on your side).
>My hunch, as a soon-to-be former mirror operator is that you are having
>a problem with your mirror of choice, not a problem on your end, and it
>may be a problem with multiple mirrors.

I've tried 3 or 4 different servers, and have had this problem with all
of them (at least some of the time).

>I just checked out xenocara from that mirror, and then did an update on
>my amd64 system, the update took less than one minute.  Your results
>will vary, but not to nine hours, unless you are using dialup. :)

I do have a slowish ADSL link (384Kbps/1536Kbps) which would limit me to
very roughly 1MB/min outbound, so I took advice to use '-z 9' to
compress data and that reduced the total time for a xenocara source tree
update from about 11 hours to about 2.5 hours.  (Though I discovered
that not all servers support compression.)

Then I did a test update of xenocara against your server (still using -z
9), and the entire process took barely 1 minute.  I then retried that
upgrade against the server I've been using (anoncvs.comstyle.com), and
the total time was just under 3 minutes.  As a final (for the moment)
test I did (against my usual server and with -z 9) an update of my
entire source tree and the total times were src: 7:37, xenocara: 3:55,
ports: 41:58, and www: 2:39 -- for a total of about 55 minutes.

I've no idea why I'm suddenly getting so much faster responses.

Does cvs update send a potentially large but extremely variable amount
of data from my system to the server?  If so, that (plus my slowish
uplink) might explain some of these timings.  But the cause of these
massive variations is not at all obvious from where I sit.

Thanks for any further info.

        Dave

--
Dave Anderson
<[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Janne Johansson-3
2012/1/31 Dave Anderson <[hidden email]>:
>
> I do have a slowish ADSL link (384Kbps/1536Kbps) which would limit me to
> very roughly 1MB/min outbound, so I took advice to use '-z 9' to
> compress data and that reduced the total time for a xenocara source tree
> update from about 11 hours to about 2.5 hours.  (Though I discovered
> that not all servers support compression.)

if they do anoncvs over ssh, you can ask ssh to compress the data
instead of having cvs do it, for those servers.

--
 To our sweethearts and wives.  May they never meet. -- 19th century toast

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Amit Kulkarni-5
In reply to this post by Steffen Daode Nurpmeso-2
On Sat, Jan 28, 2012 at 8:43 AM, Steffen Daode Nurpmeso
<[hidden email]> wrote:

> Hey,
>
> Dave Anderson wrote [2012-01-28 15:13+0100]:
> [.]
>> I haven't yet had a chance to look into how cvs works beyond
>> reading the man page, faq, etc.
>
> Also true for me.
>
>> >> I've run into this problem perhaps a dozen times over the past several
>> >> months
> [.]
>
> I've noted a lot of upload network traffic when doing 'cvs up'
> on OpenBSD repos; i.e., before anything else happened about
> ~70 MB (www) and ~150 MB (src) *upstream* traffic were produced,
> and it took more than an hour before the download of data began
> (src).  I never had similar problems with anoncvs.mindrot.org, nor
> cvs.savannah.gnu.org and *.cvs.sourceforge.net etc.
>
> If you're doing your updates in such a regular manner, i think
> your best bet is cvsync(1), even if that means additional local
> storage - but it is *far* more efficient in both, time and
> traffic.  (Not to talk about the possibility to do 'cvs log' and
> the like locally, without internet connection; if that's an issue.)
> Pears similar ciao,
>
> --steffen
>

Steffen,

i also used to suffer from this unexplained stuff, and yes i used
comstyle/spacehopper

thanks... this motivated me to use cvsync from anoncvs and use a
CVSROOT=/home/amit/MYLOCALREPO to update from cvs. Much much faster
and while initial checkout from cvsync takes 5-10 hrs, the subsequent
updates are lightning quick.

--amit

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Steffen Daode Nurpmeso-2
Hi Amit,

Amit Kulkarni wrote [2012-02-01 00:57+0100]:
> this motivated me to use cvsync from anoncvs and use a
> CVSROOT=/home/amit/MYLOCALREPO to update from cvs. Much much faster
> and while initial checkout from cvsync takes 5-10 hrs

Yes, that's a real pity.
I think it would be great if tarballs would be offered for
those, too.  I.e. this is *all* of OpenBSD (src, xenocara and
www, the latter however "refuse"s papers/, slides/ and songs/
"because" i also have a checkout of the Steelix translation
project), as has been burned onto my last full-backup DVD:

    $ ls -latr
    -rw-r--r--  1 steffen  wheel  516407350 26 Jan 15:41 cvsync_openbsd.tbz

I think plain gzip wouldn't be that much larger.
And hey, it's worth noting that this is smaller than
.git/objects/pack of src.git alone!

> the subsequent updates are lightning quick.

Yes.
And how easy it now is to do 'cvs up -PAC' to overwrite my messy
patches.

> --amit

Ciao,

--steffen

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Brett Mahar-2
In reply to this post by Amit Kulkarni-5
> > Dave Anderson wrote [2012-01-28 15:13+0100]:
> > [.]
> >> I haven't yet had a chance to look into how cvs works beyond
> >> reading the man page, faq, etc.
> >

> >
> > If you're doing your updates in such a regular manner, i think
> > your best bet is cvsync(1), even if that means additional local
> > storage - but it is *far* more efficient in both, time and
> > traffic.  (Not to talk about the possibility to do 'cvs log' and
> > the like locally, without internet connection; if that's an issue.)
> > Pears similar ciao,
> >
> > --steffen
> >

>
> thanks... this motivated me to use cvsync from anoncvs and use a
> CVSROOT=/home/amit/MYLOCALREPO to update from cvs. Much much faster
> and while initial checkout from cvsync takes 5-10 hrs, the subsequent
> updates are lightning quick.
>
> --amit
>

Yes, I also finally got around to trying cvsync because of this thread and its a lot quicker.

And even more speedy when I added a "scanfile /root/cvsync-scanfile" line in the "collection" part of the config file.

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Stuart Henderson
On 2012-02-02, Brett <[hidden email]> wrote:
> Yes, I also finally got around to trying cvsync because of this thread and its a lot quicker.
>
> And even more speedy when I added a "scanfile /root/cvsync-scanfile" line in the "collection" part of the config file.

You're running it as root?

It makes a lot more sense to use a separate user.

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Brett Mahar-2
On Thu, 2 Feb 2012 11:40:22 +0000 (UTC)
Stuart Henderson <[hidden email]> wrote:

> On 2012-02-02, Brett <[hidden email]> wrote:
> > Yes, I also finally got around to trying cvsync because of this thread and its a lot quicker.
> >
> > And even more speedy when I added a "scanfile /root/cvsync-scanfile" line in the "collection" part of the config file.
>
> You're running it as root?
>
> It makes a lot more sense to use a separate user.
>

I have been updating as root (the few times I have used it so far) but thanks for the reminder, will chown and switch to normal user next time.

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Stuart Henderson
On 2012/02/02 23:00, Brett wrote:

> On Thu, 2 Feb 2012 11:40:22 +0000 (UTC)
> Stuart Henderson <[hidden email]> wrote:
>
> > On 2012-02-02, Brett <[hidden email]> wrote:
> > > Yes, I also finally got around to trying cvsync because of this thread and its a lot quicker.
> > >
> > > And even more speedy when I added a "scanfile /root/cvsync-scanfile" line in the "collection" part of the config file.
> >
> > You're running it as root?
> >
> > It makes a lot more sense to use a separate user.
> >
>
> I have been updating as root (the few times I have used it so far) but thanks for the reminder, will chown and switch to normal user next time.

I use a dedicated user for this.

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Brett Mahar-2
> > > > And even more speedy when I added a "scanfile /root/cvsync-scanfile" line in the "collection" part of the config file.
> > >
> > > You're running it as root?
> > >
> > > It makes a lot more sense to use a separate user.
> > >
> >
> > I have been updating as root (the few times I have used it so far) but thanks for the reminder, will chown and switch to normal user next time.
>
> I use a dedicated user for this.
>

Sounds like a good idea.

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Steffen Daode Nurpmeso-2
In reply to this post by Henning Brauer
Henning Brauer wrote:
> there aren't all that many repositories the size of ours out there.

That's true.
But no Henning, i don't believe it's that;
you know, it's just that i don't have anything to say, because
i have no knowledge about the internals of cvs(1).

I always thought of this as some kind of misbehaviour in between
OpenCVS and GNU cvs, because i would think of cvs(1) as something
like this:

    cvs up .
    |
    read CVS/Entries
    |
    for those files with diff. timestamps, checksum file
    |
    send list [+ checksums] to server
    |
    server compare revision/timestamp/[checksum]
    - client unmodified: send diff (expected final checksum?)
    - client modified: send full file (if size < treshold),
      otherwise do blockwise checksumming etc. (i.e. rsync-like)
      [I don't really believe cvs(1) does the latter though.]
    |
    integrate diffs / replace locally modified files

Wether cvs(1) does do some rsync-like block-checksumming for
locally modified files or not, uploading 10% of the repositories
size or more before any data is sent from the server just can't be
correct anyhow.  Even more for my usage case because there were no
locally modified files at all.

And also the problem goes away if you do specify files directly,
as with a file glob, so it makes a difference wether you say
    $ cvs -fz9 up -PAC .
or
    $ cvs -fz9 up -PAC *.*
I don't remember wether i've used -d or not.

So for me this turned out as either "look into the code,
instrument some functions and try to fix it" or "turn over to
cvsync".
And GNU cvs is hard to look at, with a lot of comments which refer
to some (numeric or so) error reports.  But it would surely be
interesting to know what is going wrong.

--steffen

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Otto Moerbeek
On Thu, Feb 02, 2012 at 03:15:29PM +0100, Steffen Daode Nurpmeso wrote:

> Henning Brauer wrote:
> > there aren't all that many repositories the size of ours out there.
>
> That's true.
> But no Henning, i don't believe it's that;
> you know, it's just that i don't have anything to say, because
> i have no knowledge about the internals of cvs(1).
>
> I always thought of this as some kind of misbehaviour in between
> OpenCVS and GNU cvs, because i would think of cvs(1) as something
> like this:
>
>     cvs up .
>     |
>     read CVS/Entries
>     |
>     for those files with diff. timestamps, checksum file
>     |
>     send list [+ checksums] to server
>     |
>     server compare revision/timestamp/[checksum]
>     - client unmodified: send diff (expected final checksum?)
>     - client modified: send full file (if size < treshold),
>       otherwise do blockwise checksumming etc. (i.e. rsync-like)
>       [I don't really believe cvs(1) does the latter though.]
>     |
>     integrate diffs / replace locally modified files
>
> Wether cvs(1) does do some rsync-like block-checksumming for
> locally modified files or not, uploading 10% of the repositories
> size or more before any data is sent from the server just can't be
> correct anyhow.  Even more for my usage case because there were no
> locally modified files at all.
>
> And also the problem goes away if you do specify files directly,
> as with a file glob, so it makes a difference wether you say
>     $ cvs -fz9 up -PAC .
> or
>     $ cvs -fz9 up -PAC *.*
> I don't remember wether i've used -d or not.
>
> So for me this turned out as either "look into the code,
> instrument some functions and try to fix it" or "turn over to
> cvsync".
> And GNU cvs is hard to look at, with a lot of comments which refer
> to some (numeric or so) error reports.  But it would surely be
> interesting to know what is going wrong.
>
> --steffen

I like to say that long delays I have seen when using cvs had to do
with multiple different values of CVS/Root files in my local tree.

Those different entries can be created when doing a cvs up -d that
creates a new dir. If a cvs -d option is used at the same time, the
CVS/Root entry for tht dir wil be different than the other's.

The exact cause of the slowdown is not known to me. But when you are
switch repositories once in a while it's easy to get this case.

I repair this by find . -name Root | xargs rm and using a explicit cvs
root.

        -Otto

Reply | Threaded
Open this post in threaded view
|

Re: Long delay updating xenocara source tree?

Steffen Daode Nurpmeso-2
Otto Moerbeek wrote [2012-02-03 12:47+0100]:

> I like to say that long delays I have seen when using cvs had to do
> with multiple different values of CVS/Root files in my local tree.
>
> Those different entries can be created when doing a cvs up -d that
> creates a new dir. If a cvs -d option is used at the same time, the
> CVS/Root entry for tht dir wil be different than the other's.
>
> The exact cause of the slowdown is not known to me. But when you are
> switch repositories once in a while it's easy to get this case.
>
> I repair this by find . -name Root | xargs rm and using a explicit cvs
> root.

Now this is really another important issue of scattered
information, is it, and it's not noted in anoncvs.html!
I've slightly modified your command, i think my version is more
secure for use on a webpage.

> -Otto

--steffen

Index: anoncvs.html
===================================================================
RCS file: /cvs/www/anoncvs.html,v
retrieving revision 1.363
diff -a -p -u -r1.363 anoncvs.html
--- anoncvs.html 24 Jan 2012 09:57:35 -0000 1.363
+++ anoncvs.html 3 Feb 2012 13:33:28 -0000
@@ -542,6 +542,15 @@ add the <em>-d [hidden email]
  # <strong>cd /usr/src</strong>
  # <strong>cvs -d [hidden email]:/cvs -q up -Pd</strong>
 </pre>
+
+<p> And because cvs(1) stores the name of the server which is in use once
+a directory gets created in the file CVS/Root inside this new directory,
+it maybe wise to issue a command sequence like the following:
+<pre>
+ # <strong>cd /usr/src</strong>
+ # <strong>find . -path '*CVS/Root' | xargs rm</strong>
+ # <strong>cvs -d [hidden email]:/cvs -q up -Pd</strong>
+</pre>
 </ul>
 
 <p>

12