mirror download speed variation

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

mirror download speed variation

Mihai Popescu-3
Hi,

I use to retrieve my install sets from a mirror, after I start the
install procedure with minirootxx.fs

Since the mirrors in my country are updating late and they have some
problems in doing it right, I used ftp.hostserver.de. The download was
working fine, something around 3MBps. This mirror started few days ago
to provide me only with 64KBps constantly, no matter if i do http or
https.

I tried cdn.openbsd.org too, the download is super fast for me, like
30MBps. Still, sometimes it drops to 64KBps too, and stays there. I've
read some articles about CDN networks, but I am not able to see the
big picture.

So, I still have two questions about mirrors:
Can a mirror limit your download speed ?
Do a CDN url point to an existing mirror, or is it a diffeent server?

Thank you.

Reply | Threaded
Open this post in threaded view
|

Re: mirror download speed variation

Janne Johansson-3
Den tis 8 jan. 2019 kl 14:26 skrev Mihai Popescu <[hidden email]>:

> So, I still have two questions about mirrors:
> Can a mirror limit your download speed ?

Sure they could, I don't think many do though.

> Do a CDN url point to an existing mirror, or is it a diffeent server?

Different servers, spread around the world and you get a dns response
that is trying to be
close to you.

--
May the most significant bit of your life be positive.

Reply | Threaded
Open this post in threaded view
|

Odp.: mirror download speed variation

Kamil Monticolo-4
There is small program that helps you determine the closest mirror:

https://github.com/lukensmall/pkg_ping


I also wrote poor's man script to achieve the same:

https://github.com/kmonticolo/OpenBSD/blob/master/testmirrors.sh



Kamil

________________________________
Od: [hidden email] <[hidden email]> w imieniu użytkownika Janne Johansson <[hidden email]>
Wysłane: wtorek, 8 stycznia 2019 15:08:44
Do: Mihai Popescu
DW: Misc
Temat: Re: mirror download speed variation

Den tis 8 jan. 2019 kl 14:26 skrev Mihai Popescu <[hidden email]>:

> So, I still have two questions about mirrors:
> Can a mirror limit your download speed ?

Sure they could, I don't think many do though.

> Do a CDN url point to an existing mirror, or is it a diffeent server?

Different servers, spread around the world and you get a dns response
that is trying to be
close to you.

--
May the most significant bit of your life be positive.

Reply | Threaded
Open this post in threaded view
|

Re: mirror download speed variation

Peter Hessler
In reply to this post by Mihai Popescu-3
I'm one of the admins of ftp.hostserver.de, can you (privately) email me
your source IP and a traceroute to us?


On 2019 Jan 08 (Tue) at 15:24:38 +0200 (+0200), Mihai Popescu wrote:
:Hi,
:
:I use to retrieve my install sets from a mirror, after I start the
:install procedure with minirootxx.fs
:
:Since the mirrors in my country are updating late and they have some
:problems in doing it right, I used ftp.hostserver.de. The download was
:working fine, something around 3MBps. This mirror started few days ago
:to provide me only with 64KBps constantly, no matter if i do http or
:https.
:
:I tried cdn.openbsd.org too, the download is super fast for me, like
:30MBps. Still, sometimes it drops to 64KBps too, and stays there. I've
:read some articles about CDN networks, but I am not able to see the
:big picture.
:
:So, I still have two questions about mirrors:
:Can a mirror limit your download speed ?
:Do a CDN url point to an existing mirror, or is it a diffeent server?
:
:Thank you.
:

--
Eggheads unite!  You have nothing to lose but your yolks.
                -- Adlai Stevenson

Reply | Threaded
Open this post in threaded view
|

Re: mirror download speed variation

Stuart Henderson
In reply to this post by Janne Johansson-3
On 2019-01-08, Janne Johansson <[hidden email]> wrote:

> Den tis 8 jan. 2019 kl 14:26 skrev Mihai Popescu <[hidden email]>:
>
>> So, I still have two questions about mirrors:
>> Can a mirror limit your download speed ?
>
> Sure they could, I don't think many do though.
>
>> Do a CDN url point to an existing mirror, or is it a diffeent server?
>
> Different servers, spread around the world and you get a dns response
> that is trying to be
> close to you.
>

However: the CDNs are just front-ending existing servers ("origin
servers"). If they have something in cache they'll be fast; if not
they have to fetch it themselves, often from the other side of the
world (further/slower than you'd get by picking a local mirror
that actually has the files).

snapshots in particular are often not going to be in their cache.


Reply | Threaded
Open this post in threaded view
|

Re: mirror download speed variation

Stefan Wollny-2
In reply to this post by Mihai Popescu-3
On 08.01.19 14:24, Mihai Popescu wrote:

> Hi,
>
> I use to retrieve my install sets from a mirror, after I start the
> install procedure with minirootxx.fs
>
> Since the mirrors in my country are updating late and they have some
> problems in doing it right, I used ftp.hostserver.de. The download was
> working fine, something around 3MBps. This mirror started few days ago
> to provide me only with 64KBps constantly, no matter if i do http or
> https.
>
> I tried cdn.openbsd.org too, the download is super fast for me, like
> 30MBps. Still, sometimes it drops to 64KBps too, and stays there. I've
> read some articles about CDN networks, but I am not able to see the
> big picture.
>
> So, I still have two questions about mirrors:
> Can a mirror limit your download speed ?
> Do a CDN url point to an existing mirror, or is it a diffeent server?
>
> Thank you.

I can confirm Mihai's observation: http://ftp.hostserver.de is
remarkable slow for 2~3 weeks, even from within Frankfurt. From memory
I'd say from around the week before xmas download rates dropped to
20kb/s what usually was ~2mb/s. The Austrian mirror gave me this morning
~4mb/s! I fetch like so:

export SERVER=http://ftp2.eu.openbsd.org/pub/OpenBSD/snapshots
export ARCH=amd64
export VERSION=64wget -4 --no-proxy --no-cache --no-cookies --backups=2 \
    $SERVER/$ARCH/BOOT{IA32,X64}.EFI \
    $SERVER/$ARCH/INSTALL.$ARCH \
    $SERVER/$ARCH/BUILDINFO \
    $SERVER/$ARCH/index.txt \
    $SERVER/$ARCH/SHA256{,.sig} \
    $SERVER/$ARCH/bsd{,.mp,.rd} \
    $SERVER/$ARCH/{base,comp,man}$VERSION.tgz \
    $SERVER/$ARCH/x{base,font,serv,share}$VERSION.tgz

Reply | Threaded
Open this post in threaded view
|

Re: mirror download speed variation

Stuart Henderson
On 2019-01-09, Stefan Wollny <[hidden email]> wrote:

> On 08.01.19 14:24, Mihai Popescu wrote:
>>
>> So, I still have two questions about mirrors:
>> Can a mirror limit your download speed ?
>> Do a CDN url point to an existing mirror, or is it a diffeent server?
>>
>> Thank you.
>
> I can confirm Mihai's observation: http://ftp.hostserver.de is
> remarkable slow for 2~3 weeks, even from within Frankfurt. From memory
> I'd say from around the week before xmas download rates dropped to
> 20kb/s what usually was ~2mb/s.

I'm currently getting 6.7MByte/s over v6 and 3.95MByte/s over v4
from ftp.hostserver.de to the UK. So it's not a global problem.
As phessler said, send him traceroute and your source IP offlist.



Reply | Threaded
Open this post in threaded view
|

Re: Odp.: mirror download speed variation

Stuart Henderson
In reply to this post by Kamil Monticolo-4
On 2019-01-08, Kamil Monticolo <[hidden email]> wrote:
> There is small program that helps you determine the closest mirror:
>
> https://github.com/lukensmall/pkg_ping

oh, this again.

> I also wrote poor's man script to achieve the same:
>
> https://github.com/kmonticolo/OpenBSD/blob/master/testmirrors.sh
>

ping time to a mirror does not indicate what download speed you
can expect from it.


Reply | Threaded
Open this post in threaded view
|

Re: mirror download speed variation => Back to normal

Stefan Wollny-2
In reply to this post by Stuart Henderson
Am 09.01.19 um 17:53 schrieb Stuart Henderson:
> On 2019-01-09, Stefan Wollny <[hidden email]> wrote:
>> On 08.01.19 14:24, Mihai Popescu wrote:
>>>
...

>
> I'm currently getting 6.7MByte/s over v6 and 3.95MByte/s over v4
> from ftp.hostserver.de to the UK. So it's not a global problem.
> As phessler said, send him traceroute and your source IP offlist.
>
>
Apparently they (phessler@/benno@) have taken care of the issue. This
morning download rates are back to what I usually had in the past.

THANK YOU very much! Well apprecciated.

Best,
STEFAN