When will OpenBSD become a friendly place for bug reporters?

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

When will OpenBSD become a friendly place for bug reporters?

Leonid Bobrov
Hi!

We all know that bugs don't get fixed without backtraces.

After few years of using OpenBSD I am annoyed to get mocked for not
sending backtraces, but why I don't send them? The answer is: OpenBSD
doesn't provide software packages with debugging symbols.

Do I look like a Gentoo user? It's not cool to leave no choice to bug
reporters but to make them rebuild all ports they use with:
$ env CFLAGS='-pipe -g' DEBUG=-g make -j $(sysctl -n hw.ncpu) reinstall

The current OpenBSD is definetely not friendly to bug reporters, so
don't blame me when I refuse to send backtraces, I am simply not in mood
to rebuild software when it shouldn't be necessary, I value my time.

Reply | Threaded
Open this post in threaded view
|

Re: When will OpenBSD become a friendly place for bug reporters?

Theo de Raadt-2
[hidden email] wrote:

> Hi!
>
> We all know that bugs don't get fixed without backtraces.
>
> After few years of using OpenBSD I am annoyed to get mocked for not
> sending backtraces, but why I don't send them? The answer is: OpenBSD
> doesn't provide software packages with debugging symbols.
>
> Do I look like a Gentoo user? It's not cool to leave no choice to bug
> reporters but to make them rebuild all ports they use with:
> $ env CFLAGS='-pipe -g' DEBUG=-g make -j $(sysctl -n hw.ncpu) reinstall
>
> The current OpenBSD is definetely not friendly to bug reporters, so
> don't blame me when I refuse to send backtraces, I am simply not in mood
> to rebuild software when it shouldn't be necessary, I value my time.

It is definately not a friendly place for people with a tone like
yours.  Just read what is written above.  Like who do you think would
respond positively to what is written above?

Reply | Threaded
Open this post in threaded view
|

Re: When will OpenBSD become a friendly place for bug reporters?

Stuart Henderson
In reply to this post by Leonid Bobrov
moved to ports@

On 2019-07-09, STeve Andre' <[hidden email]> wrote:

>
>
> On 7/8/19 10:57 PM, [hidden email] wrote:
>> Hi!
>>
>> We all know that bugs don't get fixed without backtraces.
>>
>> After few years of using OpenBSD I am annoyed to get mocked for not
>> sending backtraces, but why I don't send them? The answer is: OpenBSD
>> doesn't provide software packages with debugging symbols.
>>
>> Do I look like a Gentoo user? It's not cool to leave no choice to bug
>> reporters but to make them rebuild all ports they use with:
>> $ env CFLAGS='-pipe -g' DEBUG=-g make -j $(sysctl -n hw.ncpu) reinstall
>>
>> The current OpenBSD is definetely not friendly to bug reporters, so
>> don't blame me when I refuse to send backtraces, I am simply not in mood
>> to rebuild software when it shouldn't be necessary, I value my time.
>
> For heavens sake, why don't you compile the code with symbols?  If you
> have the ability to go inside and look for problems, you can compile
> stuff yourself.  If you're going to submit a patch you have to build
> to test the fix!

Perhaps you have a core from a hard to trigger crash, and you aren't
able to get a matching binary by rebuilding things.

It is a valid request and has been made before by people with rather
more tact. But it's not an easy fix.

An all-arches package snapshot currently runs at 200GB and adding
symbols across the board would add a lot to this. Slower builds, more
disk space and bandwidth needed on build/signing infrastructure and
mirrors and, if they're in the main packages, also on user systems.

But done on a port-by-port opt-in basis for more important ports
(major libraries, for example) it might be viable. I think it would
need to use detached symbols in a subpackage - build with symbols,
then postprocess the various files, it looks like this might do the
trick

objcopy --only-keep-debug $file $file.debug
objcopy --strip-debug $file
objcopy --add-gnu-debuglink $file.debug $file

or

llvm-objcopy --only-keep=debug $file $file.debug
llvm-objcopy --strip-debug $file
llvm-objcopy --add-gnu-debuglink $file.debug $file

And then it needs additional infrastructure to handle putting these
into subpackages (which gets complicated where a port already uses
subpackages).

Unfortunately the tone of mazocomp's mail ather puts one off from
wanting to spend time on this... Maybe I need to polish my scorefile
as it only dropped the original mail not the replies :-)


Reply | Threaded
Open this post in threaded view
|

Re: When will OpenBSD become a friendly place for bug reporters?

Leonid Bobrov
In reply to this post by Leonid Bobrov
> It is definately not a friendly place for people with a tone like yours.

Theo, your excuse that OpenBSD is not more popular than Linux because AT&T
sued BSD in 90's is ridiculous, that's your own fault for being so
terrible in technical field, also you are terrible person, just like
me you can't communicate, so why do you think you can teach me
communication? If my tone is that important you prefer ignoring important
topic then you are even more terrible in terms of both personality and
technical field.

> An all-arches package snapshot currently runs at 200GB and adding
> symbols across the board would add a lot to this.

Stuart and Espie, have you ever heard of compression?

Again, what's wrong with my tone? I can't elaborate my thoughts in
different ways, also my tone only bothers people in this community,
most other communities don't see anything wrong about my way of speaking.

Anyway, Stuart suggests a really good solution: detaching symbols into
subpackages, this practice is already used in Void Linux, I downloaded
1 GB of compressed archives and they expanded to 27 GB, so maybe you
can learn how to use compression or at least switch packages to proper
compression method.

Reply | Threaded
Open this post in threaded view
|

Re: When will OpenBSD become a friendly place for bug reporters?

Raul Miller
On Tue, Jul 9, 2019 at 1:13 PM Leonid Bobrov <[hidden email]> wrote:
> Theo, your excuse that OpenBSD is not more popular than Linux because AT&T
> sued BSD in 90's is ridiculous,

Nah, it's a relevant issue.

That said, it's not the only issue, which I imagine was the point you
were trying to get across.

--
Raul

Reply | Threaded
Open this post in threaded view
|

Re: When will OpenBSD become a friendly place for bug reporters?

Theo de Raadt-2
In reply to this post by Leonid Bobrov
Please leave the list.



Leonid Bobrov <[hidden email]> wrote:

> > It is definately not a friendly place for people with a tone like yours.
>
> Theo, your excuse that OpenBSD is not more popular than Linux because AT&T
> sued BSD in 90's is ridiculous, that's your own fault for being so
> terrible in technical field, also you are terrible person, just like
> me you can't communicate, so why do you think you can teach me
> communication? If my tone is that important you prefer ignoring important
> topic then you are even more terrible in terms of both personality and
> technical field.
>
> > An all-arches package snapshot currently runs at 200GB and adding
> > symbols across the board would add a lot to this.
>
> Stuart and Espie, have you ever heard of compression?
>
> Again, what's wrong with my tone? I can't elaborate my thoughts in
> different ways, also my tone only bothers people in this community,
> most other communities don't see anything wrong about my way of speaking.
>
> Anyway, Stuart suggests a really good solution: detaching symbols into
> subpackages, this practice is already used in Void Linux, I downloaded
> 1 GB of compressed archives and they expanded to 27 GB, so maybe you
> can learn how to use compression or at least switch packages to proper
> compression method.

Reply | Threaded
Open this post in threaded view
|

Re: When will OpenBSD become a friendly place for bug reporters?

chohag
Perhaps rather than whining that OpenBSD lacks some specific feature, those who want it could write it? A novel idea, I know, but it IS specifically a development platform and there are precisely zero restrictions.

Or if you don't wish to start with code, at least try a tack such as "I intend to write feature $foo and would like advice for how to go about it". Notice that the act of _writing actual code_ is still implied.

I imagine that if a patch came through which adapted the build/release process such that symbols weren't removed but extracted into their own set for post-hoc installation by interested individuals, for example, that it would at least receive discussion if not eventual inclusion.

The bitching and public masturbation in this and the recent X thread, among many other examples, helps no-one.

Matthew

Reply | Threaded
Open this post in threaded view
|

Re: When will OpenBSD become a friendly place for bug reporters?

Marc Espie-2
In reply to this post by Leonid Bobrov
On Tue, Jul 09, 2019 at 08:04:23PM +0300, Leonid Bobrov wrote:
> > An all-arches package snapshot currently runs at 200GB and adding
> > symbols across the board would add a lot to this.
>
> Stuart and Espie, have you ever heard of compression?

WTF is wrong with you ?

I haven't participated to that thread yet, but I wager I know more about
compression than you do.

Just have a fucking look at the code in signify, the pkg tools, and more
generally, EVERYWHERE I have touched in BSD before spouting insults like
this.

Most specifically:
- we do use gzip because other compression systems won't work with little
memory/don't have the right licence.
- gzip allows you to stick COMMENTS in the header, which is where the
signature lives. I DID A WHOLE FUCKING PRESENTATION ABOUT THAT DESIGN
CHOICE.
- pkg_create uses an LRU cache to make updates faster.  And we use some
specificities of gzip  to make packages amenable to using rsync actually.
- Stuart has wasted hours getting mirrors to work as good as they can.
- *we* have spent hours trying to share stuff while keeping things secure.

It *still* doesn't change the fact that a full snapshot takes up a lot of
space.  And it's an important factor in having enough sites provide mirrors.

It's also an important factor in making sure snapshots are distributed
quickly.

Heck, there are design choices in package snapshots to avoid shearing.

So, to summarize. Get the hell away from this mailing-list.

I don't want to have anything to do with a condescending idiot who sends
disparaging comments my way 100% out of the blue.

Fuck you very much,

--
        Marc

Reply | Threaded
Open this post in threaded view
|

Re: When will OpenBSD become a friendly place for bug reporters?

Chris Humphries-2
Marc and Stuart (and whoever else was involved), thanks for the work
done on this stuff. It is simple and clean for me as a user and I have
trust in the integrity of what I'm receiving. I don't take it for
granted.

And there's probably a ton more involved that I'm not aware of yet,
but I trust in you guys to do the right thing (tm) all the time for
users. Trust is a big deal.


Sadly, the OP could have saved a lot of effort for everyone if they
just would have remembered there's other humans on the other side of
the email. And the fundamentals of communication include if you want
to have a real conversation with someone where they listen to you,
don't come at them guns blazing and emotional.

Thanks for explaining things (for the rest of us that are curious).

Hope you didn't let the emotional outrage of a random person on the
Internet infect you.




On Tue, Jul 09, 2019 at 08:40:29PM +0200, Marc Espie wrote:

> On Tue, Jul 09, 2019 at 08:04:23PM +0300, Leonid Bobrov wrote:
> > > An all-arches package snapshot currently runs at 200GB and adding
> > > symbols across the board would add a lot to this.
> >
> > Stuart and Espie, have you ever heard of compression?
>
> WTF is wrong with you ?
>
> I haven't participated to that thread yet, but I wager I know more about
> compression than you do.
>
> Just have a fucking look at the code in signify, the pkg tools, and more
> generally, EVERYWHERE I have touched in BSD before spouting insults like
> this.
>
> Most specifically:
> - we do use gzip because other compression systems won't work with little
> memory/don't have the right licence.
> - gzip allows you to stick COMMENTS in the header, which is where the
> signature lives. I DID A WHOLE FUCKING PRESENTATION ABOUT THAT DESIGN
> CHOICE.
> - pkg_create uses an LRU cache to make updates faster.  And we use some
> specificities of gzip  to make packages amenable to using rsync actually.
> - Stuart has wasted hours getting mirrors to work as good as they can.
> - *we* have spent hours trying to share stuff while keeping things secure.
>
> It *still* doesn't change the fact that a full snapshot takes up a lot of
> space.  And it's an important factor in having enough sites provide mirrors.
>
> It's also an important factor in making sure snapshots are distributed
> quickly.
>
> Heck, there are design choices in package snapshots to avoid shearing.
>
> So, to summarize. Get the hell away from this mailing-list.
>
> I don't want to have anything to do with a condescending idiot who sends
> disparaging comments my way 100% out of the blue.
>
> Fuck you very much,
>
> --
> Marc
>

--
Chris Humphries <[hidden email]>
5223 9548 E1DE DE87 F509  1888 8141 8451 6338 DD29

Reply | Threaded
Open this post in threaded view
|

Re: When will OpenBSD become a friendly place for bug reporters?

Kaashif Hymabaccus-2
In reply to this post by Stuart Henderson
On Tue, Jul 09, 2019 at 12:12:49PM -0000, Stuart Henderson wrote:

> [...]
>
> But done on a port-by-port opt-in basis for more important ports
> (major libraries, for example) it might be viable. I think it would
> need to use detached symbols in a subpackage - build with symbols,
> then postprocess the various files, it looks like this might do the
> trick
>
> objcopy --only-keep-debug $file $file.debug
> objcopy --strip-debug $file
> objcopy --add-gnu-debuglink $file.debug $file
>
> or
>
> llvm-objcopy --only-keep=debug $file $file.debug
> llvm-objcopy --strip-debug $file
> llvm-objcopy --add-gnu-debuglink $file.debug $file
>
> And then it needs additional infrastructure to handle putting these
> into subpackages (which gets complicated where a port already uses
> subpackages).

As I understand it, this is essentially what Debian does. See
https://wiki.debian.org/HowToGetABacktrace#Installing_the_debugging_symbols.

Like you say, reproducing a crash and getting a backtrace is not
always easy, having matching debug symbols in a package would be
useful.

Debian makes these packages automatically now (as you likely know),
after a decade (!!!) of thinking and implementation:
https://lists.debian.org/debian-devel/2015/12/msg00262.html. And
decades of lots of people putting in the manual work to make -dbg
packages.

Since I'm not sending any diffs to implement this, and wouldn't even
use the debug symbol packages, I'm not sure it would be worth the time
for OpenBSD.

> Unfortunately the tone of mazocomp's mail ather puts one off from
> wanting to spend time on this... Maybe I need to polish my scorefile
> as it only dropped the original mail not the replies :-)

If demanding a feature rudely means devs won't implement it, maybe I
should send a rude email demanding more bugs in OpenBSD :)

--
Kaashif Hymabaccus
GPG: 3E810B04

Reply | Threaded
Open this post in threaded view
|

Re: When will OpenBSD become a friendly place for bug reporters?

Janne Johansson-3
Den ons 10 juli 2019 kl 03:54 skrev Kaashif Hymabaccus <
[hidden email]>:

>
> On Tue, Jul 09, 2019 at 12:12:49PM -0000, Stuart Henderson wrote:
> > [...]
> >
> > But done on a port-by-port opt-in basis for more important ports
> > (major libraries, for example) it might be viable. I think it would
> > need to use detached symbols in a subpackage - build with symbols,
> > then postprocess the various files, it looks like this might do the
> > trick
> >
> > And then it needs additional infrastructure to handle putting these
> > into subpackages (which gets complicated where a port already uses
> > subpackages).
>
> As I understand it, this is essentially what Debian does. See
>
https://wiki.debian.org/HowToGetABacktrace#Installing_the_debugging_symbols.
>

I checked one of the ceph repos which does keep debug symbols as separate
packages and if you check the first 10-20 debs,
http://eu.ceph.com/debian-nautilus/pool/main/c/ceph/
you see a 20x size difference on the debug pkgs.

For the rpm-ish dists, all debug is in one simple package, weighing in at
whopping 1.9G
http://eu.ceph.com/rpm-nautilus/el7/x86_64/  (the ceph-debuginfo rpm)
whereas the other rpms are a few megs up to 34 perhaps.

Looking at snapshots package sizes for a few popular arches on obsd,

$ du -sh amd64/ i386/ sparc64 aarch64/


42.4G   amd64/
39.5G   i386/
23.7G   sparc64
20.0G   aarch64/

we'd see quite an interesting result from upping that with a factor of 20x,
in terms of disk space (which may or may not be a showstopper for some
mirrors) but also time to sync a new snap and the network used in order
to do it.

It would be very user-visible getting daily snaps of packages slower
than you do today, will everyone want to pay that price to one person
doesn't
have to "go gentoo on his ass" in order to contribute?

You don't get to be in the group that goes "why aren't release/snap packages
ready when I want them" at the same time as the group that suggests
"lets push 20x more data onto mirrors and sign-infra-boxes"

--
May the most significant bit of your life be positive.
Reply | Threaded
Open this post in threaded view
|

Re: When will OpenBSD become a friendly place for bug reporters?

Stuart Henderson
In reply to this post by Marc Espie-2
On Tue, Jul 09, 2019 at 08:04:23PM +0300, Leonid Bobrov wrote:
> Stuart and Espie, have you ever heard of compression?

once I ran out of space on the 80MB drive my BBS was running on so
I went through the whole download area, compressing everything with
arj/lha/zip using various flags (sometimes "lower level" compression
actually gave smaller files) and keeping the smallest file. so yeah
I've heard of compression. it took *ages*. and then it got blown
up by lightning anyway.


On 2019/07/09 20:40, Marc Espie wrote:
> Most specifically:
> - we do use gzip because other compression systems won't work with little
> memory/don't have the right licence.
> - gzip allows you to stick COMMENTS in the header, which is where the
> signature lives. I DID A WHOLE FUCKING PRESENTATION ABOUT THAT DESIGN
> CHOICE.
> - pkg_create uses an LRU cache to make updates faster.  And we use some
> specificities of gzip  to make packages amenable to using rsync actually.

I am quite interested in zstd (RFC8478) though.. it does seem to have a lot
of the characteristics we need:

+ bsd license
+ compressed files can be concatenated like gzip
+ "skippable frames", the signature could probably live here
+ usually better compression than gzip, even with zstd -1
+ so much faster than nearly everything else it isn't funny

- it does use a little more ram than gzip but not hugely, and with
library_aslr and kernel relinking we've pushed out the minimum
usable memory already.

max rss running *cat on a test package tar (varies a bit depending on
compression level):

gz   1.5M
zstd 3M
lz4  10M


On 2019-07-09, Nathan Hartman <[hidden email]> wrote:
> You might find this interesting:
> https://github.com/silentbicycle/heatshrink

interesting for a very constrained system, but we are ok with a couple
of MB rather than a couple of KB ram use, and it wouldn't be worth
moving to something with lower compression ratios than we already have.

Reply | Threaded
Open this post in threaded view
|

Re: When will OpenBSD become a friendly place for bug reporters?

Marc Espie-2
On Wed, Jul 10, 2019 at 11:39:53AM +0100, Stuart Henderson wrote:
> + "skippable frames", the signature could probably live here

As long as the logic is simple enough, because code for grabbing
those frames must be rewritten for signify and be trusted.