[Update] protobuf to 3.9.0

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

[Update] protobuf to 3.9.0

Jonathon Sisson-2
Hello,

Here is a diff to update protobuf to 3.9.0.

PLIST adjusted for singleton.h and wire_format_lite_inl.h removal.

Tested on amd64.  I lack the hardware to test on arm64 (looking for suggestions...Rock64?).

Jonathon



Index: Makefile
===================================================================
RCS file: /cvs/ports/devel/protobuf/Makefile,v
retrieving revision 1.24
diff -u -p -u -p -r1.24 Makefile
--- Makefile 12 Jul 2019 20:45:16 -0000 1.24
+++ Makefile 20 Jul 2019 10:52:06 -0000
@@ -2,7 +2,7 @@
 
 COMMENT = c++ protocol buffers
 
-V = 3.6.1
+V = 3.9.0
 DISTNAME = protobuf-cpp-$V
 PKGNAME = protobuf-$V
 
Index: distinfo
===================================================================
RCS file: /cvs/ports/devel/protobuf/distinfo,v
retrieving revision 1.5
diff -u -p -u -p -r1.5 distinfo
--- distinfo 9 Aug 2018 14:50:09 -0000 1.5
+++ distinfo 20 Jul 2019 10:52:07 -0000
@@ -1,2 +1,2 @@
-SHA256 (protobuf-cpp-3.6.1.tar.gz) = s3MuRxqbt5UPCQ/QRX69JTapugiRt/N4WRnGVP4qJSk=
-SIZE (protobuf-cpp-3.6.1.tar.gz) = 4450975
+SHA256 (protobuf-cpp-3.9.0.tar.gz) = EtX6y7nJdL4mHeu4H6SPT1viWDDUOmS1Uu3MS5XNbLY=
+SIZE (protobuf-cpp-3.9.0.tar.gz) = 4537469
Index: pkg/PLIST
===================================================================
RCS file: /cvs/ports/devel/protobuf/pkg/PLIST,v
retrieving revision 1.3
diff -u -p -u -p -r1.3 PLIST
--- pkg/PLIST 10 Jul 2018 10:47:42 -0000 1.3
+++ pkg/PLIST 20 Jul 2019 10:52:07 -0000
@@ -48,6 +48,7 @@ include/google/protobuf/dynamic_message.
 include/google/protobuf/empty.pb.h
 include/google/protobuf/empty.proto
 include/google/protobuf/extension_set.h
+include/google/protobuf/extension_set_inl.h
 include/google/protobuf/field_mask.pb.h
 include/google/protobuf/field_mask.proto
 include/google/protobuf/generated_enum_reflection.h
@@ -78,6 +79,10 @@ include/google/protobuf/message.h
 include/google/protobuf/message_lite.h
 include/google/protobuf/metadata.h
 include/google/protobuf/metadata_lite.h
+include/google/protobuf/parse_context.h
+include/google/protobuf/port.h
+include/google/protobuf/port_def.inc
+include/google/protobuf/port_undef.inc
 include/google/protobuf/reflection.h
 include/google/protobuf/reflection_ops.h
 include/google/protobuf/repeated_field.h
@@ -95,14 +100,15 @@ include/google/protobuf/stubs/fastmem.h
 include/google/protobuf/stubs/hash.h
 include/google/protobuf/stubs/logging.h
 include/google/protobuf/stubs/macros.h
+include/google/protobuf/stubs/map_util.h
 include/google/protobuf/stubs/mutex.h
 include/google/protobuf/stubs/once.h
 include/google/protobuf/stubs/platform_macros.h
 include/google/protobuf/stubs/port.h
-include/google/protobuf/stubs/singleton.h
 include/google/protobuf/stubs/status.h
 include/google/protobuf/stubs/stl_util.h
 include/google/protobuf/stubs/stringpiece.h
+include/google/protobuf/stubs/strutil.h
 include/google/protobuf/stubs/template_util.h
 include/google/protobuf/text_format.h
 include/google/protobuf/timestamp.pb.h
@@ -121,7 +127,6 @@ include/google/protobuf/util/type_resolv
 include/google/protobuf/util/type_resolver_util.h
 include/google/protobuf/wire_format.h
 include/google/protobuf/wire_format_lite.h
-include/google/protobuf/wire_format_lite_inl.h
 include/google/protobuf/wrappers.pb.h
 include/google/protobuf/wrappers.proto
 lib/libprotobuf-lite.a

Reply | Threaded
Open this post in threaded view
|

Re: [Update] protobuf to 3.9.0

Stuart Henderson
On 2019/07/20 04:05, Jonathon Sisson wrote:
> Hello,
>
> Here is a diff to update protobuf to 3.9.0.
>
> PLIST adjusted for singleton.h and wire_format_lite_inl.h removal.
>
> Tested on amd64.  I lack the hardware to test on arm64 (looking for suggestions...Rock64?).

Almost certainly all the shared library major versions need bumping for this update.

What have you done in the way of testing ports that depend on this?
(https://openports.pl/path/devel/protobuf, click "reverse dependencies")

Reply | Threaded
Open this post in threaded view
|

Re: [Update] protobuf to 3.9.0

Otto Moerbeek
On Sat, Jul 20, 2019 at 12:45:25PM +0100, Stuart Henderson wrote:

> On 2019/07/20 04:05, Jonathon Sisson wrote:
> > Hello,
> >
> > Here is a diff to update protobuf to 3.9.0.
> >
> > PLIST adjusted for singleton.h and wire_format_lite_inl.h removal.
> >
> > Tested on amd64.  I lack the hardware to test on arm64 (looking for suggestions...Rock64?).
>
> Almost certainly all the shared library major versions need bumping for this update.
>
> What have you done in the way of testing ports that depend on this?
> (https://openports.pl/path/devel/protobuf, click "reverse dependencies")
>

A quick dnsdist test shows that this version of protoc also crashes on
arm64.

        -Otto

Reply | Threaded
Open this post in threaded view
|

Re: [Update] protobuf to 3.9.0

Jonathon Sisson-2
In reply to this post by Stuart Henderson
On Sat, Jul 20, 2019 at 12:45:25PM +0100, Stuart Henderson wrote:

> On 2019/07/20 04:05, Jonathon Sisson wrote:
> > Hello,
> >
> > Here is a diff to update protobuf to 3.9.0.
> >
> > PLIST adjusted for singleton.h and wire_format_lite_inl.h removal.
> >
> > Tested on amd64.  I lack the hardware to test on arm64 (looking for suggestions...Rock64?).
>
> Almost certainly all the shared library major versions need bumping for this update.
>
> What have you done in the way of testing ports that depend on this?
> (https://openports.pl/path/devel/protobuf, click "reverse dependencies")
>
Testing was with umurmur by way of protobuf-c.

I'll return to the porting guide and read up on library major version bumps.

Jonathon

Reply | Threaded
Open this post in threaded view
|

Re: [Update] protobuf to 3.9.0

Jonathon Sisson-2
On Sat, Jul 20, 2019 at 05:09:00AM -0700, Jonathon Sisson wrote:

> On Sat, Jul 20, 2019 at 12:45:25PM +0100, Stuart Henderson wrote:
> > On 2019/07/20 04:05, Jonathon Sisson wrote:
> > > Hello,
> > >
> > > Here is a diff to update protobuf to 3.9.0.
> > >
> > > PLIST adjusted for singleton.h and wire_format_lite_inl.h removal.
> > >
> > > Tested on amd64.  I lack the hardware to test on arm64 (looking for suggestions...Rock64?).
> >
> > Almost certainly all the shared library major versions need bumping for this update.
> >
> > What have you done in the way of testing ports that depend on this?
> > (https://openports.pl/path/devel/protobuf, click "reverse dependencies")
> >
> Testing was with umurmur by way of protobuf-c.
>
> I'll return to the porting guide and read up on library major version bumps.
>
> Jonathon
>
Ahh, I see the following in shared_libs.log:

# SHARED_LIBS+= <libname>      <obsd version> # <orig version>
SHARED_LIBS +=  protobuf-lite        3.0      # 20.0
SHARED_LIBS +=  protobuf             3.0      # 20.0
SHARED_LIBS +=  protoc               3.0      # 20.0

I'd assume we would need to bump obsd version to 4.0?  (That is, of course, dependent on successful testing with the other ports that utilize protobuf).

Apologies for missing that.  I'll see if I can test a few of the other ports as well.

Jonathon

Reply | Threaded
Open this post in threaded view
|

Re: [Update] protobuf to 3.9.0

Stuart Henderson
In reply to this post by Jonathon Sisson-2
On 2019/07/20 05:09, Jonathon Sisson wrote:

> On Sat, Jul 20, 2019 at 12:45:25PM +0100, Stuart Henderson wrote:
> > On 2019/07/20 04:05, Jonathon Sisson wrote:
> > > Hello,
> > >
> > > Here is a diff to update protobuf to 3.9.0.
> > >
> > > PLIST adjusted for singleton.h and wire_format_lite_inl.h removal.
> > >
> > > Tested on amd64.  I lack the hardware to test on arm64 (looking for suggestions...Rock64?).
> >
> > Almost certainly all the shared library major versions need bumping for this update.
> >
> > What have you done in the way of testing ports that depend on this?
> > (https://openports.pl/path/devel/protobuf, click "reverse dependencies")
> >
> Testing was with umurmur by way of protobuf-c.
>
> I'll return to the porting guide and read up on library major version bumps.
>
> Jonathon
>

In a nutshell: if you don't touch the library version (in SHARED_LIBS which
then makes it through to the library name libfoo.so.X.Y), you are claiming
that the new library and old library are fully interchangeable: if you have
an installed program that was built against the old library+headers, you can
just update the library to the new version without recompiling the program
and everything will still work (and vice-versa with a new-compiled program
and an old library).

If any function was removed in the new version, or the prototype was changed
- e.g. "function(int a, int b)" changed to "function(long a, long b)" or
"function (int a)" - then a program compiled for the old version of the
library will no longer work. In this case the major version (X from .so.X.Y)
must be bumped.

Additionally if members of a struct were changed then in many cases there is
no compatibility in either direction and again major must be bumped. (This
is hard to spot without either using compatibility checking tools, or
comparing diffs of source trees).

If any function was *added* in the new version, but no existing function was
changed/removed and structs are all compatible, old programs can be used with
the new library, so no major bump is needed. But new-compiled programs *might*
use that new function, so the minor (Y from .so.X.Y) is bumped to prevent
them running with the old library.

These version numbers are used both to ensure that the correct library is
used for a particular program, and to trigger updates in pkg_add, so it's
important to either get things right, or err on the side of unnecessary
bumps, otherwise things will break for users.

In this case looking at the diff between the list of functions in old+new
library versions it's thousands of lines (this is quite often the case for
C++ libraries) and the chances of old binaries working with new library are
very low.

This is also useful: https://abi-laboratory.pro/?view=timeline&l=protobuf

Reply | Threaded
Open this post in threaded view
|

Re: [Update] protobuf to 3.9.0

Stuart Henderson
In reply to this post by Jonathon Sisson-2
On 2019/07/20 05:24, Jonathon Sisson wrote:

> On Sat, Jul 20, 2019 at 05:09:00AM -0700, Jonathon Sisson wrote:
> > On Sat, Jul 20, 2019 at 12:45:25PM +0100, Stuart Henderson wrote:
> > > On 2019/07/20 04:05, Jonathon Sisson wrote:
> > > > Hello,
> > > >
> > > > Here is a diff to update protobuf to 3.9.0.
> > > >
> > > > PLIST adjusted for singleton.h and wire_format_lite_inl.h removal.
> > > >
> > > > Tested on amd64.  I lack the hardware to test on arm64 (looking for suggestions...Rock64?).
> > >
> > > Almost certainly all the shared library major versions need bumping for this update.
> > >
> > > What have you done in the way of testing ports that depend on this?
> > > (https://openports.pl/path/devel/protobuf, click "reverse dependencies")
> > >
> > Testing was with umurmur by way of protobuf-c.
> >
> > I'll return to the porting guide and read up on library major version bumps.
> >
> > Jonathon
> >
> Ahh, I see the following in shared_libs.log:
>
> # SHARED_LIBS+= <libname>      <obsd version> # <orig version>
> SHARED_LIBS +=  protobuf-lite        3.0      # 20.0
> SHARED_LIBS +=  protobuf             3.0      # 20.0
> SHARED_LIBS +=  protoc               3.0      # 20.0
>
> I'd assume we would need to bump obsd version to 4.0?  (That is, of course, dependent on successful testing with the other ports that utilize protobuf).

Yes.

> Apologies for missing that.  I'll see if I can test a few of the other ports as well.
>
> Jonathon
>

Reply | Threaded
Open this post in threaded view
|

Re: [Update] protobuf to 3.9.0

Jonathon Sisson-2
On Sat, Jul 20, 2019 at 02:01:27PM +0100, Stuart Henderson wrote:

> On 2019/07/20 05:24, Jonathon Sisson wrote:
> > On Sat, Jul 20, 2019 at 05:09:00AM -0700, Jonathon Sisson wrote:
> > > On Sat, Jul 20, 2019 at 12:45:25PM +0100, Stuart Henderson wrote:
> > > > On 2019/07/20 04:05, Jonathon Sisson wrote:
> > > > > Hello,
> > > > >
> > > > > Here is a diff to update protobuf to 3.9.0.
> > > > >
> > > > > PLIST adjusted for singleton.h and wire_format_lite_inl.h removal.
> > > > >
> > > > > Tested on amd64.  I lack the hardware to test on arm64 (looking for suggestions...Rock64?).
> > > >
> > > > Almost certainly all the shared library major versions need bumping for this update.
> > > >
> > > > What have you done in the way of testing ports that depend on this?
> > > > (https://openports.pl/path/devel/protobuf, click "reverse dependencies")
> > > >
> > > Testing was with umurmur by way of protobuf-c.
> > >
> > > I'll return to the porting guide and read up on library major version bumps.
> > >
> > > Jonathon
> > >
> > Ahh, I see the following in shared_libs.log:
> >
> > # SHARED_LIBS+= <libname>      <obsd version> # <orig version>
> > SHARED_LIBS +=  protobuf-lite        3.0      # 20.0
> > SHARED_LIBS +=  protobuf             3.0      # 20.0
> > SHARED_LIBS +=  protoc               3.0      # 20.0
> >
> > I'd assume we would need to bump obsd version to 4.0?  (That is, of course, dependent on successful testing with the other ports that utilize protobuf).
>
> Yes.
>
> > Apologies for missing that.  I'll see if I can test a few of the other ports as well.
> >
> > Jonathon
> >
>

Revised patch below, with SHARED_LIBS bump as requested.  I've tested with mosh, which works as expected (protobuf-3.6.1 client to protobuf-3.9.0 server).

I'll test more ports as free time allows.

Thank you for the pointers, Stuart =)

Jonathon



Index: Makefile
===================================================================
RCS file: /cvs/ports/devel/protobuf/Makefile,v
retrieving revision 1.24
diff -u -p -u -p -r1.24 Makefile
--- Makefile 12 Jul 2019 20:45:16 -0000 1.24
+++ Makefile 20 Jul 2019 13:10:46 -0000
@@ -2,15 +2,15 @@
 
 COMMENT = c++ protocol buffers
 
-V = 3.6.1
+V = 3.9.0
 DISTNAME = protobuf-cpp-$V
 PKGNAME = protobuf-$V
 
 WRKDIST = ${WRKDIR}/protobuf-${V}
 
-SHARED_LIBS += protobuf 3.0   # 11.0
-SHARED_LIBS += protoc 3.0   # 11.0
-SHARED_LIBS += protobuf-lite 3.0   # 11.0
+SHARED_LIBS += protobuf 4.0   # 20.0
+SHARED_LIBS += protoc 4.0   # 20.0
+SHARED_LIBS += protobuf-lite 4.0   # 20.0
 
 CATEGORIES = devel
 
Index: distinfo
===================================================================
RCS file: /cvs/ports/devel/protobuf/distinfo,v
retrieving revision 1.5
diff -u -p -u -p -r1.5 distinfo
--- distinfo 9 Aug 2018 14:50:09 -0000 1.5
+++ distinfo 20 Jul 2019 13:10:46 -0000
@@ -1,2 +1,2 @@
-SHA256 (protobuf-cpp-3.6.1.tar.gz) = s3MuRxqbt5UPCQ/QRX69JTapugiRt/N4WRnGVP4qJSk=
-SIZE (protobuf-cpp-3.6.1.tar.gz) = 4450975
+SHA256 (protobuf-cpp-3.9.0.tar.gz) = EtX6y7nJdL4mHeu4H6SPT1viWDDUOmS1Uu3MS5XNbLY=
+SIZE (protobuf-cpp-3.9.0.tar.gz) = 4537469
Index: pkg/PLIST
===================================================================
RCS file: /cvs/ports/devel/protobuf/pkg/PLIST,v
retrieving revision 1.3
diff -u -p -u -p -r1.3 PLIST
--- pkg/PLIST 10 Jul 2018 10:47:42 -0000 1.3
+++ pkg/PLIST 20 Jul 2019 13:10:47 -0000
@@ -48,6 +48,7 @@ include/google/protobuf/dynamic_message.
 include/google/protobuf/empty.pb.h
 include/google/protobuf/empty.proto
 include/google/protobuf/extension_set.h
+include/google/protobuf/extension_set_inl.h
 include/google/protobuf/field_mask.pb.h
 include/google/protobuf/field_mask.proto
 include/google/protobuf/generated_enum_reflection.h
@@ -78,6 +79,10 @@ include/google/protobuf/message.h
 include/google/protobuf/message_lite.h
 include/google/protobuf/metadata.h
 include/google/protobuf/metadata_lite.h
+include/google/protobuf/parse_context.h
+include/google/protobuf/port.h
+include/google/protobuf/port_def.inc
+include/google/protobuf/port_undef.inc
 include/google/protobuf/reflection.h
 include/google/protobuf/reflection_ops.h
 include/google/protobuf/repeated_field.h
@@ -95,14 +100,15 @@ include/google/protobuf/stubs/fastmem.h
 include/google/protobuf/stubs/hash.h
 include/google/protobuf/stubs/logging.h
 include/google/protobuf/stubs/macros.h
+include/google/protobuf/stubs/map_util.h
 include/google/protobuf/stubs/mutex.h
 include/google/protobuf/stubs/once.h
 include/google/protobuf/stubs/platform_macros.h
 include/google/protobuf/stubs/port.h
-include/google/protobuf/stubs/singleton.h
 include/google/protobuf/stubs/status.h
 include/google/protobuf/stubs/stl_util.h
 include/google/protobuf/stubs/stringpiece.h
+include/google/protobuf/stubs/strutil.h
 include/google/protobuf/stubs/template_util.h
 include/google/protobuf/text_format.h
 include/google/protobuf/timestamp.pb.h
@@ -121,7 +127,6 @@ include/google/protobuf/util/type_resolv
 include/google/protobuf/util/type_resolver_util.h
 include/google/protobuf/wire_format.h
 include/google/protobuf/wire_format_lite.h
-include/google/protobuf/wire_format_lite_inl.h
 include/google/protobuf/wrappers.pb.h
 include/google/protobuf/wrappers.proto
 lib/libprotobuf-lite.a

Reply | Threaded
Open this post in threaded view
|

Re: [Update] protobuf to 3.9.0

Otto Moerbeek
On Sat, Jul 20, 2019 at 06:15:19AM -0700, Jonathon Sisson wrote:

> On Sat, Jul 20, 2019 at 02:01:27PM +0100, Stuart Henderson wrote:
> > On 2019/07/20 05:24, Jonathon Sisson wrote:
> > > On Sat, Jul 20, 2019 at 05:09:00AM -0700, Jonathon Sisson wrote:
> > > > On Sat, Jul 20, 2019 at 12:45:25PM +0100, Stuart Henderson wrote:
> > > > > On 2019/07/20 04:05, Jonathon Sisson wrote:
> > > > > > Hello,
> > > > > >
> > > > > > Here is a diff to update protobuf to 3.9.0.
> > > > > >
> > > > > > PLIST adjusted for singleton.h and wire_format_lite_inl.h removal.
> > > > > >
> > > > > > Tested on amd64.  I lack the hardware to test on arm64 (looking for suggestions...Rock64?).
> > > > >
> > > > > Almost certainly all the shared library major versions need bumping for this update.
> > > > >
> > > > > What have you done in the way of testing ports that depend on this?
> > > > > (https://openports.pl/path/devel/protobuf, click "reverse dependencies")
> > > > >
> > > > Testing was with umurmur by way of protobuf-c.
> > > >
> > > > I'll return to the porting guide and read up on library major version bumps.
> > > >
> > > > Jonathon
> > > >
> > > Ahh, I see the following in shared_libs.log:
> > >
> > > # SHARED_LIBS+= <libname>      <obsd version> # <orig version>
> > > SHARED_LIBS +=  protobuf-lite        3.0      # 20.0
> > > SHARED_LIBS +=  protobuf             3.0      # 20.0
> > > SHARED_LIBS +=  protoc               3.0      # 20.0
> > >
> > > I'd assume we would need to bump obsd version to 4.0?  (That is, of course, dependent on successful testing with the other ports that utilize protobuf).
> >
> > Yes.
> >
> > > Apologies for missing that.  I'll see if I can test a few of the other ports as well.
> > >
> > > Jonathon
> > >
> >
>
> Revised patch below, with SHARED_LIBS bump as requested.  I've tested with mosh, which works as expected (protobuf-3.6.1 client to protobuf-3.9.0 server).
>
> I'll test more ports as free time allows.
>
> Thank you for the pointers, Stuart =)
>
> Jonathon
>
>
>
> Index: Makefile
> ===================================================================
> RCS file: /cvs/ports/devel/protobuf/Makefile,v
> retrieving revision 1.24
> diff -u -p -u -p -r1.24 Makefile
> --- Makefile 12 Jul 2019 20:45:16 -0000 1.24
> +++ Makefile 20 Jul 2019 13:10:46 -0000
> @@ -2,15 +2,15 @@
>  
>  COMMENT = c++ protocol buffers
>  
> -V = 3.6.1
> +V = 3.9.0
>  DISTNAME = protobuf-cpp-$V
>  PKGNAME = protobuf-$V
>  
>  WRKDIST = ${WRKDIR}/protobuf-${V}
>  
> -SHARED_LIBS += protobuf 3.0   # 11.0
> -SHARED_LIBS += protoc 3.0   # 11.0
> -SHARED_LIBS += protobuf-lite 3.0   # 11.0
> +SHARED_LIBS += protobuf 4.0   # 20.0
> +SHARED_LIBS += protoc 4.0   # 20.0
> +SHARED_LIBS += protobuf-lite 4.0   # 20.0
>  
>  CATEGORIES = devel
>  
> Index: distinfo
> ===================================================================
> RCS file: /cvs/ports/devel/protobuf/distinfo,v
> retrieving revision 1.5
> diff -u -p -u -p -r1.5 distinfo
> --- distinfo 9 Aug 2018 14:50:09 -0000 1.5
> +++ distinfo 20 Jul 2019 13:10:46 -0000
> @@ -1,2 +1,2 @@
> -SHA256 (protobuf-cpp-3.6.1.tar.gz) = s3MuRxqbt5UPCQ/QRX69JTapugiRt/N4WRnGVP4qJSk=
> -SIZE (protobuf-cpp-3.6.1.tar.gz) = 4450975
> +SHA256 (protobuf-cpp-3.9.0.tar.gz) = EtX6y7nJdL4mHeu4H6SPT1viWDDUOmS1Uu3MS5XNbLY=
> +SIZE (protobuf-cpp-3.9.0.tar.gz) = 4537469
> Index: pkg/PLIST
> ===================================================================
> RCS file: /cvs/ports/devel/protobuf/pkg/PLIST,v
> retrieving revision 1.3
> diff -u -p -u -p -r1.3 PLIST
> --- pkg/PLIST 10 Jul 2018 10:47:42 -0000 1.3
> +++ pkg/PLIST 20 Jul 2019 13:10:47 -0000
> @@ -48,6 +48,7 @@ include/google/protobuf/dynamic_message.
>  include/google/protobuf/empty.pb.h
>  include/google/protobuf/empty.proto
>  include/google/protobuf/extension_set.h
> +include/google/protobuf/extension_set_inl.h
>  include/google/protobuf/field_mask.pb.h
>  include/google/protobuf/field_mask.proto
>  include/google/protobuf/generated_enum_reflection.h
> @@ -78,6 +79,10 @@ include/google/protobuf/message.h
>  include/google/protobuf/message_lite.h
>  include/google/protobuf/metadata.h
>  include/google/protobuf/metadata_lite.h
> +include/google/protobuf/parse_context.h
> +include/google/protobuf/port.h
> +include/google/protobuf/port_def.inc
> +include/google/protobuf/port_undef.inc
>  include/google/protobuf/reflection.h
>  include/google/protobuf/reflection_ops.h
>  include/google/protobuf/repeated_field.h
> @@ -95,14 +100,15 @@ include/google/protobuf/stubs/fastmem.h
>  include/google/protobuf/stubs/hash.h
>  include/google/protobuf/stubs/logging.h
>  include/google/protobuf/stubs/macros.h
> +include/google/protobuf/stubs/map_util.h
>  include/google/protobuf/stubs/mutex.h
>  include/google/protobuf/stubs/once.h
>  include/google/protobuf/stubs/platform_macros.h
>  include/google/protobuf/stubs/port.h
> -include/google/protobuf/stubs/singleton.h
>  include/google/protobuf/stubs/status.h
>  include/google/protobuf/stubs/stl_util.h
>  include/google/protobuf/stubs/stringpiece.h
> +include/google/protobuf/stubs/strutil.h
>  include/google/protobuf/stubs/template_util.h
>  include/google/protobuf/text_format.h
>  include/google/protobuf/timestamp.pb.h
> @@ -121,7 +127,6 @@ include/google/protobuf/util/type_resolv
>  include/google/protobuf/util/type_resolver_util.h
>  include/google/protobuf/wire_format.h
>  include/google/protobuf/wire_format_lite.h
> -include/google/protobuf/wire_format_lite_inl.h
>  include/google/protobuf/wrappers.pb.h
>  include/google/protobuf/wrappers.proto
>  lib/libprotobuf-lite.a
>

Testing with -O0 or -O1 makes the bug go away.
This is with a -g build (default -O2)

#0  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long (this=0x0) at /usr/include/c++/v1/string:1427
1427        {return bool(__r_.first().__s.__size_ & __short_mask);}
(gdb) bt
#0  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long (this=0x0) at /usr/include/c++/v1/string:1427
#1  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__get_pointer (this=0x0) at /usr/include/c++/v1/string:1521
#2  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::data (this=0x0) at /usr/include/c++/v1/string:1249
#3  google::protobuf::StringPiece::StringPiece<std::__1::allocator<char> > (
    this=<optimized out>, str=...)
    at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/stubs/stringpiece.h:224
#4  google::protobuf::CEscape (src=...)
    at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/stubs/strutil.cc:617
#5  0x00000018e0ae8e54 in google::protobuf::compiler::cpp::DefaultValue (
    options=..., field=<optimized out>)
    at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/compiler/cpp/cpp_helpers.cc:678
#6  0x00000018e0b1f5b0 in google::protobuf::compiler::cpp::(anonymous namespace)::SetStringVariables (descriptor=0x1872f6bc00, variables=0x19422e2398,
    options=...)
    at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/compiler/cpp/cpp_string_field.cc:54
#7  0x00000018e0b1f524 in google::protobuf::compiler::cpp::StringFieldGenerator::StringFieldGenerator (this=0x19422e2380, descriptor=0x1872f6bc00, options=...)
    at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/compiler/cpp/cpp_string_field.cc:93
#8  0x00000018e0ad0c6c in google::protobuf::compiler::cpp::FieldGeneratorMap::MakeGenerator (field=0x1872f6bc00, options=..., scc_analyzer=0x7fffff40b8)
    at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/compiler/cpp/cpp_field.cc:186
#9  0x00000018e0ad08cc in google::protobuf::compiler::cpp::FieldGeneratorMap::FieldGeneratorMap (this=0x18f9eb1ea8, descriptor=0x18a64f7c00, options=...,
    scc_analyzer=<optimized out>)
    at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/compiler/cpp/cpp_field.cc:134
#10 0x00000018e0af7174 in google::protobuf::compiler::cpp::MessageGenerator::MessageGenerator (this=0x18f9eb1e00, descriptor=<optimized out>, vars=...,
    index_in_file_messages=<optimized out>, options=...,
    scc_analyzer=<optimized out>)
---Type <return> to continue, or q <return> to quit---q

this being NULL is a typical case of "cannot or should not" happen.
This requires some deep digging I'm afraid. Or weasel out and compile
with -O1 on aarch64.

        -Otto

Reply | Threaded
Open this post in threaded view
|

Re: [Update] protobuf to 3.9.0

Stuart Henderson
On 2019/07/20 16:57, Otto Moerbeek wrote:

> On Sat, Jul 20, 2019 at 06:15:19AM -0700, Jonathon Sisson wrote:
>
> > On Sat, Jul 20, 2019 at 02:01:27PM +0100, Stuart Henderson wrote:
> > > On 2019/07/20 05:24, Jonathon Sisson wrote:
> > > > On Sat, Jul 20, 2019 at 05:09:00AM -0700, Jonathon Sisson wrote:
> > > > > On Sat, Jul 20, 2019 at 12:45:25PM +0100, Stuart Henderson wrote:
> > > > > > On 2019/07/20 04:05, Jonathon Sisson wrote:
> > > > > > > Hello,
> > > > > > >
> > > > > > > Here is a diff to update protobuf to 3.9.0.
> > > > > > >
> > > > > > > PLIST adjusted for singleton.h and wire_format_lite_inl.h removal.
> > > > > > >
> > > > > > > Tested on amd64.  I lack the hardware to test on arm64 (looking for suggestions...Rock64?).
> > > > > >
> > > > > > Almost certainly all the shared library major versions need bumping for this update.
> > > > > >
> > > > > > What have you done in the way of testing ports that depend on this?
> > > > > > (https://openports.pl/path/devel/protobuf, click "reverse dependencies")
> > > > > >
> > > > > Testing was with umurmur by way of protobuf-c.
> > > > >
> > > > > I'll return to the porting guide and read up on library major version bumps.
> > > > >
> > > > > Jonathon
> > > > >
> > > > Ahh, I see the following in shared_libs.log:
> > > >
> > > > # SHARED_LIBS+= <libname>      <obsd version> # <orig version>
> > > > SHARED_LIBS +=  protobuf-lite        3.0      # 20.0
> > > > SHARED_LIBS +=  protobuf             3.0      # 20.0
> > > > SHARED_LIBS +=  protoc               3.0      # 20.0
> > > >
> > > > I'd assume we would need to bump obsd version to 4.0?  (That is, of course, dependent on successful testing with the other ports that utilize protobuf).
> > >
> > > Yes.
> > >
> > > > Apologies for missing that.  I'll see if I can test a few of the other ports as well.
> > > >
> > > > Jonathon
> > > >
> > >
> >
> > Revised patch below, with SHARED_LIBS bump as requested.  I've tested with mosh, which works as expected (protobuf-3.6.1 client to protobuf-3.9.0 server).
> >
> > I'll test more ports as free time allows.
> >
> > Thank you for the pointers, Stuart =)
> >
> > Jonathon
> >
> >
> >
> > Index: Makefile
> > ===================================================================
> > RCS file: /cvs/ports/devel/protobuf/Makefile,v
> > retrieving revision 1.24
> > diff -u -p -u -p -r1.24 Makefile
> > --- Makefile 12 Jul 2019 20:45:16 -0000 1.24
> > +++ Makefile 20 Jul 2019 13:10:46 -0000
> > @@ -2,15 +2,15 @@
> >  
> >  COMMENT = c++ protocol buffers
> >  
> > -V = 3.6.1
> > +V = 3.9.0
> >  DISTNAME = protobuf-cpp-$V
> >  PKGNAME = protobuf-$V
> >  
> >  WRKDIST = ${WRKDIR}/protobuf-${V}
> >  
> > -SHARED_LIBS += protobuf 3.0   # 11.0
> > -SHARED_LIBS += protoc 3.0   # 11.0
> > -SHARED_LIBS += protobuf-lite 3.0   # 11.0
> > +SHARED_LIBS += protobuf 4.0   # 20.0
> > +SHARED_LIBS += protoc 4.0   # 20.0
> > +SHARED_LIBS += protobuf-lite 4.0   # 20.0
> >  
> >  CATEGORIES = devel
> >  
> > Index: distinfo
> > ===================================================================
> > RCS file: /cvs/ports/devel/protobuf/distinfo,v
> > retrieving revision 1.5
> > diff -u -p -u -p -r1.5 distinfo
> > --- distinfo 9 Aug 2018 14:50:09 -0000 1.5
> > +++ distinfo 20 Jul 2019 13:10:46 -0000
> > @@ -1,2 +1,2 @@
> > -SHA256 (protobuf-cpp-3.6.1.tar.gz) = s3MuRxqbt5UPCQ/QRX69JTapugiRt/N4WRnGVP4qJSk=
> > -SIZE (protobuf-cpp-3.6.1.tar.gz) = 4450975
> > +SHA256 (protobuf-cpp-3.9.0.tar.gz) = EtX6y7nJdL4mHeu4H6SPT1viWDDUOmS1Uu3MS5XNbLY=
> > +SIZE (protobuf-cpp-3.9.0.tar.gz) = 4537469
> > Index: pkg/PLIST
> > ===================================================================
> > RCS file: /cvs/ports/devel/protobuf/pkg/PLIST,v
> > retrieving revision 1.3
> > diff -u -p -u -p -r1.3 PLIST
> > --- pkg/PLIST 10 Jul 2018 10:47:42 -0000 1.3
> > +++ pkg/PLIST 20 Jul 2019 13:10:47 -0000
> > @@ -48,6 +48,7 @@ include/google/protobuf/dynamic_message.
> >  include/google/protobuf/empty.pb.h
> >  include/google/protobuf/empty.proto
> >  include/google/protobuf/extension_set.h
> > +include/google/protobuf/extension_set_inl.h
> >  include/google/protobuf/field_mask.pb.h
> >  include/google/protobuf/field_mask.proto
> >  include/google/protobuf/generated_enum_reflection.h
> > @@ -78,6 +79,10 @@ include/google/protobuf/message.h
> >  include/google/protobuf/message_lite.h
> >  include/google/protobuf/metadata.h
> >  include/google/protobuf/metadata_lite.h
> > +include/google/protobuf/parse_context.h
> > +include/google/protobuf/port.h
> > +include/google/protobuf/port_def.inc
> > +include/google/protobuf/port_undef.inc
> >  include/google/protobuf/reflection.h
> >  include/google/protobuf/reflection_ops.h
> >  include/google/protobuf/repeated_field.h
> > @@ -95,14 +100,15 @@ include/google/protobuf/stubs/fastmem.h
> >  include/google/protobuf/stubs/hash.h
> >  include/google/protobuf/stubs/logging.h
> >  include/google/protobuf/stubs/macros.h
> > +include/google/protobuf/stubs/map_util.h
> >  include/google/protobuf/stubs/mutex.h
> >  include/google/protobuf/stubs/once.h
> >  include/google/protobuf/stubs/platform_macros.h
> >  include/google/protobuf/stubs/port.h
> > -include/google/protobuf/stubs/singleton.h
> >  include/google/protobuf/stubs/status.h
> >  include/google/protobuf/stubs/stl_util.h
> >  include/google/protobuf/stubs/stringpiece.h
> > +include/google/protobuf/stubs/strutil.h
> >  include/google/protobuf/stubs/template_util.h
> >  include/google/protobuf/text_format.h
> >  include/google/protobuf/timestamp.pb.h
> > @@ -121,7 +127,6 @@ include/google/protobuf/util/type_resolv
> >  include/google/protobuf/util/type_resolver_util.h
> >  include/google/protobuf/wire_format.h
> >  include/google/protobuf/wire_format_lite.h
> > -include/google/protobuf/wire_format_lite_inl.h
> >  include/google/protobuf/wrappers.pb.h
> >  include/google/protobuf/wrappers.proto
> >  lib/libprotobuf-lite.a
> >
>
> Testing with -O0 or -O1 makes the bug go away.
> This is with a -g build (default -O2)
>
> #0  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long (this=0x0) at /usr/include/c++/v1/string:1427
> 1427        {return bool(__r_.first().__s.__size_ & __short_mask);}
> (gdb) bt
> #0  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long (this=0x0) at /usr/include/c++/v1/string:1427
> #1  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__get_pointer (this=0x0) at /usr/include/c++/v1/string:1521
> #2  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::data (this=0x0) at /usr/include/c++/v1/string:1249
> #3  google::protobuf::StringPiece::StringPiece<std::__1::allocator<char> > (
>     this=<optimized out>, str=...)
>     at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/stubs/stringpiece.h:224
> #4  google::protobuf::CEscape (src=...)
>     at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/stubs/strutil.cc:617
> #5  0x00000018e0ae8e54 in google::protobuf::compiler::cpp::DefaultValue (
>     options=..., field=<optimized out>)
>     at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/compiler/cpp/cpp_helpers.cc:678
> #6  0x00000018e0b1f5b0 in google::protobuf::compiler::cpp::(anonymous namespace)::SetStringVariables (descriptor=0x1872f6bc00, variables=0x19422e2398,
>     options=...)
>     at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/compiler/cpp/cpp_string_field.cc:54
> #7  0x00000018e0b1f524 in google::protobuf::compiler::cpp::StringFieldGenerator::StringFieldGenerator (this=0x19422e2380, descriptor=0x1872f6bc00, options=...)
>     at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/compiler/cpp/cpp_string_field.cc:93
> #8  0x00000018e0ad0c6c in google::protobuf::compiler::cpp::FieldGeneratorMap::MakeGenerator (field=0x1872f6bc00, options=..., scc_analyzer=0x7fffff40b8)
>     at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/compiler/cpp/cpp_field.cc:186
> #9  0x00000018e0ad08cc in google::protobuf::compiler::cpp::FieldGeneratorMap::FieldGeneratorMap (this=0x18f9eb1ea8, descriptor=0x18a64f7c00, options=...,
>     scc_analyzer=<optimized out>)
>     at /usr/ports/pobj/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/compiler/cpp/cpp_field.cc:134
> #10 0x00000018e0af7174 in google::protobuf::compiler::cpp::MessageGenerator::MessageGenerator (this=0x18f9eb1e00, descriptor=<optimized out>, vars=...,
>     index_in_file_messages=<optimized out>, options=...,
>     scc_analyzer=<optimized out>)
> ---Type <return> to continue, or q <return> to quit---q
>
> this being NULL is a typical case of "cannot or should not" happen.
> This requires some deep digging I'm afraid. Or weasel out and compile
> with -O1 on aarch64.
>
> -Otto
>

So here is the "weasel out" diff below ... I've also run "make update-patches"
to update the line numbers.

"make test" fails following the update though:

=====
c++ -DHAVE_CONFIG_H -I. -I/usr/obj/ports/protobuf-3.9.0/protobuf-3.9.0/src -I.. -I/usr/local/include  -I/usr/local/include  -pthread -DHAVE_PTHREAD=1 -DHAVE_ZLIB=1 -Wall -Wno-sign-compare -O2 -pipe -MT google/protobuf/protobuf_test-descriptor_unittest.o -MD -MP -MF google/protobuf/.deps/protobuf_test-descriptor_unittest.Tpo -c -o google/protobuf/protobuf_test-descriptor_unittest.o `test -f 'google/protobuf/descriptor_unittest.cc' || echo '/usr/obj/ports/protobuf-3.9.0/protobuf-3.9.0/src/'`google/protobuf/descriptor_unittest.cc
/usr/obj/ports/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/descriptor_unittest.cc:2972:26: error: C++ requires a type specifier for all declarations
INSTANTIATE_TEST_SUITE_P(DatabaseSource, AllowUnknownDependenciesTest,
                         ^
/usr/obj/ports/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/descriptor_unittest.cc:2972:70: error: parameter type 'google::protobuf::descriptor_unittest::AllowUnknownDependenciesTest' is an abstract class
INSTANTIATE_TEST_SUITE_P(DatabaseSource, AllowUnknownDependenciesTest,
                                                                     ^
/usr/local/include/gtest/gtest.h:446:16: note: unimplemented pure virtual method 'TestBody' in 'AllowUnknownDependenciesTest'
  virtual void TestBody() = 0;
               ^
/usr/obj/ports/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/descriptor_unittest.cc:2973:35: error: no type named 'Values' in namespace 'testing'
                         testing::Values(NO_DATABASE, FALLBACK_DATABASE));
                         ~~~~~~~~~^
/usr/obj/ports/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/descriptor_unittest.cc:2973:53: error: expected ')'
                         testing::Values(NO_DATABASE, FALLBACK_DATABASE));
                                                    ^
/usr/obj/ports/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/descriptor_unittest.cc:2973:41: note: to match this '('
                         testing::Values(NO_DATABASE, FALLBACK_DATABASE));
                                        ^
/usr/obj/ports/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/descriptor_unittest.cc:2972:1: error: C++ requires a type specifier for all declarations
INSTANTIATE_TEST_SUITE_P(DatabaseSource, AllowUnknownDependenciesTest,
^
5 errors generated.
=====





Index: Makefile
===================================================================
RCS file: /cvs/ports/devel/protobuf/Makefile,v
retrieving revision 1.24
diff -u -p -r1.24 Makefile
--- Makefile 12 Jul 2019 20:45:16 -0000 1.24
+++ Makefile 20 Jul 2019 21:52:41 -0000
@@ -2,15 +2,15 @@
 
 COMMENT = c++ protocol buffers
 
-V = 3.6.1
+V = 3.9.0
 DISTNAME = protobuf-cpp-$V
 PKGNAME = protobuf-$V
 
 WRKDIST = ${WRKDIR}/protobuf-${V}
 
-SHARED_LIBS += protobuf 3.0   # 11.0
-SHARED_LIBS += protoc 3.0   # 11.0
-SHARED_LIBS += protobuf-lite 3.0   # 11.0
+SHARED_LIBS += protobuf 4.0   # 20.0
+SHARED_LIBS += protoc 4.0   # 20.0
+SHARED_LIBS += protobuf-lite 4.0   # 20.0
 
 CATEGORIES = devel
 
@@ -31,6 +31,12 @@ SEPARATE_BUILD = Yes
 USE_GMAKE = Yes
 CONFIGURE_STYLE = gnu
 CONFIGURE_ARGS += --with-zlib
+
+.if ${MACHINE_ARCH} == aarch64
+# XXX restrict to -O1 or less to avoid runtime crashes e.g. in dnsdist
+# https://marc.info/?l=openbsd-ports&m=156363469501634&w=2
+CFLAGS := ${CFLAGS:S/-O2/-O1/}
+.endif
 
 .include <bsd.port.mk>
 
Index: distinfo
===================================================================
RCS file: /cvs/ports/devel/protobuf/distinfo,v
retrieving revision 1.5
diff -u -p -r1.5 distinfo
--- distinfo 9 Aug 2018 14:50:09 -0000 1.5
+++ distinfo 20 Jul 2019 21:52:41 -0000
@@ -1,2 +1,2 @@
-SHA256 (protobuf-cpp-3.6.1.tar.gz) = s3MuRxqbt5UPCQ/QRX69JTapugiRt/N4WRnGVP4qJSk=
-SIZE (protobuf-cpp-3.6.1.tar.gz) = 4450975
+SHA256 (protobuf-cpp-3.9.0.tar.gz) = EtX6y7nJdL4mHeu4H6SPT1viWDDUOmS1Uu3MS5XNbLY=
+SIZE (protobuf-cpp-3.9.0.tar.gz) = 4537469
Index: patches/patch-src_Makefile_in
===================================================================
RCS file: /cvs/ports/devel/protobuf/patches/patch-src_Makefile_in,v
retrieving revision 1.5
diff -u -p -r1.5 patch-src_Makefile_in
--- patches/patch-src_Makefile_in 9 Aug 2018 14:50:10 -0000 1.5
+++ patches/patch-src_Makefile_in 20 Jul 2019 21:52:41 -0000
@@ -3,7 +3,7 @@ $OpenBSD: patch-src_Makefile_in,v 1.5 20
 Index: src/Makefile.in
 --- src/Makefile.in.orig
 +++ src/Makefile.in
-@@ -475,10 +475,7 @@ protobuf_lazy_descriptor_test_OBJECTS =  \
+@@ -483,10 +483,7 @@ protobuf_lazy_descriptor_test_OBJECTS =  \
  $(am_protobuf_lazy_descriptor_test_OBJECTS) \
  $(nodist_protobuf_lazy_descriptor_test_OBJECTS)
  protobuf_lazy_descriptor_test_DEPENDENCIES = $(am__DEPENDENCIES_1) \
@@ -15,7 +15,7 @@ Index: src/Makefile.in
  protobuf_lazy_descriptor_test_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CXX \
  $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CXXLD) \
  $(protobuf_lazy_descriptor_test_CXXFLAGS) $(CXXFLAGS) \
-@@ -498,9 +495,7 @@ protobuf_lite_arena_test_OBJECTS =  \
+@@ -506,9 +503,7 @@ protobuf_lite_arena_test_OBJECTS =  \
  $(am_protobuf_lite_arena_test_OBJECTS) \
  $(nodist_protobuf_lite_arena_test_OBJECTS)
  protobuf_lite_arena_test_DEPENDENCIES = $(am__DEPENDENCIES_1) \
@@ -26,7 +26,7 @@ Index: src/Makefile.in
  protobuf_lite_arena_test_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CXX \
  $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CXXLD) \
  $(protobuf_lite_arena_test_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) \
-@@ -521,9 +516,7 @@ nodist_protobuf_lite_test_OBJECTS = $(am__objects_10)
+@@ -529,9 +524,7 @@ nodist_protobuf_lite_test_OBJECTS = $(am__objects_10)
  protobuf_lite_test_OBJECTS = $(am_protobuf_lite_test_OBJECTS) \
  $(nodist_protobuf_lite_test_OBJECTS)
  protobuf_lite_test_DEPENDENCIES = $(am__DEPENDENCIES_1) \
@@ -37,7 +37,7 @@ Index: src/Makefile.in
  protobuf_lite_test_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CXX \
  $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CXXLD) \
  $(protobuf_lite_test_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) \
-@@ -660,9 +653,7 @@ nodist_protobuf_test_OBJECTS = $(am__objects_13)
+@@ -670,9 +663,7 @@ nodist_protobuf_test_OBJECTS = $(am__objects_13)
  protobuf_test_OBJECTS = $(am_protobuf_test_OBJECTS) \
  $(nodist_protobuf_test_OBJECTS)
  protobuf_test_DEPENDENCIES = $(am__DEPENDENCIES_1) libprotobuf.la \
@@ -48,7 +48,7 @@ Index: src/Makefile.in
  protobuf_test_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CXX \
  $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CXXLD) \
  $(protobuf_test_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) \
-@@ -676,7 +667,7 @@ am_test_plugin_OBJECTS = google/protobuf/compiler/test
+@@ -686,7 +677,7 @@ am_test_plugin_OBJECTS = google/protobuf/compiler/test
  google/protobuf/compiler/test_plugin-test_plugin.$(OBJEXT)
  test_plugin_OBJECTS = $(am_test_plugin_OBJECTS)
  test_plugin_DEPENDENCIES = $(am__DEPENDENCIES_1) libprotobuf.la \
@@ -57,7 +57,7 @@ Index: src/Makefile.in
  am__zcgunzip_SOURCES_DIST = google/protobuf/testing/zcgunzip.cc
  @HAVE_ZLIB_TRUE@am_zcgunzip_OBJECTS =  \
  @HAVE_ZLIB_TRUE@ google/protobuf/testing/zcgunzip.$(OBJEXT)
-@@ -701,7 +692,7 @@ AM_V_at = $(am__v_at_@AM_V@)
+@@ -711,7 +702,7 @@ AM_V_at = $(am__v_at_@AM_V@)
  am__v_at_ = $(am__v_at_@AM_DEFAULT_V@)
  am__v_at_0 = @
  am__v_at_1 =
@@ -66,7 +66,7 @@ Index: src/Makefile.in
  depcomp = $(SHELL) $(top_srcdir)/depcomp
  am__depfiles_maybe = depfiles
  am__mv = mv -f
-@@ -1850,12 +1841,9 @@ GOOGLEMOCK_BUILD_DIR = ../third_party/googletest/googl
+@@ -1882,12 +1873,9 @@ GOOGLEMOCK_BUILD_DIR = ../third_party/googletest/googl
  GOOGLETEST_SRC_DIR = $(srcdir)/../third_party/googletest/googletest
  GOOGLEMOCK_SRC_DIR = $(srcdir)/../third_party/googletest/googlemock
  protobuf_test_LDADD = $(PTHREAD_LIBS) libprotobuf.la libprotoc.la \
@@ -81,7 +81,7 @@ Index: src/Makefile.in
 
  # Disable optimization for tests unless the user explicitly asked for it,
  # since test_util.cc takes forever to compile with optimization (with GCC).
-@@ -1944,12 +1932,9 @@ nodist_protobuf_test_SOURCES = $(protoc_outputs)
+@@ -1977,12 +1965,9 @@ nodist_protobuf_test_SOURCES = $(protoc_outputs)
  # Run cpp_unittest again with PROTOBUF_TEST_NO_DESCRIPTORS defined.
  protobuf_lazy_descriptor_test_LDADD = $(PTHREAD_LIBS) libprotobuf.la \
                        libprotoc.la                                   \
@@ -96,7 +96,7 @@ Index: src/Makefile.in
                                           -DPROTOBUF_TEST_NO_DESCRIPTORS
 
  protobuf_lazy_descriptor_test_CXXFLAGS = $(NO_OPT_CXXFLAGS)
-@@ -1972,12 +1957,9 @@ COMMON_LITE_TEST_SOURCES = \
+@@ -2005,12 +1990,9 @@ COMMON_LITE_TEST_SOURCES = \
  # full runtime and we want to make sure this test builds without full
  # runtime.
  protobuf_lite_test_LDADD = $(PTHREAD_LIBS) libprotobuf-lite.la     \
@@ -111,7 +111,7 @@ Index: src/Makefile.in
 
  protobuf_lite_test_CXXFLAGS = $(NO_OPT_CXXFLAGS)
  protobuf_lite_test_SOURCES = \
-@@ -1990,12 +1972,9 @@ nodist_protobuf_lite_test_SOURCES = $(protoc_lite_outp
+@@ -2023,12 +2005,9 @@ nodist_protobuf_lite_test_SOURCES = $(protoc_lite_outp
  # gtest when building the test internally our memory sanitizer doesn't detect
  # memory leaks (don't know why).
  protobuf_lite_arena_test_LDADD = $(PTHREAD_LIBS) libprotobuf-lite.la \
@@ -126,7 +126,7 @@ Index: src/Makefile.in
 
  protobuf_lite_arena_test_CXXFLAGS = $(NO_OPT_CXXFLAGS)
  protobuf_lite_arena_test_SOURCES = \
-@@ -2006,9 +1985,9 @@ nodist_protobuf_lite_arena_test_SOURCES = $(protoc_lit
+@@ -2039,9 +2018,9 @@ nodist_protobuf_lite_arena_test_SOURCES = $(protoc_lit
 
  # Test plugin binary.
  test_plugin_LDADD = $(PTHREAD_LIBS) libprotobuf.la libprotoc.la \
Index: pkg/PLIST
===================================================================
RCS file: /cvs/ports/devel/protobuf/pkg/PLIST,v
retrieving revision 1.3
diff -u -p -r1.3 PLIST
--- pkg/PLIST 10 Jul 2018 10:47:42 -0000 1.3
+++ pkg/PLIST 20 Jul 2019 21:52:41 -0000
@@ -48,6 +48,7 @@ include/google/protobuf/dynamic_message.
 include/google/protobuf/empty.pb.h
 include/google/protobuf/empty.proto
 include/google/protobuf/extension_set.h
+include/google/protobuf/extension_set_inl.h
 include/google/protobuf/field_mask.pb.h
 include/google/protobuf/field_mask.proto
 include/google/protobuf/generated_enum_reflection.h
@@ -78,6 +79,10 @@ include/google/protobuf/message.h
 include/google/protobuf/message_lite.h
 include/google/protobuf/metadata.h
 include/google/protobuf/metadata_lite.h
+include/google/protobuf/parse_context.h
+include/google/protobuf/port.h
+include/google/protobuf/port_def.inc
+include/google/protobuf/port_undef.inc
 include/google/protobuf/reflection.h
 include/google/protobuf/reflection_ops.h
 include/google/protobuf/repeated_field.h
@@ -95,14 +100,15 @@ include/google/protobuf/stubs/fastmem.h
 include/google/protobuf/stubs/hash.h
 include/google/protobuf/stubs/logging.h
 include/google/protobuf/stubs/macros.h
+include/google/protobuf/stubs/map_util.h
 include/google/protobuf/stubs/mutex.h
 include/google/protobuf/stubs/once.h
 include/google/protobuf/stubs/platform_macros.h
 include/google/protobuf/stubs/port.h
-include/google/protobuf/stubs/singleton.h
 include/google/protobuf/stubs/status.h
 include/google/protobuf/stubs/stl_util.h
 include/google/protobuf/stubs/stringpiece.h
+include/google/protobuf/stubs/strutil.h
 include/google/protobuf/stubs/template_util.h
 include/google/protobuf/text_format.h
 include/google/protobuf/timestamp.pb.h
@@ -121,7 +127,6 @@ include/google/protobuf/util/type_resolv
 include/google/protobuf/util/type_resolver_util.h
 include/google/protobuf/wire_format.h
 include/google/protobuf/wire_format_lite.h
-include/google/protobuf/wire_format_lite_inl.h
 include/google/protobuf/wrappers.pb.h
 include/google/protobuf/wrappers.proto
 lib/libprotobuf-lite.a

Reply | Threaded
Open this post in threaded view
|

audio/mumble / protobuf 3.90 fallout

Stuart Henderson
In reply to this post by Jonathon Sisson-2
On 2019/07/20 06:15, Jonathon Sisson wrote:
>
> Revised patch below, with SHARED_LIBS bump as requested.  I've tested with mosh, which works as expected (protobuf-3.6.1 client to protobuf-3.9.0 server).
>
> I'll test more ports as free time allows.
>
> Thank you for the pointers, Stuart =)

Turns out this broke audio/mumble, could you take a look please?

https://github.com/mumble-voip/mumble/pull/3623

Reply | Threaded
Open this post in threaded view
|

Re: audio/mumble / protobuf 3.90 fallout

Stuart Henderson
On 2019/08/03 20:07, Stuart Henderson wrote:

> On 2019/07/20 06:15, Jonathon Sisson wrote:
> >
> > Revised patch below, with SHARED_LIBS bump as requested.  I've tested with mosh, which works as expected (protobuf-3.6.1 client to protobuf-3.9.0 server).
> >
> > I'll test more ports as free time allows.
> >
> > Thank you for the pointers, Stuart =)
>
> Turns out this broke audio/mumble, could you take a look please?
>
> https://github.com/mumble-voip/mumble/pull/3623
>

Done it myself.

Reply | Threaded
Open this post in threaded view
|

Re: [Update] protobuf to 3.9.0

Theo Buehler-3
In reply to this post by Stuart Henderson
> So here is the "weasel out" diff below ... I've also run "make update-patches"
> to update the line numbers.
>
> "make test" fails following the update though:
>
> =====
> c++ -DHAVE_CONFIG_H -I. -I/usr/obj/ports/protobuf-3.9.0/protobuf-3.9.0/src -I.. -I/usr/local/include  -I/usr/local/include  -pthread -DHAVE_PTHREAD=1 -DHAVE_ZLIB=1 -Wall -Wno-sign-compare -O2 -pipe -MT google/protobuf/protobuf_test-descriptor_unittest.o -MD -MP -MF google/protobuf/.deps/protobuf_test-descriptor_unittest.Tpo -c -o google/protobuf/protobuf_test-descriptor_unittest.o `test -f 'google/protobuf/descriptor_unittest.cc' || echo '/usr/obj/ports/protobuf-3.9.0/protobuf-3.9.0/src/'`google/protobuf/descriptor_unittest.cc
> /usr/obj/ports/protobuf-3.9.0/protobuf-3.9.0/src/google/protobuf/descriptor_unittest.cc:2972:26: error: C++ requires a type specifier for all declarations
> INSTANTIATE_TEST_SUITE_P(DatabaseSource, AllowUnknownDependenciesTest,
>                          ^

This is due to patches/patch-src_Makefile_in forcing the use of the
devel/gtest port instead of using the bundled gtest modules. Our port
ships a gtest.h that doesn't support INSTANTIATE_TEST_SUITE_P while the
bundled version has what looks like a compatibility #define.

If I remove that patch (which was carried forward since import) and
remove TEST_DEPENDS=devel/gtest, the tests compile and I obtain the
results below. The protobuf-test seems to hang indefinitely, so I
eventually lost patience and killed it.

PASS: protobuf-lite-test
PASS: protobuf-lazy-descriptor-test
PASS: google/protobuf/compiler/zip_output_unittest.sh
PASS: protobuf-lite-arena-test
PASS: google/protobuf/io/gzip_stream_unittest.sh
PASS: no-warning-test
FAIL: protobuf-test
============================================================================
Testsuite summary for Protocol Buffers 3.9.0
============================================================================
# TOTAL: 7
# PASS:  6
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

It seems plausible that the REVISION bump is not needed, but since I
haven't checked, I included it:


Index: Makefile
===================================================================
RCS file: /var/cvs/ports/devel/protobuf/Makefile,v
retrieving revision 1.25
diff -u -p -r1.25 Makefile
--- Makefile 31 Jul 2019 20:58:10 -0000 1.25
+++ Makefile 8 Aug 2019 21:16:13 -0000
@@ -5,6 +5,7 @@ COMMENT = c++ protocol buffers
 V = 3.9.0
 DISTNAME = protobuf-cpp-$V
 PKGNAME = protobuf-$V
+REVISION = 0
 
 WRKDIST = ${WRKDIR}/protobuf-${V}
 
@@ -22,8 +23,6 @@ PERMIT_PACKAGE = Yes
 WANTLIB += c m pthread ${COMPILER_LIBCXX} z
 
 MASTER_SITES = https://github.com/google/protobuf/releases/download/v$V/
-
-TEST_DEPENDS = devel/gtest
 
 COMPILER = base-clang ports-gcc
 
Index: patches/patch-src_Makefile_in
===================================================================
RCS file: patches/patch-src_Makefile_in
diff -N patches/patch-src_Makefile_in
--- patches/patch-src_Makefile_in 31 Jul 2019 20:58:10 -0000 1.6
+++ /dev/null 1 Jan 1970 00:00:00 -0000
@@ -1,140 +0,0 @@
-$OpenBSD: patch-src_Makefile_in,v 1.6 2019/07/31 20:58:10 sthen Exp $
-
-Index: src/Makefile.in
---- src/Makefile.in.orig
-+++ src/Makefile.in
-@@ -483,10 +483,7 @@ protobuf_lazy_descriptor_test_OBJECTS =  \
- $(am_protobuf_lazy_descriptor_test_OBJECTS) \
- $(nodist_protobuf_lazy_descriptor_test_OBJECTS)
- protobuf_lazy_descriptor_test_DEPENDENCIES = $(am__DEPENDENCIES_1) \
-- libprotobuf.la libprotoc.la \
-- $(GOOGLETEST_BUILD_DIR)/lib/libgtest.la \
-- $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock.la \
-- $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock_main.la
-+ libprotobuf.la libprotoc.la
- protobuf_lazy_descriptor_test_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CXX \
- $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CXXLD) \
- $(protobuf_lazy_descriptor_test_CXXFLAGS) $(CXXFLAGS) \
-@@ -506,9 +503,7 @@ protobuf_lite_arena_test_OBJECTS =  \
- $(am_protobuf_lite_arena_test_OBJECTS) \
- $(nodist_protobuf_lite_arena_test_OBJECTS)
- protobuf_lite_arena_test_DEPENDENCIES = $(am__DEPENDENCIES_1) \
-- libprotobuf-lite.la $(GOOGLETEST_BUILD_DIR)/lib/libgtest.la \
-- $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock.la \
-- $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock_main.la
-+ libprotobuf-lite.la
- protobuf_lite_arena_test_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CXX \
- $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CXXLD) \
- $(protobuf_lite_arena_test_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) \
-@@ -529,9 +524,7 @@ nodist_protobuf_lite_test_OBJECTS = $(am__objects_10)
- protobuf_lite_test_OBJECTS = $(am_protobuf_lite_test_OBJECTS) \
- $(nodist_protobuf_lite_test_OBJECTS)
- protobuf_lite_test_DEPENDENCIES = $(am__DEPENDENCIES_1) \
-- libprotobuf-lite.la $(GOOGLETEST_BUILD_DIR)/lib/libgtest.la \
-- $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock.la \
-- $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock_main.la
-+ libprotobuf-lite.la
- protobuf_lite_test_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CXX \
- $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CXXLD) \
- $(protobuf_lite_test_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) \
-@@ -670,9 +663,7 @@ nodist_protobuf_test_OBJECTS = $(am__objects_13)
- protobuf_test_OBJECTS = $(am_protobuf_test_OBJECTS) \
- $(nodist_protobuf_test_OBJECTS)
- protobuf_test_DEPENDENCIES = $(am__DEPENDENCIES_1) libprotobuf.la \
-- libprotoc.la $(GOOGLETEST_BUILD_DIR)/lib/libgtest.la \
-- $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock.la \
-- $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock_main.la
-+ libprotoc.la
- protobuf_test_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CXX \
- $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CXXLD) \
- $(protobuf_test_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) \
-@@ -686,7 +677,7 @@ am_test_plugin_OBJECTS = google/protobuf/compiler/test
- google/protobuf/compiler/test_plugin-test_plugin.$(OBJEXT)
- test_plugin_OBJECTS = $(am_test_plugin_OBJECTS)
- test_plugin_DEPENDENCIES = $(am__DEPENDENCIES_1) libprotobuf.la \
-- libprotoc.la $(GOOGLETEST_BUILD_DIR)/lib/libgtest.la
-+ libprotoc.la
- am__zcgunzip_SOURCES_DIST = google/protobuf/testing/zcgunzip.cc
- @HAVE_ZLIB_TRUE@am_zcgunzip_OBJECTS =  \
- @HAVE_ZLIB_TRUE@ google/protobuf/testing/zcgunzip.$(OBJEXT)
-@@ -711,7 +702,7 @@ AM_V_at = $(am__v_at_@AM_V@)
- am__v_at_ = $(am__v_at_@AM_DEFAULT_V@)
- am__v_at_0 = @
- am__v_at_1 =
--DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir)
-+DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir) -I${LOCALBASE}/include
- depcomp = $(SHELL) $(top_srcdir)/depcomp
- am__depfiles_maybe = depfiles
- am__mv = mv -f
-@@ -1882,12 +1873,9 @@ GOOGLEMOCK_BUILD_DIR = ../third_party/googletest/googl
- GOOGLETEST_SRC_DIR = $(srcdir)/../third_party/googletest/googletest
- GOOGLEMOCK_SRC_DIR = $(srcdir)/../third_party/googletest/googlemock
- protobuf_test_LDADD = $(PTHREAD_LIBS) libprotobuf.la libprotoc.la \
--                      $(GOOGLETEST_BUILD_DIR)/lib/libgtest.la     \
--                      $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock.la     \
--                      $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock_main.la
-+      -L${LOCALBASE}/lib -lgtest -lgmock -lgmock_main
-
--protobuf_test_CPPFLAGS = -I$(GOOGLETEST_SRC_DIR)/include \
--                         -I$(GOOGLEMOCK_SRC_DIR)/include
-+protobuf_test_CPPFLAGS = -I${LOCALBASE}/include
-
- # Disable optimization for tests unless the user explicitly asked for it,
- # since test_util.cc takes forever to compile with optimization (with GCC).
-@@ -1977,12 +1965,9 @@ nodist_protobuf_test_SOURCES = $(protoc_outputs)
- # Run cpp_unittest again with PROTOBUF_TEST_NO_DESCRIPTORS defined.
- protobuf_lazy_descriptor_test_LDADD = $(PTHREAD_LIBS) libprotobuf.la \
-                       libprotoc.la                                   \
--                      $(GOOGLETEST_BUILD_DIR)/lib/libgtest.la        \
--                      $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock.la        \
--                      $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock_main.la
-+      -L${LOCALBASE}/lib -lgtest -lgmock -lgmock_main
-
--protobuf_lazy_descriptor_test_CPPFLAGS = -I$(GOOGLEMOCK_SRC_DIR)/include \
--                                         -I$(GOOGLETEST_SRC_DIR)/include \
-+protobuf_lazy_descriptor_test_CPPFLAGS = -I${LOCALBASE}/include          \
-                                          -DPROTOBUF_TEST_NO_DESCRIPTORS
-
- protobuf_lazy_descriptor_test_CXXFLAGS = $(NO_OPT_CXXFLAGS)
-@@ -2005,12 +1990,9 @@ COMMON_LITE_TEST_SOURCES = \
- # full runtime and we want to make sure this test builds without full
- # runtime.
- protobuf_lite_test_LDADD = $(PTHREAD_LIBS) libprotobuf-lite.la     \
--                           $(GOOGLETEST_BUILD_DIR)/lib/libgtest.la \
--                           $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock.la \
--                           $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock_main.la
-+   -L${LOCALBASE}/lib -lgtest -lgmock -lgmock_main
-
--protobuf_lite_test_CPPFLAGS = -I$(GOOGLEMOCK_SRC_DIR)/include \
--                             -I$(GOOGLETEST_SRC_DIR)/include
-+protobuf_lite_test_CPPFLAGS = -I${LOCALBASE}/include
-
- protobuf_lite_test_CXXFLAGS = $(NO_OPT_CXXFLAGS)
- protobuf_lite_test_SOURCES = \
-@@ -2023,12 +2005,9 @@ nodist_protobuf_lite_test_SOURCES = $(protoc_lite_outp
- # gtest when building the test internally our memory sanitizer doesn't detect
- # memory leaks (don't know why).
- protobuf_lite_arena_test_LDADD = $(PTHREAD_LIBS) libprotobuf-lite.la \
--                      $(GOOGLETEST_BUILD_DIR)/lib/libgtest.la        \
--                      $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock.la        \
--                      $(GOOGLEMOCK_BUILD_DIR)/lib/libgmock_main.la
-+ -L${LOCALBASE}/lib -lgtest -lgmock -lgmock_main
-
--protobuf_lite_arena_test_CPPFLAGS = -I$(GOOGLEMOCK_SRC_DIR)/include  \
--                                    -I$(GOOGLETEST_SRC_DIR)/include
-+protobuf_lite_arena_test_CPPFLAGS = -I${LOCALBASE}/include
-
- protobuf_lite_arena_test_CXXFLAGS = $(NO_OPT_CXXFLAGS)
- protobuf_lite_arena_test_SOURCES = \
-@@ -2039,9 +2018,9 @@ nodist_protobuf_lite_arena_test_SOURCES = $(protoc_lit
-
- # Test plugin binary.
- test_plugin_LDADD = $(PTHREAD_LIBS) libprotobuf.la libprotoc.la \
--                    $(GOOGLETEST_BUILD_DIR)/lib/libgtest.la
-+    -L${LOCALBASE}/lib -lgtest -lgmock -lgmock_main
-
--test_plugin_CPPFLAGS = -I$(GOOGLETEST_SRC_DIR)/include
-+test_plugin_CPPFLAGS = -I${$LOCALBASE}/include
- test_plugin_SOURCES = \
-   google/protobuf/compiler/mock_code_generator.cc              \
-   google/protobuf/testing/file.cc                              \