[new] minio server/client

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[new] minio server/client

Landry Breuil-5
Hi,

here's a quick & easy port for https://minio.io/, an 'open source object
storage server with Amazon S3 compatible API.'. Two ports, the server
side and the cli, builds easily and works surprisingly fine.

start it with minio server /path/to/dir, it outputs the credz on stdout
then listens on all ifaces:

$minio server /usr/obj/ports
Endpoint:  http://10.0.3.254:9000  http://10.0.2.2:9000 http://127.0.0.1:9000
AccessKey: xxxxxxxxxxxx
SecretKey: yyyyyyyyyyyy

there's a web iface at localhost:9000 which allows you to browse files,
you connect with an access key and the secret token.

for the cli, i renamed it from mc to minioc to avoid conflicts.
https://github.com/minio/mc#add-a-cloud-storage-service
You can list files from your server once configured in the client, or
list the files in the test instance provided by minio and configured by
default, well do all kind of usual file operations:

$minioc ls play/test-bucket
[2017-12-08 21:25:08 CET]    87B test.txt

$minioc config host add myminio http://10.0.3.254:9000 xxxxxxxxx yyyyyyy
$minioc ls myminio/bouquet  
[2017-12-09 23:08:21 CET]   378B home.desktop

And on the server filesystem:
378 Dec  9 23:08 /usr/obj/ports/bouquet/home.desktop

$minioc find myminio -name *.desktop
myminio/bouquet/home.desktop

$minioc share download myminio/bouquet/home.desktop
URL: http://10.0.3.254:9000/bouquet/home.desktop
Expire: 7 days 0 hours 0 minutes 0 seconds
Share:
http://10.0.3.254:9000/bouquet/home.desktop?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=yyyyyyyyyyy%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20171209T222035Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Dead simple. Just works. minioc claims to be compatible with S3/GCS like
aws-cli and s3cmd but i have no accounts there to test.

Feedback on the ports welcome :) I havent done an rc script for the
server, since i wouldnt know what to share by default... and it can run
as whatever user. Does it even need a README ? joe user could set it
up..

Landry

minio.tgz (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [new] minio server/client

trondd-2
On Sat, December 9, 2017 5:23 pm, Landry Breuil
> Feedback on the ports welcome :) I havent done an rc script for the
> server, since i wouldnt know what to share by default... and it can run
> as whatever user. Does it even need a README ? joe user could set it
> up..
>
> Landry
>

Thanks.  I might have a need for this in a -stable prod environment, so
that's where I am actually testing.

I would like to see an rc script to run this persistently as a system
user.  Similar to net/syncthing which can also run as a user, but also as
a system daemon sharing out of /var/syncthing/.

A similar setup might work here.  My particular usecase for a tool like
this would be as a system service.  Though, the tool seems to be designed
for per-user usage.

Tim.

Reply | Threaded
Open this post in threaded view
|

Re: [new] minio server/client

Landry Breuil-5
On Tue, Dec 12, 2017 at 01:51:06PM -0500, trondd wrote:

> On Sat, December 9, 2017 5:23 pm, Landry Breuil
> > Feedback on the ports welcome :) I havent done an rc script for the
> > server, since i wouldnt know what to share by default... and it can run
> > as whatever user. Does it even need a README ? joe user could set it
> > up..
> >
> > Landry
> >
>
> Thanks.  I might have a need for this in a -stable prod environment, so
> that's where I am actually testing.
>
> I would like to see an rc script to run this persistently as a system
> user.  Similar to net/syncthing which can also run as a user, but also as
> a system daemon sharing out of /var/syncthing/.
>
> A similar setup might work here.  My particular usecase for a tool like
> this would be as a system service.  Though, the tool seems to be designed
> for per-user usage.
Myeah. Here's a version with an rc script, running by default as _minio and
exporting /var/minio/export, storing the config in
/var/minio/.minio/config.json.

Starting from this i dunno if it warrants a readme or a MESSAGE, you
just need to go fetch the access/secret keys in the config file after
first startup, and maybe bump maxfiles limits for minio (as advised on
https://github.com/minio/minio-service - the sysv init script bumps to
8k)

Landry

minio_2.tgz (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [new] minio server/client

trondd-2
On Wed, December 13, 2017 10:39 am, Landry Breuil wrote:

> On Tue, Dec 12, 2017 at 01:51:06PM -0500, trondd wrote:
>> On Sat, December 9, 2017 5:23 pm, Landry Breuil
>> > Feedback on the ports welcome :) I havent done an rc script for the
>> > server, since i wouldnt know what to share by default... and it can
>> run
>> > as whatever user. Does it even need a README ? joe user could set it
>> > up..
>> >
>> > Landry
>> >
>>
>> Thanks.  I might have a need for this in a -stable prod environment, so
>> that's where I am actually testing.
>>
>> I would like to see an rc script to run this persistently as a system
>> user.  Similar to net/syncthing which can also run as a user, but also
>> as
>> a system daemon sharing out of /var/syncthing/.
>>
>> A similar setup might work here.  My particular usecase for a tool like
>> this would be as a system service.  Though, the tool seems to be
>> designed
>> for per-user usage.
>
> Myeah. Here's a version with an rc script, running by default as _minio
> and
> exporting /var/minio/export, storing the config in
> /var/minio/.minio/config.json.
>
> Starting from this i dunno if it warrants a readme or a MESSAGE, you
> just need to go fetch the access/secret keys in the config file after
> first startup, and maybe bump maxfiles limits for minio (as advised on
> https://github.com/minio/minio-service - the sysv init script bumps to
> 8k)
>
> Landry
>

That looks good (as far as functionality).  Thanks for considering my
needs. :)

I was also able to test the client against AWS and was able to list,
upload and delete.

Tim.

Reply | Threaded
Open this post in threaded view
|

Re: [new] minio server/client

Klemens Nanni
In reply to this post by Landry Breuil-5
On Wed, Dec 13, 2017 at 04:39:57PM +0100, Landry Breuil wrote:
> Starting from this i dunno if it warrants a readme or a MESSAGE, you
> just need to go fetch the access/secret keys in the config file after
> first startup, and maybe bump maxfiles limits for minio (as advised on
> https://github.com/minio/minio-service - the sysv init script bumps to
> 8k)
Users should know how to use minio, I'd say it's fine without
further explanation.

        $ portcheck
        Makefile.inc does not have $OpenBSD$ RCS tag at the top
        hardcoded paths detected in server/pkg/minio.rc, consider \
        using SUBST_VARS and TRUEPREFIX/LOCALBASE/LOCALSTATEDIR/VARBASE
        net/minio

s,/var,${VARBASE},

Reply | Threaded
Open this post in threaded view
|

Re: [new] minio server/client

Landry Breuil-5
On Thu, Dec 14, 2017 at 09:17:11PM +0100, Klemens Nanni wrote:

> On Wed, Dec 13, 2017 at 04:39:57PM +0100, Landry Breuil wrote:
> > Starting from this i dunno if it warrants a readme or a MESSAGE, you
> > just need to go fetch the access/secret keys in the config file after
> > first startup, and maybe bump maxfiles limits for minio (as advised on
> > https://github.com/minio/minio-service - the sysv init script bumps to
> > 8k)
> Users should know how to use minio, I'd say it's fine without
> further explanation.
>
> $ portcheck
> Makefile.inc does not have $OpenBSD$ RCS tag at the top
> hardcoded paths detected in server/pkg/minio.rc, consider \
> using SUBST_VARS and TRUEPREFIX/LOCALBASE/LOCALSTATEDIR/VARBASE
> net/minio
>
> s,/var,${VARBASE},
With those fixed.

minio_3.tgz (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [new] minio server/client

Landry Breuil-5
On Mon, Dec 18, 2017 at 08:23:34PM +0100, Landry Breuil wrote:

> On Thu, Dec 14, 2017 at 09:17:11PM +0100, Klemens Nanni wrote:
> > On Wed, Dec 13, 2017 at 04:39:57PM +0100, Landry Breuil wrote:
> > > Starting from this i dunno if it warrants a readme or a MESSAGE, you
> > > just need to go fetch the access/secret keys in the config file after
> > > first startup, and maybe bump maxfiles limits for minio (as advised on
> > > https://github.com/minio/minio-service - the sysv init script bumps to
> > > 8k)
> > Users should know how to use minio, I'd say it's fine without
> > further explanation.
> >
> > $ portcheck
> > Makefile.inc does not have $OpenBSD$ RCS tag at the top
> > hardcoded paths detected in server/pkg/minio.rc, consider \
> > using SUBST_VARS and TRUEPREFIX/LOCALBASE/LOCALSTATEDIR/VARBASE
> > net/minio
> >
> > s,/var,${VARBASE},
>
> With those fixed.
Now with mc updated to 20171212
(cf https://github.com/minio/mc/releases/tag/RELEASE.2017-12-12T01-08-02Z) and the single patch in an upstream PR
at https://github.com/minio/mc/pull/2342

Landry

minio_4.tgz (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [new] minio server/client

Klemens Nanni
On Sat, Dec 23, 2017 at 01:20:05PM +0100, Landry Breuil wrote:

> On Mon, Dec 18, 2017 at 08:23:34PM +0100, Landry Breuil wrote:
> > On Thu, Dec 14, 2017 at 09:17:11PM +0100, Klemens Nanni wrote:
> > > On Wed, Dec 13, 2017 at 04:39:57PM +0100, Landry Breuil wrote:
> > > > Starting from this i dunno if it warrants a readme or a MESSAGE, you
> > > > just need to go fetch the access/secret keys in the config file after
> > > > first startup, and maybe bump maxfiles limits for minio (as advised on
> > > > https://github.com/minio/minio-service - the sysv init script bumps to
> > > > 8k)
> > > Users should know how to use minio, I'd say it's fine without
> > > further explanation.
> > >
> > > $ portcheck
> > > Makefile.inc does not have $OpenBSD$ RCS tag at the top
> > > hardcoded paths detected in server/pkg/minio.rc, consider \
> > > using SUBST_VARS and TRUEPREFIX/LOCALBASE/LOCALSTATEDIR/VARBASE
> > > net/minio
> > >
> > > s,/var,${VARBASE},
> >
> > With those fixed.
>
> Now with mc updated to 20171212
> (cf https://github.com/minio/mc/releases/tag/RELEASE.2017-12-12T01-08-02Z) and the single patch in an upstream PR
> at https://github.com/minio/mc/pull/2342
Looks good to me. I'd probably sync usage of blank lines in all
Makefiles but that's personal taste.

GH_PROJECT is set in both {client/server}/Makefile so this will do:

--- Makefile.inc.orig
+++ Makefile.inc
@@ -1,7 +1,6 @@
 # $OpenBSD$
 
-GH_ACCOUNT ?= minio
-GH_PROJECT ?= minio
+GH_ACCOUNT = minio
 
 CATEGORIES = net sysutils

Reply | Threaded
Open this post in threaded view
|

Re: [new] minio server/client

Daniel Jakots-3
In reply to this post by Landry Breuil-5
On Sat, 23 Dec 2017 13:20:05 +0100, Landry Breuil <[hidden email]>
wrote:

> On Mon, Dec 18, 2017 at 08:23:34PM +0100, Landry Breuil wrote:
> > On Thu, Dec 14, 2017 at 09:17:11PM +0100, Klemens Nanni wrote:  
> > > On Wed, Dec 13, 2017 at 04:39:57PM +0100, Landry Breuil wrote:  
> > > > Starting from this i dunno if it warrants a readme or a
> > > > MESSAGE, you just need to go fetch the access/secret keys in
> > > > the config file after first startup, and maybe bump maxfiles
> > > > limits for minio (as advised on
> > > > https://github.com/minio/minio-service - the sysv init script
> > > > bumps to 8k)  
> > > Users should know how to use minio, I'd say it's fine without
> > > further explanation.
> > >
> > > $ portcheck
> > > Makefile.inc does not have $OpenBSD$ RCS tag at the top
> > > hardcoded paths detected in server/pkg/minio.rc, consider
> > > \ using SUBST_VARS and TRUEPREFIX/LOCALBASE/LOCALSTATEDIR/VARBASE
> > > net/minio
> > >
> > > s,/var,${VARBASE},  
> >
> > With those fixed.  
>
> Now with mc updated to 20171212
> (cf
> https://github.com/minio/mc/releases/tag/RELEASE.2017-12-12T01-08-02Z)
> and the single patch in an upstream PR at
> https://github.com/minio/mc/pull/2342

Can you register the uid 796 in user.list asap to be sure there won't
be another race please?

As stated in a previous email for yours:

> Myeah. Here's a version with an rc script, running by default as
> _minio and exporting /var/minio/export, storing the config in
> /var/minio/.minio/config.json.
>
> Starting from this i dunno if it warrants a readme or a MESSAGE, you
> just need to go fetch the access/secret keys in the config file after
> first startup, and maybe bump maxfiles limits for minio (as advised on
> https://github.com/minio/minio-service - the sysv init script bumps to
> 8k)

I think a readme would be really worth it.

Here's the diff from my review (with Klemens's remark).

diff -pruN /tmp/hometmp/minio/Makefile minio/Makefile
--- /tmp/hometmp/minio/Makefile Sat Dec  9 17:04:03 2017
+++ minio/Makefile Sat Dec 23 14:17:16 2017
@@ -1,6 +1,7 @@
 # $OpenBSD$
 
-SUBDIR = client
+SUBDIR =
+SUBDIR += client
 SUBDIR += server
 
 .include <bsd.port.subdir.mk>
diff -pruN /tmp/hometmp/minio/Makefile.inc minio/Makefile.inc
--- /tmp/hometmp/minio/Makefile.inc Thu Dec 14 16:06:31 2017
+++ minio/Makefile.inc Sat Dec 23 14:26:39 2017
@@ -1,7 +1,6 @@
 # $OpenBSD$
 
 GH_ACCOUNT ?= minio
-GH_PROJECT ?= minio
 
 CATEGORIES = net sysutils
 
@@ -9,5 +8,7 @@ HOMEPAGE = https://minio.io
 
 # APACHEv2
 PERMIT_PACKAGE_CDROM = Yes
+
+WANTLIB += c pthread
 
 MODULES = lang/go
diff -pruN /tmp/hometmp/minio/client/pkg/DESCR minio/client/pkg/DESCR
--- /tmp/hometmp/minio/client/pkg/DESCR Sat Dec  9 16:54:49 2017
+++ minio/client/pkg/DESCR Sat Dec 23 14:18:42 2017
@@ -1,3 +1,3 @@
-Minio Client (mc) provides a modern alternative to UNIX commands like
+Minio Client (minioc) provides a modern alternative to UNIX commands like
 ls, cat, cp, mirror, diff, find etc. It supports filesystems and Amazon
 S3 compatible cloud storage service (AWS Signature v2 and v4).
diff -pruN /tmp/hometmp/minio/server/pkg/PLIST minio/server/pkg/PLIST
--- /tmp/hometmp/minio/server/pkg/PLIST Wed Dec 13 10:27:33 2017
+++ minio/server/pkg/PLIST Sat Dec 23 14:20:17 2017
@@ -1,6 +1,6 @@
 @comment $OpenBSD$
-@newgroup _minio:795
-@newuser _minio:795:_minio:daemon:Minio user:${VARBASE}/minio:/sbin/nologin
+@newgroup _minio:796
+@newuser _minio:796:_minio:daemon:Minio user:${VARBASE}/minio:/sbin/nologin
 @bin bin/minio
 @rcscript ${RCDIR}/minio
 @owner _minio


Once you show me the readme, I'll give you the long awaited ok ;)

Reply | Threaded
Open this post in threaded view
|

Re: [new] minio server/client

Landry Breuil-5
On Sat, Dec 23, 2017 at 02:37:04PM -0500, Daniel Jakots wrote:

> On Sat, 23 Dec 2017 13:20:05 +0100, Landry Breuil <[hidden email]>
> wrote:
>
> > On Mon, Dec 18, 2017 at 08:23:34PM +0100, Landry Breuil wrote:
> > > On Thu, Dec 14, 2017 at 09:17:11PM +0100, Klemens Nanni wrote:  
> > > > On Wed, Dec 13, 2017 at 04:39:57PM +0100, Landry Breuil wrote:  
> > > > > Starting from this i dunno if it warrants a readme or a
> > > > > MESSAGE, you just need to go fetch the access/secret keys in
> > > > > the config file after first startup, and maybe bump maxfiles
> > > > > limits for minio (as advised on
> > > > > https://github.com/minio/minio-service - the sysv init script
> > > > > bumps to 8k)  
> > > > Users should know how to use minio, I'd say it's fine without
> > > > further explanation.
> > > >
> > > > $ portcheck
> > > > Makefile.inc does not have $OpenBSD$ RCS tag at the top
> > > > hardcoded paths detected in server/pkg/minio.rc, consider
> > > > \ using SUBST_VARS and TRUEPREFIX/LOCALBASE/LOCALSTATEDIR/VARBASE
> > > > net/minio
> > > >
> > > > s,/var,${VARBASE},  
> > >
> > > With those fixed.  
> >
> > Now with mc updated to 20171212
> > (cf
> > https://github.com/minio/mc/releases/tag/RELEASE.2017-12-12T01-08-02Z)
> > and the single patch in an upstream PR at
> > https://github.com/minio/mc/pull/2342
>
Ya ya ya ya.

minio_5.tgz (3K) Download Attachment