videos in httpd

classic Classic list List threaded Threaded
9 messages Options
jsg
Reply | Threaded
Open this post in threaded view
|

videos in httpd

jsg
  Hi folks
   For those of you running http in support of your business, are any of you providing
videos for your customers ?
  If so what packages and set-up are you using?
  Any advice guidance appreciated.

  As usual
  Thanks in Advance

Reply | Threaded
Open this post in threaded view
|

Re: videos in httpd

temp+101
>  Hi folks
>   For those of you running http in support of your business, are any of you providing
>videos for your customers ?
>  If so what packages and set-up are you using?
>  Any advice guidance appreciated.

Hi,

I used httpd for serving file in personal usages. I found that httpd is poor for large
files and cause consuming resources for that purpose.

I recommend using another http server, like nginx (I don't test that) or write a cgi
(fcgi) for that purpose.

Reply | Threaded
Open this post in threaded view
|

Re: videos in httpd

Kevin Chadwick-4
> >  Hi folks
> >   For those of you running http in support of your business, are any of you providing
> >videos for your customers ?
> >  If so what packages and set-up are you using?
> >  Any advice guidance appreciated.  
>
> Hi,
>
> I used httpd for serving file in personal usages. I found that httpd is poor for large
> files and cause consuming resources for that purpose.
>
> I recommend using another http server, like nginx (I don't test that) or write a cgi
> (fcgi) for that purpose.


Really? Is this unstated resource consumption saturated before your
bandwidth is?

If you expect your users to actually watch your whole video then
streaming isn't really required to save bandwidth and so you can avoid
javascript with a simple html5 <video> element and just httpd is
required.

Otherwise why not host it on youtube and just add the link to your site?

--

KISSIS - Keep It Simple So It's Securable

Reply | Threaded
Open this post in threaded view
|

Re: videos in httpd

lists-3
In reply to this post by temp+101
On 6/22/2016 6:51 AM, [hidden email] wrote:
> I used httpd for serving file in personal usages. I found that httpd is poor for large
> files and cause consuming resources for that purpose.
>
> I recommend using another http server, like nginx (I don't test that) or write a cgi
> (fcgi) for that purpose.

I found that adding the file extensions to mime types into httpd.conf
solved my problem and allowed streaming of video and music files.

I was serving lots of large files and they would eat up all my ram and
crashing httpd. I compared nginx's mime types to httpd and added in the
ones for my large files and have never had another issue.

Reply | Threaded
Open this post in threaded view
|

Re: videos in httpd

Nick Holland
In reply to this post by temp+101
On 06/22/16 07:50, [hidden email] wrote:

>>  Hi folks
>>   For those of you running http in support of your business, are any of you providing
>>videos for your customers ?
>>  If so what packages and set-up are you using?
>>  Any advice guidance appreciated.
>
> Hi,
>
> I used httpd for serving file in personal usages. I found that httpd is poor for large
> files and cause consuming resources for that purpose.

This was a somewhat subtle, but interesting problem...and it is now
fixed (in -current, and will be in 6.0).

The problem was, as I recall, not an issue of "large" files itself, but
raised its head mostly on large files.  The problem was when someone
would start a file download and stop it prematurely for any reason, file
descriptors (hope I'm using the right word properly here) were not being
released.  Well, it turns out it is rather difficult to abort the
transfer of a 5k or 20k file...but a 300m file?  Easy.  And if running
something where lots of people download lots of files, seemingly
frequent...resulting in an unhappy system.

I can now say with some confidence, that I this problem has been
squished very effectively, and httpd (now) does a great job of running
at least a few OpenBSD mirrors.

Nick.

Reply | Threaded
Open this post in threaded view
|

Re: videos in httpd

temp+101
In reply to this post by jsg
[hidden email]:

>On 06/22/16 07:50, [hidden email] wrote:
>>>  Hi folks
>>>   For those of you running http in support of your business, are any of you providing
>>>videos for your customers ?
>>>  If so what packages and set-up are you using?
>>>  Any advice guidance appreciated.
>>
>> Hi,
>>
>> I used httpd for serving file in personal usages. I found that httpd is poor for large
>> files and cause consuming resources for that purpose.
>
>This was a somewhat subtle, but interesting problem...and it is now
>fixed (in -current, and will be in 6.0).
>
>The problem was, as I recall, not an issue of "large" files itself, but
>raised its head mostly on large files.  The problem was when someone
>would start a file download and stop it prematurely for any reason, file
>descriptors (hope I'm using the right word properly here) were not being
>released.  Well, it turns out it is rather difficult to abort the
>transfer of a 5k or 20k file...but a 300m file?  Easy.  And if running
>something where lots of people download lots of files, seemingly
>frequent...resulting in an unhappy system.
>
>I can now say with some confidence, that I this problem has been
>squished very effectively, and httpd (now) does a great job of running
>at least a few OpenBSD mirrors.
>
>Nick.
>
My server powers OpenBSD 5.9, and I forgot to mention that.
Yes, by my tests, I noticed that 300MB for each file is good for me,
then I split my 1GB file into 300MB pieces :D

Thanks

Reply | Threaded
Open this post in threaded view
|

Re: videos in httpd

john slee
In reply to this post by jsg
Hi,

Replying off-list because not an OpenBSD issue.

On 22 June 2016 at 01:49, jsg <[hidden email]> wrote:

>    For those of you running http in support of your business, are any of
> you providing
> videos for your customers ?
>   If so what packages and set-up are you using?
>   Any advice guidance appreciated.
>

Some background: I look after video transcoding and delivery for a
news-media organisation. We serve around 200-250TB of video content a
month, mostly short videos up to a couple of minutes long, sometimes much
longer. New videos every day, sometimes hundreds of them.

There's a lot to learn here but I'll make a few major points:

* there's two major types of video delivery (plus Adobe Flash, but we're in
2016 now!):
- Progressive: one big file per video bit-rate. Nice and simple, and works
on pretty much all non-Apple devices, including Chrome/Firefox
- Adaptive: multiple files per video bit-rate, with manifest files to help
devices find all the files. Used on Apple devices. Otherwise known as HLS,
or HTTP Live Streaming. Check browser support carefully. When I last tried
it, this HLS was not supported in Chrome/Firefox.

* some web platforms also only support one or the other. Eg. Facebook, as
far as I'm aware, only supports Progressive delivery.

* a webserver that supports Range headers/HTTP 206 responses is important
if you are serving Progressive videos that aren't very short; this can help
the user seek within your video

* don't forget to set Cache-Control headers

* if you expect to serve a lot of content, consider a CDN. We use Akamai,
but they are quite expensive and if you don't need all the fancy features,
not worth the money. Maybe Fastly, Amazon Cloudfront or even Varnish
Software's "DIY CDN" toolset?

* it's easy to provide users on desktop and iOS a consistent, pleasant user
experience with video

* two years ago it was pretty much impossible to provide users on Android a
consistent user experience with video. I don't know if this situation has
improved or not. Probably not :-(

For transcoding, the standard seems to be ffmpeg. I strongly suspect this
is what underpins most commercial transcoding platforms (eg. Brightcove's
Zencoder [which is what we use...], Amazon ElasticTranscoder, Akamai's
transcoding product, etc) and that what you pay for with the commercial
products is support and "glue" infrastructure. I don't have any references
to back this up, though. I do know that ffmpeg is what we used before we
(for unrelated reasons) switched over to Brightcove's platform, and at
least one other major media organisation here also uses ffmpeg. Probably
most of them do, really.

Put some serious effort into tracking what your users are viewing. Eg. with
the previous version of our video platform, we noticed that >98% of
Progressive video views were of the maximum bitrate, and ~95% for Adaptive.
So we were able to eliminate the renditions that almost nobody looked at,
and saved a lot of storage. It also tells us that maybe we could look at
offering our users higher bitrates.

I'd like to echo the other comment on-list: seriously consider YouTube.
It's free, it works everywhere, and doesn't need Flash. The only reason we
don't use it ourselves is that people in our organisation wanted (much)
more control of video advertising than YouTube offer.

Happy to help with anything else video, just ask :-)

John

Reply | Threaded
Open this post in threaded view
|

Re: videos in httpd

john slee
apologies, that was *supposed* to be off-list but I failed at mail :-/

John

On 23 June 2016 at 21:37, john slee <[hidden email]> wrote:

> Hi,
>
> Replying off-list because not an OpenBSD issue.
>
> On 22 June 2016 at 01:49, jsg <[hidden email]> wrote:
>
>>    For those of you running http in support of your business, are any of
>> you providing
>> videos for your customers ?
>>   If so what packages and set-up are you using?
>>   Any advice guidance appreciated.
>>
>
> Some background: I look after video transcoding and delivery for a
> news-media organisation. We serve around 200-250TB of video content a
> month, mostly short videos up to a couple of minutes long, sometimes much
> longer. New videos every day, sometimes hundreds of them.
>
> There's a lot to learn here but I'll make a few major points:
>
> * there's two major types of video delivery (plus Adobe Flash, but we're
> in 2016 now!):
> - Progressive: one big file per video bit-rate. Nice and simple, and works
> on pretty much all non-Apple devices, including Chrome/Firefox
> - Adaptive: multiple files per video bit-rate, with manifest files to help
> devices find all the files. Used on Apple devices. Otherwise known as HLS,
> or HTTP Live Streaming. Check browser support carefully. When I last tried
> it, this HLS was not supported in Chrome/Firefox.
>
> * some web platforms also only support one or the other. Eg. Facebook, as
> far as I'm aware, only supports Progressive delivery.
>
> * a webserver that supports Range headers/HTTP 206 responses is important
> if you are serving Progressive videos that aren't very short; this can help
> the user seek within your video
>
> * don't forget to set Cache-Control headers
>
> * if you expect to serve a lot of content, consider a CDN. We use Akamai,
> but they are quite expensive and if you don't need all the fancy features,
> not worth the money. Maybe Fastly, Amazon Cloudfront or even Varnish
> Software's "DIY CDN" toolset?
>
> * it's easy to provide users on desktop and iOS a consistent, pleasant
> user experience with video
>
> * two years ago it was pretty much impossible to provide users on Android
> a consistent user experience with video. I don't know if this situation has
> improved or not. Probably not :-(
>
> For transcoding, the standard seems to be ffmpeg. I strongly suspect this
> is what underpins most commercial transcoding platforms (eg. Brightcove's
> Zencoder [which is what we use...], Amazon ElasticTranscoder, Akamai's
> transcoding product, etc) and that what you pay for with the commercial
> products is support and "glue" infrastructure. I don't have any references
> to back this up, though. I do know that ffmpeg is what we used before we
> (for unrelated reasons) switched over to Brightcove's platform, and at
> least one other major media organisation here also uses ffmpeg. Probably
> most of them do, really.
>
> Put some serious effort into tracking what your users are viewing. Eg.
> with the previous version of our video platform, we noticed that >98% of
> Progressive video views were of the maximum bitrate, and ~95% for Adaptive.
> So we were able to eliminate the renditions that almost nobody looked at,
> and saved a lot of storage. It also tells us that maybe we could look at
> offering our users higher bitrates.
>
> I'd like to echo the other comment on-list: seriously consider YouTube.
> It's free, it works everywhere, and doesn't need Flash. The only reason we
> don't use it ourselves is that people in our organisation wanted (much)
> more control of video advertising than YouTube offer.
>
> Happy to help with anything else video, just ask :-)
>
> John

Reply | Threaded
Open this post in threaded view
|

Re: videos in httpd

bodie
In reply to this post by jsg
On 23.06.2016 13:39, john slee wrote:
> apologies, that was *supposed* to be off-list but I failed at mail
> :-/
>

In fact it was INTERESTING inside. Thx for that


> John
>
> On 23 June 2016 at 21:37, john slee <[hidden email]> wrote:
>
>> Hi,
>>
>> Replying off-list because not an OpenBSD issue.
>>
>> On 22 June 2016 at 01:49, jsg <[hidden email]> wrote:
>>
>>>    For those of you running http in support of your business, are
>>> any of
>>> you providing
>>> videos for your customers ?
>>>   If so what packages and set-up are you using?
>>>   Any advice guidance appreciated.
>>>
>>
>> Some background: I look after video transcoding and delivery for a
>> news-media organisation. We serve around 200-250TB of video content
>> a
>> month, mostly short videos up to a couple of minutes long, sometimes
>> much
>> longer. New videos every day, sometimes hundreds of them.
>>
>> There's a lot to learn here but I'll make a few major points:
>>
>> * there's two major types of video delivery (plus Adobe Flash, but
>> we're
>> in 2016 now!):
>> - Progressive: one big file per video bit-rate. Nice and simple, and
>> works
>> on pretty much all non-Apple devices, including Chrome/Firefox
>> - Adaptive: multiple files per video bit-rate, with manifest files
>> to help
>> devices find all the files. Used on Apple devices. Otherwise known
>> as HLS,
>> or HTTP Live Streaming. Check browser support carefully. When I last
>> tried
>> it, this HLS was not supported in Chrome/Firefox.
>>
>> * some web platforms also only support one or the other. Eg.
>> Facebook, as
>> far as I'm aware, only supports Progressive delivery.
>>
>> * a webserver that supports Range headers/HTTP 206 responses is
>> important
>> if you are serving Progressive videos that aren't very short; this
>> can help
>> the user seek within your video
>>
>> * don't forget to set Cache-Control headers
>>
>> * if you expect to serve a lot of content, consider a CDN. We use
>> Akamai,
>> but they are quite expensive and if you don't need all the fancy
>> features,
>> not worth the money. Maybe Fastly, Amazon Cloudfront or even Varnish
>> Software's "DIY CDN" toolset?
>>
>> * it's easy to provide users on desktop and iOS a consistent,
>> pleasant
>> user experience with video
>>
>> * two years ago it was pretty much impossible to provide users on
>> Android
>> a consistent user experience with video. I don't know if this
>> situation has
>> improved or not. Probably not :-(
>>
>> For transcoding, the standard seems to be ffmpeg. I strongly suspect
>> this
>> is what underpins most commercial transcoding platforms (eg.
>> Brightcove's
>> Zencoder [which is what we use...], Amazon ElasticTranscoder,
>> Akamai's
>> transcoding product, etc) and that what you pay for with the
>> commercial
>> products is support and "glue" infrastructure. I don't have any
>> references
>> to back this up, though. I do know that ffmpeg is what we used
>> before we
>> (for unrelated reasons) switched over to Brightcove's platform, and
>> at
>> least one other major media organisation here also uses ffmpeg.
>> Probably
>> most of them do, really.
>>
>> Put some serious effort into tracking what your users are viewing.
>> Eg.
>> with the previous version of our video platform, we noticed that
>> >98% of
>> Progressive video views were of the maximum bitrate, and ~95% for
>> Adaptive.
>> So we were able to eliminate the renditions that almost nobody
>> looked at,
>> and saved a lot of storage. It also tells us that maybe we could
>> look at
>> offering our users higher bitrates.
>>
>> I'd like to echo the other comment on-list: seriously consider
>> YouTube.
>> It's free, it works everywhere, and doesn't need Flash. The only
>> reason we
>> don't use it ourselves is that people in our organisation wanted
>> (much)
>> more control of video advertising than YouTube offer.
>>
>> Happy to help with anything else video, just ask :-)
>>
>> John