Mux Blog

I could go pretty deep here, so let me know if I should elaborate on anything.

The backend is a Ruby on Rails application that serves the frontend app's API. This interfaces with the user tables, database, and handles all the "state" of the app.

The serverless stuff has changed over the months, but primarily it handles the stuff I don't want Rails to handle: file uploads, video processing and transcription.

First, huge props to the Mux (https://mux.com) team and product. I can not express how easy it has been to build video (and audio) products. File uploads are handled to AWS/GCP (depending on a few things) and then trigger a serverless callback to Mux.com. Mux was the fastest way we found to turn an arbitrary video file (mp4/mov/etc) into HLS format for quick streaming.

Then once the video is uploaded, we have another serverless callback that sends the video for transcription using Assembly AI (https://assemblyai.com). There are a ton of transcription based services and they vary dramatically in quality, based on the media content. I believe Google/Amazon services were largely built around the need to process phone calls, so unless you may for their "enhanced" models, the quality is surprisingly bad (and surprisingly slow).

I *highly highly* recommend Mux and Assembly AI if you are doing anything video/transcription based work.

To get an immediate update to the end user, we actually process two transcript requests - one that is just the first 60 seconds, and then the…

Video is the sort of thing that benefits significantly from economies of scale. Encoding and bandwidth costs are your two biggest factors. Encoding has to be done once -- well, once per rendition you ship, more on that in a second -- but bandwidth costs are, modulo volume discounts, very linear, and thus a direct per-viewer-minute line item. Those bandwidth costs typically depend on your desired quality of video (bitrate) and capacity of service. CDNs and traditional cloud services tend to have expensive outbound bandwidth. VPSes etc. don't, but it's not fast enough to serve many users, and what happens when you hit transfer limits is Not Great.

It is pretty rare in 2021 to serve video just as "video files". HLS and DASH allow for adaptive quality streaming for bandwidth- or performance-constrained devices -- which can also help keep costs down, if you're shipping lower bitrate 480p renditions to some clients rather than full-boat 1080p -- and make resumption of partially played media easier. (Because you can append segments to an HLS or DASH manifest, you can also use this technology more easily for live-streamed content.)

This stuff is complicated. You can build out the pipelines to do this stuff yourself with either cloud services or a lot of duct tape -- I've done it before -- but it's A Lot Of Work to get right and economies of scale are a real thing. There exist services like Mux Video (YC W16, https://mux.com -- this is a shameless plug; I work on the devex team)…

Mux (https://mux.com) | San Francisco | Front-End Developer | Full-time | Onsite

Mux is building the future of online video infrastructure. Our first product is analytics and monitoring - think “NewRelic for Video”. We’re solving a real need in a fast-growing industry, and customers love our product: we’re monitoring hundreds of millions of video streams for companies like PBS, Funny or Die, IGN, and Wistia, and we’re just getting started.

As a front-end engineer at Mux, you will build and expand our React/Redux front-end application, our Elixir (Phoenix) APIs, and our JavaScript video player SDKs. We’re looking for a strong engineer with solid fundamentals, who can work quickly on a small team to deliver new features and products to our customers.

You’ll be joining an amazing team from places like Google, Facebook, Twitch, Brightcove, and Bain. Our founders previously started (and sold) Zencoder, an early leader in cloud video technology. We also authored Video.js, the biggest HTML5 video player on the web. We were named one of the best Y Combinator startups of 2016, and are funded by SV Angel, Lowercase (Chris Sacca), the founders of Heroku, Parse, and other top-tier investors.

https://muxhq.workable.com/jobs/398415

Exactly, reliably streaming video is much more complicated and expensive than given credit for. YouTube being $0 does not actually mean it's free, it just means the creator/publisher isn't paying for it directly in money, but in skimmed traffic. The viewer pays by exchanging personal data on what they watch and are interested in, as well as their time in watching commercials.

All of these costs are much higher than anyone actually estimates as people incorrectly valued them while YouTube arbitrages the real value in this.

We created an alternative to YouTube for those who have discovered the real value of these costs at Swarmify[1], but there are other options as well depending on needs/wants.[2][3]

[1] https://swarmify.com/ - No code/low code easy YouTube migration option

[2] https://mux.com/ - API based option

[3] https://vdocipher.com/ - Enterprise DRM option

Most of the time, we work on processing video data after it's been hosted somewhere for us to seek by frame or download. We don't yet support video streams but have heard multiple people bring it up.

Our infrastructure is really good at filtering for parts of video where things are happening ("interesting") and being able to parallelize video processing by cutting it up into multiple pieces, running more granular models on those parts, and smoothly interpolating metadata using surrounding frames. You should check out the processing section of our FAQ page (linked in original comment) for a more detailed explanation of how our processing works.

If you're worried about just streaming seamlessly, Mux has a great solution:
https://mux.com/live.

Also, for detecting things in e-sports feeds, feel free to reach out to the email in my bio!

I would like to know what the costs come out to per minute of video encoded and how many outputs they're getting in order to compare this to something like Media Convert (AWS), Google Transcode or something dedicated like Mux.

For reference Google Transcoder is about $0.13 per minute of encoding (at four resolutions). Mux is at $0.032 / min, AWS Media Convert $0.0188.

I should note I know Mux's pricing well, use them a lot, happily. It gets a bit confusing with Google and Media Convert because I'm not sure how these costs map to the resulting bitrate renditions that get created and I've not got the time for a deeper dive to get a more straight apples to apples comparison (ignore scale discounts)

[1] https://cloud.google.com/transcoder/pricing

[2] https://www.mux.com/pricing/video

[3] https://aws.amazon.com/mediaconvert/pricing/