AssemblyAI, a speech-to-text API company, publishes explainers on machine learning, generative AI, speech recognition, and model frameworks.
HN commenters mostly recommend the company's speech API as good, cheap, and easy to use; one doubts its blog is the best way to learn ML.
Great question!
There are three main parts to note: designing the videos, the backend for handling media, and the download renderer.
The video designing process is a React app that leverages a Ruby on Rails based backend API. The React app handles the views, and if you look at the UI, you will see how the app steps are persisted in the URL. Our React app is based on Redux Toolkit, which is phenomenal. The Rails application is a normal API with Sidekiq (aka Redis) workers, which handle asynchronous tasks.
Our UI renders a number of interesting elements which are respectively generated/prepared in various service oriented streams. The most important is our transcript API, which is from https://www.AssemblyAI.com - they are the best, cheapest, highest quality and best development experience transcription tool. We also have a series of Lambda functions that handle uploaded audio/video file prep, such that we can encode files in a unified format and parse out the audio data needed for visualizing things like the audio waveform or the animated audio frequency data.
A few interesting tidbits are that we use Lambda Layers extensively. We have functions written Ruby and JavaScript where we move the vendor Gems or node_modules into a shared Lambda Layer, and then we also use EFS to run Python based functions that have dependencies that are too big for the Lambda itself.
Our video rendering is also pretty neat, in that we leverage the browser as a rendering view, and batch process…
AssemblyAI (YC S17) | Senior Backend Engineer | Full-time | Remote
We're building the Top Rated Speech-to-Text API for Developers.
Customers use our API to transcribe phone calls, meetings, videos, podcasts, and other types of media. Our accurate transcripts are used to power features like visual voicemail, call analytics, closed captioning, meeting summaries, and a slew of other features. We're growing fast, and processing millions of API calls per day in production.
We're a fully remote company and our development team is currently spread out across the US.
We're looking for a senior backend engineer with strong software and cloud engineering skills. You'll help us build highly scalable developer-centric APIs powered by state-of-the-art ML models.
Job description: https://www.notion.so/Senior-Backend-Engineer-187f72231e4946f8a5b37e22ca1d6fed
Website: https://www.assemblyai.com
You can reach me directly at: mgroves@assemblyai.com (Please put "HN" in your emails)
Things are indeed moving really fast now, and that's really exciting, but I'd be cautious about extrapolating that pace too far into the future. Nearly all of the recent image generating AIs are all based on a single breakthrough technique called image diffusion[1]. Once we start hitting the limits of diffusion models progress will likely start to plateau a bit, at least until the next major breakthrough.
[1]: https://www.assemblyai.com/blog/diffusion-models-for-machine-learning-introduction/
I don't know if this is the best resource for learning AI/ML. I'd sooner recommend Fast.AI, Caltech's open courses, etc.[1]
Also, my understanding is that TensorFlow is becoming less and less popular, so you might want to focus on PyTorch. Curious if others agree?
[1] See: https://phaseai.com/resources/free-resources-ai-ml-2024
[2] For example, scroll down to see the chart here: https://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-2023/
The thing I find most interesting about LLMs is the emergent skills that happen almost instantly(based on model size) that weren't inherently trained for.
https://www.assemblyai.com/blog/emergent-abilities-of-large-language-models/#:~:text=These%20emergent%20abilities%20include%20performing,simply%20by%20observing%20natural%20language.&text=As%20Language%20Models%20grow%2C%20novel%20abilities%20suddenly%20appear%20%5B8%5D.
They have their own ASR Conformer-2[0] and support 9 languages (they count it as 12)[1]
It looks like their synchronous transcribe is much slower than whisper, but if you need it fast, you need their realtime ASR (or amazon or google's).
[0] Conformer-2 is trained on 1.1M hours of English https://www.assemblyai.com/blog/conformer-2/
[1] https://www.assemblyai.com/docs/Concepts/supported_languages