fast.ai

Thanks! I saw your earlier question related to working with AI at FAANG, from the context of being 30ish, and self-taught. I was around 27 when I started learning software engineering and AI.

I made a free e-book here (https://emilwallner.gumroad.com/l/no-ml-degree), and I believe most of the key points are still valid, however, when I learned software engineering and ML, ML was a rather small field and tools like chatGPT and claude didn’t exist.

Imo, asking about a curriculum is the wrong framing, for me, it was more about how to find resources to focus full-time and being in an environment that increases my motivation.

I started learning software engineering at home taking courses, but I procrastinated too much to be effective, maybe I did around 10 hours of effective learning per week. For me, studying C at 42 (https://www.42network.org/42-schools/), a free peer-to-peer school was crucial, and I recommend something similar. It enabled me to focus 70-90 hours a week, and after 6 months I was good enough to get competitive startup job offers.

During my time, the FastAI course (https://www.fast.ai/) was the best practical AI course. I'd probably spend a week looking for ambitious projects made by recent autodidacts, and ask them which course they think is best now. And spend max 1-2 months taking the course.

As for picking projects and building a portfolio, the advice in my e-book is still valid. An ambitious but realistic timeframe for landing a FAANG job is 3-5 years.…

That Tim Urban piece is great. It's also an interesting time capsule in terms of which AI problems were and were not considered hard in 2015 (when the post was written). From the post:

> Build a computer that can multiply two ten-digit numbers in a split second—incredibly easy. Build one that can look at a dog and answer whether it’s a dog or a cat—spectacularly difficult. Make AI that can beat any human in chess? Done. Make one that can read a paragraph from a six-year-old’s picture book and not just recognize the words but understand the meaning of them? Google is currently spending billions of dollars trying to do it. Hard things—like calculus, financial market strategy, and language translation—are mind-numbingly easy for a computer, while easy things—like vision, motion, movement, and perception—are insanely hard for it.

The children's picture book problem is solved; those billions of dollars were well-spent after all. (See, e.g., DeepMind's recent Flamingo model [1].) We can do whatever we want in vision, more or less [2]. Motion and movement might be the least developed area, but it's still made major progress; we have robotic parkour [3] and physical Rubik's cube solvers [4], and we can tell a robot to follow simple domestic instructions [5]. And Perceiver (again from DeepMind [6]) took a big chunk out of the perception problem.

Getting a computer to carry on a conversation [7], let alone draw art on par with human professionals [8], weren't even mentioned as…

We have open positions for putting LLMs to use (won't link, not a work account). Seems difficult to hire people with sufficient knowledge of our niche and of LLMs.

Whether they will continue to grow is hard to say. Putting open models to productive use is difficult. Like stable diffusion can make a pretty image but can it make the image you want? For example, can it make a 2d video game sprite, can it make the same character consistently in different poses.

That's a personal project problem I have but it is the same vibe as LLMs with code. Like they can produce some code, but a more complex algorithm or larger program is tough. I'm not sure if there will be an upper limit on what they can do. Right now it is a lot of engineering to get them to do stuff with no human in the loop.

Not a lot of companies are training models. There are probably a number that are trying to integrate OpenAI's API in some way.

They can do powerful things and it is a good skill to learn, but it feels like the market right now is for PhDs from top schools or low paid data labelers. Maybe others will have different experience.

For resources, check out [1], [2], and [3]. The third being my least favorite, but others like it.

1. https://karpathy.ai/zero-to-hero.html

2. https://deeplearning.ai

3. https://fast.ai

I have also started learning NLP this year. Here is what have followed.
If you are completely new to deeplearning I would recommend course from https://www.fast.ai. Since it will teach you the concepts in a more practical way and also the next courses will more theory than practical.

For theory on NLP with deeplearning you can follow Stanford course by Christopher Manning.

http://web.stanford.edu/class/cs224n/index.html#coursework

it will give you a good understanding of how deeplearning is used in a certain area of NLP. But remember deeplearning is one of the techniques for solving NLP problems if you are more interested in Understanding NLP then I would recommend the following book.

https://web.stanford.edu/~jurafsky/slp3/ed3book.pdf

Although, Stanford course should give you a great high-level understanding of how GPT models work but if you really need to go under the hood then I would suggest after finishing this course you should learn Unsupervised Learning in DeepLearning.

I haven't experienced any real issues with GPU availability on Colab, I suggest that you just go ahead and use it and wait with the premature optimization until you actually hit a wall and need it.

For general advice focused on beginners and ESPECIALLY practical, cheap and efficient methods and hacks to do DL, I recommend searching in https://www.fast.ai/
and their forums
https://forums.fast.ai/

I'll try to search inside fast.ai if there is a more specific link to give.
I know that one of their chief pieces of advice has been to use Colab and take advantage of the 300$ free credit you get (per credit card) when signing up to Google Cloud, which you can use for DL.

Disclaimer - I'm one of the creators of DagsHub, we created the platform especially to help people like you with the difficulties of managing things like data and model versioning, experiment tracking, labeling, etc. we'd love to have you onboard, and thanks for reading until the end :)

Ah, now that really is random! One idea for sifting parking pages that comes to my mind is using image recognition. I can easily tell when something is a parking page by looking at it, I bet you could get an ML model to do so with pretty good accuracy too.

There is a class called fast.ai [1] which I personally found to be an amazing introduction to machine learning / deep learning. The first hour and a half course gets you using their library to make an image classifier that can differentiate dog and cat breeds. I bet by simply gathering training data (sites that are and are not parking pages) and reusing the same architecture used in that first class you could train a model that does a pretty good job of it.

Once you could classify pages you could just pre-compute a large list, classify, throwaway all the parking pages, and make the remainder available to your user.

1 - https://www.fast.ai/