
Also more to your question, "The Browser Hacker's Handbook" and "The Web Application Hacker's Handbook" I also think are really helpful in understanding how browsers work. Sadly a not better documented subject.
Visiting the list of APIs on Mozilla's website is also helpful.
In particular, I really like this:
"When you submit bugs, remember that you aren't actually entitled to anything. Unfortunately, that's how bug bounties work. It's a sellers market. If a program doesn't pay as much as you'd expect for a bug, just don't participate in that program again. What's the point of causing drama over a bug or two? Who is the magic internet man who's going to buy your exploit for $1,000,000 using magic internet money (bitcoin) that those Hacker News users keep on referencing? If anyone knows who this person is, do tell me! There's no such thing as a "union" for bug bounty hunters nor an easily accessible secondary blackmarket that pays for your bugs in a company."
Fascinating. It's almost as though a successful security researcher and bug bounty participant is saying the same things Tom and I keep saying here, despite a constant hoard of people outside the industry who believe web application bugs are worth anything on the black market.
For anyone who would like to replicate this success, know the following:
1. The Web Application Hacker's Handbook is your friend. It's outdated but it's still the best foundation. Follow it up with The Browser Hacker's Handbook or The Mobile Application Hacker's Handbook.
2. You can optimize for exotic vulnerabilities in Google, Facebook et al which pay the highest bounties or you can optimize for low hanging fruit in companies which are just opening up a bounty program. The longer a company has a bounty program, the less of a chance you'll successfully XSS them in a day of assessing their web applications. Optimizing for finding security vulnerabilities serially in the highest paying bounty programs requires a much greater level of skill, vulnerability intuition ("what did the developers not think about or consider here?") and patience.
3. To do this seriously, learn to recognize vulnerabilities as contextual faults in the the way a particular software's implementation matches the developer's design expectations. Try not to think of vulnerabilities as classifications, like XSS, CSRF, IDOR, etc. For a first pass to find low hanging fruit it's helpful to proceed this way, but for more advanced work you want to look holistically.
4. If you want to specialize in application security at the binary level (sandbox escapes, memory corruption - vulnerabilities in operating systems and major OS applications), you'll want to learn reverse engineering. Start with Hacking: The Art of Exploitation and then proceed to Reversing: Secrets of Reverse Engineering or Practical Reverse Engineering. You'll also want to read The Art of Software Security Assessment and Security Engineering alongside those, to develop excellent source code auditing and binary penetration testing skills. Frankly you should read those no matter what you do in security.
5. It's very possible to earn a competitive tech salary just by finding vulnerabilities and reporting them (or selling them, if that's your bag, but, critically, you won't be selling website bugs). However, it's difficult and time consuming. It requires an orthogonal skill set to straight development, and while it might not be more time consuming it will certainly appear that way while you're doing it. Like much of security work, bug bounty hunting consists of long periods of research and analysis, during which you might feel like you haven't made any progress. This is interposed with brief moments of inspiration and a final moment of euphoria when you finally break something after poking at it for days or weeks or months.
You can earn a lot by finding lots of bug bounties, or you can do it by finding a few serious flaws in widely used "foundational" software in a year (think browsers, Flash, Rails, Windows, etc.). The latter method requires less context-switching, so if you can swing for that it's the way I'd recommend. All you need is three or four bugs in an OS or browser to match a Bay Area median salary.
As always, I'm very happy to help anyone who would like to get into security professionally.
You can look at the Web Application Hacker's Handbook or the Browser Hacker's Handbook, if you want. But TTW tops them all.