Lex Fridman Podcast

I'm in a similar position. Top x% in my field, more than fair compensation, somewhat financial security. I worked my butt off towards burnout to get there (currently slowly recovering).

I recently listened to a conversation between Lex Fridman and Stephen Wolfram [1]. Stephen said something that stuck with me for a while. Something along the lines:

GPTs are statistical word guessing machines. They regurgitate the average of the internet. You can get around that by placing with temperature and top-k, but that'll only make them less accurate, and increase the chances for them to hallucinate.

You, however, are an expert. Better than the average of the internet. Years of experience, and likely with higher standards. You understand the requirements, architect maintainable systems, and align with the company goals. You learned how to read requirements from context and between the lines.

Yes, GPTs can do highly complex things, and sometimes it feels magical or even scary. But you'll have to prime them to do so, and you'll have to guide them. They are your junior programmer, and you are their lead. You understand context, and they don't (simply because they don't have any).

Start to accept that from now on you have your personal junior dev. Learn to guide him into the right direction, and learn to provide accurate tasks. I'd also suggest looking into GitHub copilot chat to see where this is going.

If we're lucky, they'll get better with time and lots of research, but that's (likely) none of your concerns.

Uneducated opinion:

From this point on, I think it's likely the corpus they consume gets worse because they are starting to consume their own generated, average-of-the-internet content. You built your own Reddit bot, and know how easy it is to get generated content out there. It's a problem that isn't solved yet, and with time, it's becoming even more difficult because the number of different public models (which suffer from the same challenges) is steadily increasing, and therefore detection of such content gets a lot harder. The technology might get exponentially better, but in the end it boils down to the corpus they consume and learn from. This will slow things down, and will give you some time to learn to use these tools to your advantage, not see them as competition.

[1] https://lexfridman.com/stephen-wolfram-4/

I had a similar moment of existential career crisis as OP.

I'm at the 20 year mark as well, in terms of developing software professionally. I've always felt like with new technology, I could grok at a high level how things worked. But LLMs like GPT seem like magic and I went through stages of initial astonishment -> despair realizing the potential impact it would have on the industry -> acceptance.

While I still feel uncertainty and fear about the future, as others have echoed, I'm realizing it's a tool for developers to use. We can either choose to accept it and understand how to work with it, or reject it. The things GPT can generate amazes me, but I'm finding that it's a good starting point or reference to build on... not a final solution. It will generate things that are sometimes completely wrong, and it's your own experience and judgement that has to be used to determine that. GPT cannot do that... at least not yet.

I think back 20 years ago and remember reading through a lot of physical books with occasional web searches landing on experts-exchange or random forums. Then came Stack Overflow and that became in invaluable tool, along with the ubiquity of free tutorials on YouTube and elsewhere. And now we have GPT which I'll ask if I really get stuck on something and it gives me new ideas to try. Perhaps in the near future, GPT is the tool that I'll use first.

I found this podcast episode helpful for me to process what I’ve felt: [Lex Fridman Podcast #376 – Stephen Wolfram: ChatGPT and the Nature of Truth, Reality & Computation][1].

It's an unsettling feeling (in general) to feel like a foundation you've built and live on could potentially be made quickly irrelevant. I'd like to say I have words of wisdom to get rid of that feeling but I don't. What has helped me is to acknowledge these feelings as valid, and then try to get clarity in what direction to move. It's not the foundation itself that's important per se, but it's the skills you've acquired in building the foundation that's more important.

[1]: https://lexfridman.com/stephen-wolfram-4/

I didn’t watch that one.

So far I’ve only listened to a few. I mostly listen via the podcast feed.

I posted the ones I liked to my linkblog (https://links.markjgsmith.com):

[2020-12-26] Lex Friedman Podcast #80 - Vitalik Buterin - Ethereum, Cryptocurrency, and the Future of Money - Fascinating and well paced conversation with the founder of Etherium covering a wide range of topics including Satoshi Nakamoto, blockchains, proof of work and identity, PKI and digital signatures, Bitcoin, money, the Etherium origin story, smart contracts, software engineering and project governance challenges, proof of stake and consensus algorithms, sharding of storage and computation, Etherium 2.0, games built using smart contracts, Uniswap, AI and crypto, and closes on Immortality

https://m.youtube.com/watch?v=3x1b_S6Qp2Q

[2021-02-13] Lex Friedman Podcast #159 – Richard Craib: WallStreetBets, Numerai, and the Future of Stock Trading - Looks at the recent WalkStreetBets and Gamestop Saga that caused such a stir in the trading markets, then moves on to talk about hedge funds, shorting, new ways crypto and AI could change the world of finance, aswell as more esoteric topics like running a startup and the meaning of life

https://lexfridman.com/richard-craib

I might have a listen to the cringey one you linked to. I wonder if it was just an off day for him or perhaps he didn’t gel well with the guest.

> you start with a baby brain and let it grow up

You can grow biological brain structures from stem cell. It's called Brain Organoids and Neural Organoids, that may one day be used in machines and may one day have consciousness properties. Interview about this with Lex Fridman (MIT Artificial Intelligence teacher) and Paola Arlotta (Harvard brain neuroscientist): https://lexfridman.com/paola-arlotta/

> We don’t think of a few cells as having consciousness, but somehow the broader collection does.

This is called "emergent property" (see "Emergence" in Wikipedia), that is a sub-area of Complex Systems theory. There's a theory that states it may be possible that consciousness may appear as an emergent property from a high complex, dynamical and evolutionary AI system.

> imagine an incredibly-detailed computer simulation

Some paradigms to approach AGI (Artificial General Intelligence / Superintelligence) are: 1. Simulate the brain in low level details; 2. Simulate the functioning of higher level structures of the brain (cortical columns, visual cortex, hippocampus...); 3. Create artificial neurons (that can be used to neural networks, deep learning, etc.); 4. Simulating simple brains and nervous systems (e.g jellyfish, insects, C.elegans connectome project, etc) to understand how intelligence works; 5. Simulating the behavior of the brain ("cognitive architectures"); 6. Creating en evolutionary system that could converge to brain-like funcionality.

Slightly off topic but I love little gems in his podcast that take me down rabbit holes. For instance, while interviewing James Gosling, inventor of Java, he asked what a virtual machine was and this was the answer:

> [01:29:20] So And what is it? So the Java virtual machine, you can think of it in different ways. Because it was carefully designed to have different ways of viewing it, so one view of it that most people don't really realize is there is that you can view it as sort of an encoding of the abstract syntax tree in reverse Polish notation. I don't know if that makes any sense at all. I could explain it and that would blow all of our time. But the other way to think of it and the way that it ends up being explained is that it's it's like the the instruction set of an abstract machine that's designed such that you can translate that abstract machine to a physical machine. And the reason that that's important. So if you win back to the early 90s when we were talking to all of these these companies doing consumer electronics.

To have some fun I asked my coworker who is a lot smarter than me what that means which led to a fun 30 minute tangent.

https://www.happyscribe.com/public/lex-fridman-podcast-artif...

https://lexfridman.com/james-gosling/

> We’re still in disagreement but I wouldn’t bet on either side of the AGI debate with any significant conviction.

That is probably a superior position to hold. I am agnostic by nature, and interestingly this is one of the rare topics I've taken a hard position on. It could be a result of the years spent in the field but also some kind of bias.

> What are the manipulations you’re referring to?

Need to take a step back and mention that in the field of AI there is a great debate between symbolic and non-symbolic approaches. (and after decades spent with AI under symbolic approaches domination we are now in the golden age of non-symbolic AI; with symbolic starting to have a comeback. this podcast can be a good starting point to learn more https://lexfridman.com/gary-marcus/ - although I disagree with GM on many things - and this tweet for learning about symbolic making a comeback https://twitter.com/hardmaru/status/1470847417193209856)

Basically embeddings are "non-symbolic AI" (which is great and this is where their huge potential stems from), but the very way they are generated and then later utlized is completely "symbolic". Which means the the limits of embeddings is defined by the limits of (in this case human written) symbols used to define them. Hope that makes sense.