Engraved is a blog on machine learning, including a well-known post about building a virtual machine inside ChatGPT.
HN commenters cite the virtual machine post as mind-blowing and as evidence of understanding, though one finds a claim in it implausible.
> This is not surprising. A human would suffer from similar errors at a similar rate if it were exclusively fed an interpretation of reality that only consisted of text from the internet.
I think this is surprising at least if the bot actually understands, especially for domains like math. It makes errors (like in adding large numbers) that shouldn't occur if it wasn't smearing together internet data. We would expect there to be many homework examples on the internet of adding relatively small numbers but less of large numbers. A large portion of what makes math interesting is that many of the structures we are interested in exist in large examples and in small examples (though not always) so if you understand the structure, it should be able to guide you pretty far. Presumably most humans (assuming they understand natural language) can read a description of addition then (with some trial and error) get it right for small cases. Then when presented with a large case would generalize easily. I don't usually guess out the output and instead internally try to generate and algorithm I follow.
> Take for example: https://www.engraved.blog/building-a-virtual-machine-inside/
When I first saw that a while back, I thought that was a more impressive example but only marginally more so than the natural language examples. Like how these models are trained under supervised learning imply that it should be able to capture relationships between text well. Like you said, there's a lot…
>People’s susceptibility to anthropomorphizing an even slightly convincing computer program has been known since ELIZA, one of the first chatbots, in 1966. It’s called the ELIZA effect.
I'm tired of these arguments. Very very few people are anthropomorphizing chatgpt.. very few. The majority of people both technical and non technical who have played with chatGPT in a non trivial way are aware of the chatbots limitations. It's like recognizing a crazy person on the street. Humans are well equipped for that.
This argument characterizes the average person as some kind of stupid buffoon as if he/she can't tell chatGPT really screws shit up. Sure there are a few gullible outliers but as a generality his claim is simply completely false. Pretty much everyone and I mean everyone is aware about the limitations of LLMs.
This is a weak and repeated trope that's being regurgitated as if the critics are LLMs themselves.
Let me specify exactly what's going on. People who are afraid of/support AI are more speaking to the potential of AI. Why? Because just as much as this thing hallucinates about half the time it answers a complex question with an equally complex and correct answer and that answer in isolation is often indistinguishable or even at times superior to a humans answer.
Yes we know it hallucinates and forgets shit. This is obvious, no need to readdress an obvious weakness that everyone is aware about. If critics want to have a real discussion then they seriously need to…
I use it a lot for understanding programming concepts or features of programming languages. Sometimes to write the code itself. Really handy for maintenance type scripts around the local pc but also for more involved tasks.
I use it in place of a search engine if I get bad results from Google. Farily often the result from ChatGPT is better at this point.
It's good for book summaries if it knows about the book, which it often does.
It's great for learning new concepts. I have a enthusiast level interest in quantum physics and quantum computing. I always have a bunch of naive questions after I read book related to the subject. ChatGPT's explanations are limited but fascinating and have introduced me to concepts I was previously unfamiliar with such as quantum teleportation and quantum communication.
It's also good for learning new programming languages. I think I will start with ChatGPT from now on with any language I learn. Paticularly the ability to type in 'I know how to do X with python, How would I do it with Rust? This part doesn't make sense to me - can you explain why it does that? Could you show what the code would look like in Go?' This is just endlessly useful. You still want to incorporate the official docs and other sources (I think?) but it's just such a delightful path to learn. It's worth noting that with learning being much easier this way you could also probably learn more langugages per year now than might have otherwise been possible if that was your…
This caught my attention as I found it implausible:
> One DeepMind engineer even reported being able to convince ChatGPT that it was a Linux terminal and getting it to run some simple mathematical code to compute the first 10 prime numbers. Remarkably, it could finish the task faster than the same code running on a real Linux machine.
Following the link, there's a screenshot to a screenshot [0] of a code-golf solution to finding primes which is quite inefficient, and the author notes
> I want to note here that this codegolf python implementation to find prime numbers is very inefficient. It takes 30 seconds to evaluate the command on my machine, but it only takes about 10 seconds to run the same command on ChatGPT. So, for some applications, this virtual machine is already faster than my laptop.
So it's not quite calculating primes; more likely it recognizes the code as being code to do so, and recites the numbers from memory. That's interesting in its own right, but we won't be running Python on an LLM for a performance boost any time soon. In my experience this interpreting is apparent as a limitation of the model when it keeps insisting on broken code being correct, or having its mistake pointed out, then apologizing, saying it's got some new code that fixes the issue, and proceeding to output the exact same code.
[0] https://www.engraved.blog/content/images/2022/12/image-13.png
>https://www.engraved.blog/building-a-virtual-machine-inside/
Read to the end. Nothing amazing here, but the tasks that chatGPT did in that article, especially at the end indicates understanding. It shows that the responses are more then a probabilistic word jumble.
Keep in mind. Bullshit and understanding are orthogonal concepts. If someone bullshits does that mean they don't understand things? No. It does not.
Honestly, I can't imagine how someone can read that article in it's entirety and can still say chatGPT doesn't understand anything.
Let's be clear. I don't think chatGPT has emotions, I don't think chatGPT does any ponderings outside of answering a response, I don't think chatGPT has any motivations.
What I do believe is chatGPT understands the input given to it. That understanding allows chatGPT to craft a truthful or bullshit response as output.
I'm just saying the tech is already here. The core engine can do it.
Before you go on and write such a system it's better to test if the LLM can do debugging to an efficacy level that we require. I don't think anyone has tried this yet and we do know LLMs have certain issues.
But make no mistake, the possibility that an LLM knows how to debug programs is actually quite high. If it can do this: https://www.engraved.blog/building-a-virtual-machine-inside/ it can likely debug a program, but I can't say definitively because I'm too lazy to try.