
A love triangle involving 17-year-old misfit Arnie Cunningham, his new girlfriend and a haunted 1958 Plymouth Fury. Dubbed Christine by her previous owner, Arnie's first car is jealous, possessive and deadly.
Human context windows are not linear. They have "holes" in them which are quickly filled with extrapolation that is frequently correct.
It's why you can give a human an entire novel, say "Christine" by Stephen King, then ask them questions about some other novel until their "context window" is filled, then switch to questions about "Christine" and they'll "remember" that they read the book (even if they get some of the details wrong).
> Can you provide some examples of problems where humans have such large context windows?
See above.
The reason is because humans don't just have a "context window", they have a working memory that is also their primary source of information.
IOW, if we change LLMs so that each query modifies the weights (i.e. each query is also another training data-point), then you wouldn't need a context window.
With humans, each new problem effectively retrains the weights to incorporate the new information. With current LLMs the architecture does not allow this.
Tried As I Lay Dying recently, maybe I just wasn't in the right headspace for it but I really couldn't make head nor tails of it. I was looking forward to reading Faulkner too, and this has soured me for a little bit.
Napoleon by Andrew Roberts defeated me. I was finding it interesting but I guess the sheer size put me off. I don't often read non-fiction so I find those a lot more difficult to get through, I want to work on that.
See also: Christine (Stephen King)