Eugene Yan, an applied ML practitioner, writes about LLM evaluation, recommender systems, diffusion models, data teams and engineering practice.
HN commenters recommend his posts on LLM patterns, text-to-image diffusion and simplicity, and call his blog fantastic.
Um... can someone explain what this actually does?
In the video the user chooses the 'Copilot: Draft' action, and wow, it generates code...
...but, the 'draft' action [1] calls `/get_chunks` and then runs 'queryLLM' [2] which then just invokes 'https://api.openai.com/v1/chat/completions' directly.
So, generating text this way is 100% not interesting or relevant.
What's interesting here is how it's building the prompt to send to the openai-api.
So... can anyone shed some light on what the actual code [3] in get_chunks() does, and why you would... hm... I guess, do a lookup and pass the results to the openai api, instead of just the raw text?
The repo says: "You write a section header and the copilot retrieves relevant notes & docs to draft that section for you.", and you can see in the linked post [4], this is basically what the OP is trying to implement here; you write 'I want X', and the plugin (a bit like copilot) does a lookup of related documents, crafts a meta-prompt and passes the prompt to the openai api.
...but, it doesn't seem to do that. It seems to ignore your actual prompt, lookup related documents by embedding similarity... and then... pass those documents in as the prompt?
I'm pretty confused as to why you would want that.
It basically requires that you write your prompt separately before hand, so you can invoke it magically with a one-line prompt later. Did I misunderstand how this works?
[1] -…
Hi, Hamel here. I'm one of the co-authors. I'm an independent consultant and not all clients allow me to talk about their work.
However, I have two that do, which I've discussed in the article. These are two production use cases that I have supported (which again, are explicitly mentioned in the article):
1. https://www.honeycomb.io/blog/introducing-query-assistant
2. https://www.youtube.com/watch?v=B_DMMlDuJB0
Other co-authors have worked on significant bodies of work:
Bryan Bischoff lead the creation of Magic in Hex: https://www.latent.space/p/bryan-bischof
Jason Liu created the most popular OSS libraries for structured data called instructor https://github.com/jxnl/instructor, and works with some of the leading companies in the space like Limitless and Raycast (https://jxnl.co/services/#current-and-past-clients)
Eugene Yan works with LLMs extensively at Amazon and uses that to inform his writing: https://eugeneyan.com/writing/ (However he isn't allowed to share specifics about Amazon)
I believe you might find these worth looking at.
bluesky uniquely hates AI. very toxic around it. https://eugeneyan.com/writing/anti/
its sad that the informal social media bubbles we all have formally forked[0] into Red Twitter and Blue Twitter. just a direct consequence of filters, feeds, algorithms that we say we dont want but vote for with our actions.
with Latent Space I try very hard to stay practical and grounded. but this means i lose out to other podcasters who get huge hits by asking biglab people their AGI timelines (perpetually 2 years away every 2 years, upper bounds of every estimation conveniently landing within the lifetime of $current_middle_aged_generation)
with AIE i started my agents conference with a loud agents skeptic (https://www.youtube.com/@aiDotEngineer/search?query=sayash). that did better online but the audience/sponsors didn't like being told they were overhyping.
[0]: wont stop there - i have called it the Great Unzippening: https://swyx.io/the-great-unzippening
Lately, I've been enjoying a lot of posts in this blog right here:
https://eugeneyan.com/writing
My company just hired some new VPs from Amazon, and the writings in that blog convey a bit of the Amazon WOW. It's pretty insightful stuff.
Also, many posts from Julia Evans :D
More concretely, here are a few blog posts I go back to occasionally:
https://doordash.engineering/2019/02/20/experiment-rigor-for-switchback-experiment-analysis/
https://www.unofficialgoogledatascience.com/2019/12/humans-in-loop-forecasting-integrating.html
https://www.unofficialgoogledatascience.com/2016/10/practical-advice-for-analysis-of-large.html
https://eugeneyan.com/writing/ml-design-docs/
https://eugeneyan.com/writing/what-i-do-before-a-data-science-project-to-ensure-success/#first-draw-the-map-to-the-destination-one-pager
https://www.fharrell.com/post/addvalue/
https://www.fharrell.com/post/stat-ml2/
Putting random links in your comments is of limited value, and it shouldn't add authority to your claims. It's actually too common on HN for people to make things up and then put some sort of citation that actually doesn't support what they are claiming whatsoever, knowing that 99.9% of users just don't care that much. The fact that it actually works on some people is hugely unfortunate.
However while I'm not the other guy, how about [1] and [2]. I desperately hope that this ups my links to post ratio!
The whole "block list" thing on bsky has been a massive torch mob/brigading tactic, and it's boorish.
[1] https://eugeneyan.com/writing/anti/
[2] https://bsky.app/profile/eugeneyan.com/post/3lcykteugys2w
I completely agree with you. I failed to articulate it well.
Like the current top HN post suggests (https://eugeneyan.com/writing/llm-patterns/), we’re still discovering patterns that work well with LLMs.
That said, anecdotally - they already excel at being logic engines. Capable of filling in the gaps between instructions. Using their worldly knowledge or “common sense” to do so.
But ever so often, they’ll miss an important bit. And I have to be quite involved to catch that. Kinda defeats the purpose. Here, I think we can benefit from supervisor LLMs. A second layer, whose sole job is to ensure the output quality. A QA bot - essentially.