Marc Brooker

I've had that experience on multiple occasions. Initially, when younger, I did attribute it to myopism. Today, I recognize that credibility also needs to be earned. All architects are unhappy except those who manage to find the right client. The client-architect relationship is fundamental. A successful architectural undertaking is minimally a two variable equation. Client (aka stakeholders) are hugely important.

(this is related imo to your post: https://brooker.co.za/blog/2020/10/19/big-changes.html - )

On a related tangent, I've been re-reading "The Open Hand - Essays on Le Corbusier". Specifically, the "Le Corbusier at Pessac - Professional and Client Responsibilities" essay by Brian Brace Taylor.

https://www.worldcat.org/title/open-hand-essays-on-le-corbusier/oclc/3003322

Pessac: https://www.archdaily.com/940878/le-corbusiers-cite-fruges-lessons-from-a-modern-social-housing-neighborhood

That project basically failed its intended purpose. The client, M. Frugès, was apparently an enlightened capitalist, willing to invest in the vision of the young architect, even when incurring financial losses, because of Pessac. Corbu also learned, on the job, how to, and how not to, build his modernist vision. The whole thing was a fiasco on many levels. But it was a necessary project for progress in the field.

Which is the point: architectural innovation is costly and even great architects leave a trail of failed early projects. Yes, Corbu went on to master his field, and he did…

I think framing it as a site for people that are positive about AI is probably the wrong way to go, but around it kf you don't want just a bunch of AI psychosis posts and nothing else.

I think it would be better if, instead of being defined in terms of being "positive" about AI (in the sense of saying that everything to do with AI is all great and rosy and awesome, and thinking every new AI thing is the next big thing, and praising every vibe-coded piece of slop), the focus was explicitly on having mature, software engineering-focused discussions of how to actually understand, make, and use AI and AI tools/harnesses, where it's taken as common ground that these are tools, not magical genies that will usher in the rapture of the nerds or make software development obsolete, nor uniquely evil technologies that only fascists could ever use or like or that are completely useless, or tools that only talentless hacks who don't care about the craft, good use. So basically to avoid shallow dismissals and endless culture of war arguments.

I would love to have a place that discusses the pros and cons of different ways of approaching specification, natural language programming, DSLs, property-based testing, behavior-driven development, lightweight formal methods like model checking, or even more heavyweight formal methods like doing proofs, etc, as means of using the extra time and lowered cognitive load allowed by AI coding agents to actually increase the reliability and improve the…

Thanks Marc, I'm glad, you too!

Ever since I came across https://brooker.co.za/blog/2018/01/01/balls-into-bins.html, I've been really excited to see and follow all the fantastic design work you've been doing.

>> VR's view change protocol for leader election is also entirely in-memory so it's far more fault tolerant compared to Raft if you have a storage fault model and not only a network fault model.

> Only in the fail-stop model, right? Or does this property extend to other models (like omissions)?

You're far more knowledgeable about the domain... but I was thinking that with VRR's completely in-memory view change and replication protocol, this would then extend past the fail-stop model to include byzantine disk storage faults (misdirected reads/writes, corruption, latent sector errors) since VRR requires no guarantees from the disk in order for the consensus protocol to be correct, whereas Raft does.

To me this is just another reason that makes VRR such a fantastic protocol.

To be fair, I guess we could say that Raft assumes a fail-stop disk model, but then again Raft is supposed to be a practical implementation and disks are not fail-stop in reality. I'm sure you're also well familiar with WISC's Protocol-Aware Recovery for
Consensus-Based Storage: https://www.usenix.org/system/files/conference/fast18/fast18-alagappan.pdf

Beyond that, I've also been wondering about this from the perspective of taking VRR's in-memory view change for the leader election phase, but…

> this does not mean that a general algorithm has to be slow when run on typical instances

True, but we have a few decades of experience with model checkers running on real programs. Yes, they work very well and are quite affordable, but their domain is rather specific (there is amazing work on model checkers designed to expand their domain and work on ever-larger programs, but the results aren't in yet).

> Unless you're generating programs randomly, programming generally entails carrying at least an informal proof of correctness in mind.

There are various results showing that the difference in size between an informal proof (even when correct, which, as experience shows often isn't[1]) and a formal one may be exponential.

> We need better tooling, more clever proof-finding algorithm to bridge the gaps, clearer interfaces.

Absolutely, but we must understand there are fundamental problems in play.

> Importantly, it is not a computational barrier. There is nothing about the formal verification of typical software that intrinsically requires expensive computation.

This is unknown, but not very promising; certainly not promising enough to assume this will definitely be possible some day (let alone soon). See the result about symbolic model checking (basically the cost of proof that is allowed to exploit linguistic constructs) that I mention in my post. It turns out that in the general case (this time, we're not talking about completely "random" programs, but "random"…

> Also, I see you amended your original comment with a jab against me for the type keys post.

I didn't realize you were the same person who wrote that blog post, and I edited it to include it when I realized. I linked your amended version, not your initial version if that's of any worth.

And you're correct it's not closed. I still stand by that particular proposal is going nowhere. You can make any proposal you want. It hasn't even moved to the design stage yet in 3 years, so I'm not sure why it's something you hang on to as a signal error handling is changing. Take generics for example, and the years (decade) of work that it took.

This is unsolicited blog feedback:

Your blog and writing style doesn't read like it comes from a place of humility and learning, but from a place of authority. Sometimes that's OK, but in your case, feels unwarranted. And the constant push to get it on to HN or Reddit or N other platforms so it can spread... Thought leadership as an aspirational goal has never been something I look for in blogs I read (I recoil and go elsewhere when I sense this is the goal).

I understand it's hard work, and you want others to see it, but just someone else's perspective.

Some blogs that I really enjoy (and I hope others emulate so I'm sharing):

- http://neugierig.org/software/blog/

- https://brooker.co.za/blog/

Exactly this.

DynamoDB is a pain in the ass if you want to do too many relational or arbitrary queries. It's not for data exploration.

It is my favourite database though (next to S3)! For cases where my queries are pretty much known upfront, and I want predictable great performance. As Marc Brooker wrote in [1], "DynamoDB’s Best Feature: Predictability".

I consistently get single digit millisecond GETs, 10-15ms PUTs, and a few more milliseconds for TransactWriteItems.

Are you able to complex joins? No. Are you able to do queries based on different hash/sort keys easily? Not without adding GSIs or a new table. The issue in the past few years was the whole craze around "single-table design". Folks took it literally as having to shove all their data in a single table, instead of understanding the reason and the cases that worked well. And with ongoing improvements of DynamoDB those cases were getting fewer and fewer over time.

But, that's what tradeoffs are about. With on-demand tables, one-shot transactions, actually serverless storage/scaling, and predictable performance you get very, very far.

1. https://brooker.co.za/blog/2022/01/19/predictability.html