Pinterest Engineering

> Does that give access to the full history quickly?

It doesn't because no append-only log does. Apache Kafka is that "unique web3-only thing never seen before in traditional systems" that no only has been seen in traditional systems, but has been successfully used in those systems at scale dwarfing all of crypto, and has well-understood behaviours and patterns.

Edit: In 2019 Linkedin (who developed Kafka) were processing 7 trillion messages a day: https://engineering.linkedin.com/blog/2019/apache-kafka-trillion-messages

Edit: In 2018 Pinterest was handling 15 million messages a day with 1.2 petabytes of data per day https://medium.com/pinterest-engineering/how-pinterest-runs-kafka-at-scale-ff9c6f735be

A view of the final state of data is literally one of the things they require, and everyone knows how to make one.

> For those that don't want to do this, and want to quickly access just the latest state, that's exactly what services like this offer.

So this service just creates a "traditional" database with a "traditional" access method (actually an ancient access method, RPC) with a "traditional" ElasticSearch on top and pretends it's something new, never made at scale, and something no "traditional system" was ever prepared for.

Edit: see LinkedIn and Pinterest above. Or Facebook. Or Amazon. Or AliExpress. Or... "Omg they traditional web2 tech cannot work at the amazing scale of web3" is such a bold-faced lie that I can't imagine people who can look at it and say…

Hi Jon,
I was a PM on Growth at Pinterest many, many moons ago (~2013-2014?) so it was cool to see your post and read through your observations on Growth at Pinterest. A couple points I might add that might be of interest to your readers would be that the biggest changes can be technically driven, and the biggest changes are sometimes the smallest. I was at Pinterest from 2013 - 2017 and iirc the two biggest experiments in terms of percentage user signups and retention were the conversion of unauth pages to react [1] and the change from "Pin" to "Save". They were both roughly equal magnitude in impact (again, iirc), and completely outsized all other growth experiments shipped in those respective years. Two very different projects, approached from very different angles, that ultimately lead to a faster, and more understandable, product.

[1] https://medium.com/pinterest-engineering/driving-user-growth-with-performance-improvements-cfc50dafadd7

Author of the post here. I was the tech lead for Zipkin at Twitter, implemented tracing system at Pinterest[1] and a contributor to the Open tracing spec[2]. So, limited consideration of existing tools may not be too accurate of a description :).

A SpanEvent is a simpler span.

Regular Span - Annotations(nested structures) = SpanEvent.

Most developers don't think about using spans to represent DAGs in their applications. The goal of a SpanEvent is to aid developers to think in terms of DAGs.

In my experience, with some care and clever code, you can use existing Span formats. But, most developers give up and do something else instead.

[1] https://medium.com/pinterest-engineering/distributed-tracing-at-pinterest-with-new-open-source-tools-a4f8a5562f6b
[2] https://opentracing.io/specification/organization/

Time-series is definitely not synonymous column-oriented. The data model is separate from the storage layer which is separate from the use-case.

You're talking about relational databases (which is the formal type) designed for large-scale analytics using column-oriented storage and processing and supporting a time-series use-case.

Storage and querying just for time-series specifically is more about product features than the underlying type. For example, here's Pinterest doing the same on HBase: https://medium.com/pinterest-engineering/pinalyticsdb-a-time-series-database-on-top-of-hbase-946f236bb29a

One big reason why Pinterest ranks so highly, and why you are seeing different tld's in the results, is because they have a variety of tld's that they use and those tld's all have separate crawl budgets from each other. So Google crawls Pinterest about an order of magnitude more than most other sites. You can read more about this here: https://medium.com/pinterest-engineering/how-switching-our-domain-structure-unlocked-international-growth-e00c8184d5dd

Source: I run a think tank focused on Google’s web crawling advantage and have been studying stuff like this for a couple years now.

There's no intentional or manual effort at Google to promote pinterest.

Pinterest shows up because they understand how the Google algorithm works and built their website to display all the signals that Google looks for in relevant image content.

They understand user intent and generate URLs that present content in a way that google expects to see.

Examples of how they do this from their engineering team:

https://medium.com/pinterest-engineering/demystifying-seo-with-experiments-a183b325cf4c

More:

https://medium.com/pinterest-engineering/tagged/seo