Skip to content
HomeVideosBooksBlogs
AllTop postsTopics
Podcasts
HistoryListsWatch laterLiked videosFollowing

The videos, books, blogs and podcasts Hacker News keeps recommending, all in one place.

VideosBooksBlogsPodcastsAboutPrivacySettings

Data from Hacker News, HackerTube, HackerBooks, HackerBlogs and Open Library. Credits · Not affiliated with Y Combinator.

HomeVideosBooksBlogsPodcasts
Ctrl K
    D

    DBMS2

    dbms2.com · Personal blog · on HN since 2008

    Curt Monash's blog on database management systems, analytics, data warehouses, Hadoop, and the vendors in those markets.

    Visit blogHN submissions
    DatabasesBusiness & Strategy
    What HN says

    HN commenters use it as a reference for data warehouse vendors and the technical details of products like IBM System S.

    Stats for nerds
    46HN people shared it
    47linked in comments
    282story points
    Jan 2022last shared
    2008people per year2022

    Top posts on HN

    1. 1
      Details on the Oracle failure at JPMorgan Chase
      1 person · 66 points · first shared 2010 · 24 comments on HN
    2. 2
      Notes on indexes and index-like structures
      1 person · 48 points · first shared 2015 · 2 comments on HN
    3. 3
      EBay’s two enormous data warehouses
      2 people · 23 points · first shared 2009 · Discussion on HN
    4. 4
      Three big myths about MapReduce
      1 person · 20 points · first shared 2009 · 5 comments on HN
    5. 5
      Facebook, Hadoop, and Hive
      2 people · 17 points · first shared 2009 · 1 comments on HN
    6. 6
      What to do about “unstructured data”
      1 person · 16 points · first shared 2011 · Discussion on HN
    7. 7
      Toward a NoSQL taxonomy
      1 person · 15 points · first shared 2010 · 1 comments on HN
    8. 8
      The Clustrix story
      1 person · 14 points · first shared 2010 · Discussion on HN
    9. 9
      The cardinal rules of DBMS development
      4 people · 5 points · first shared 2017 · Discussion on HN
    10. 10
      Traditional databases will eventually wind up in RAM
      1 person · 5 points · first shared 2011 · Discussion on HN

    What HN people wrote when they linked it

    • jwilliamsAug 2013

      Agree. & this makes sense from an architectural perspective, since the demands of each domain are different.

      I was arguing from practical terms and my own experience. Even in established, DB-heavy, "we have hundreds of DBAs" organisations, you can't have a porcelain schema. It evolves over time, breaks, get optimised and de-optimised.

      The Oracle shops I've worked in don't have one schema, they'll have dozens. Usually because they can't shoehorn emerging requirements into the databases they have. Or because of project timelines. Or they need to be performance tuned in some special way, which in the case of Oracle could be considered a dark art.

      These orgs may have just one big data warehouse, but that becomes a dumping ground. It's possible to do your dimensional modelling (I worked on star schemas in the past, but that's about it) -- but it's a big ball of mud to try and weave together.

      Even then your OLAP is under-stress because it's got a lot of ground to cover. Usually they start out as marketing engines, but get co-opted into all sorts of things. The most common and troublesome is regulatory reporting. Once your data warehouse gets used for something like regulatory reporting you're a bit stuck as nobody wants to touch it.

      Then, because point to point integrations between the databases is brittle and cost-prohibitive -- they break the golden rule and take data out of the OLAP and put it back into the OLTP. I won't name names, but this is commonplace.

      SalesForce is…

    • yaaangJun 2011

      First off - whatever you're doing, you'll probably be fine using either.

      That said, I'm generally more frustrated when using MySQL than when using PG. Here's a sample of the problems I've encountered from using both MySQL and PG. I haven't updated my list in a while now - please feel free to correct me on things - but hopefully it's a little more illustrative than that Wikipedia feature matrix, and a little more specific to MySQL vs. PG. (This list is a cleaned-up selection from my notes wiki at http://yz.mit.edu/notes/Hackery.)

      No referential integrity.

      No constraints (CHECK).

      No sort merge join, let alone hash-join. http://www.dbms2.com/2008/07/10/how-is-mysqls-join-performance-these-days/, http://www.mysqlperformanceblog.com/2006/06/09/why-mysql-could-be-slow-with-large-tables/

      Generally poor at analytical workloads, since it's designed for transactional workloads.

      Can't reopen TEMP table - WTF? (Still not fixed!) http://bugs.mysql.com/bug.php?id=10327

      Multiple storage engines has always restricted progress: http://www.mysqlperformanceblog.com/2010/05/08/the-doom-of-multiple-storage-engines/ (PG also supported multiple storage engines in 80s, then concentrated on one)

      No WITH clause: http://stackoverflow.com/questions/324935/mysql-with-clause

      Crappy errors: “Incorrect key file for table ‘stock’; try to repair it” on “alter table stock add constraint pk_stock primary key (s_w_id, s_i_id);” where stock is in InnoDB (which has no “repair table”) means I have no…

    • ahazred8taJul 2021

      It's for companies that have large data warehouse analytics systems. -- highlights: "Syncsort cites its main data integration competitors as being, no surprise, Ab Initio (their favorite choice to compare themselves to), Informatica (with PowerCenter), and IBM (with DataStage). Syncsort DMExpress runs on dedicated boxes. Unlike most of its competitors, this is not a real-time product it is a batch product that you schedule. Syncsort also told a story of an unnamed customer for whom Oracle utterly choked on joining 5 tables of 1 terabyte each. (27 days to run with clever workarounds.) DMExpress did the join in 6 hours." -- So, it's apparently so large and enterprisey that getting it to work right would take a large amount of effort, a sizable budget, and paid tech support.

      https://www.dbms2.com/2011/04/07/introduction-to-syncsort-and-dmexpress/

    • smokinnMay 2009

      People seem to be confused about what IBM's System S actually is. For a more technical look at their new offering I suggest these two posts:
      http://www.dbms2.com/2009/05/13/ibm-system-s-infosphere-streams-processing/
      http://www.dbms2.com/2009/05/18/followup-on-ibm-system-sinfosphere-streams/

      Quick summary:
      It's for doing complex event processing of large streams of data. Think financial data or any other such sort of never ending flow of massive amounts of data that you would need to process very quickly to identify patterns and for other analysis such as determining when you hit certain thresholds or trends.

      It's supposed to be very flexible so that it can act on a large amount of unstructured data as well as very scalable. It defines its own processing language too.

    • cpleppertMay 2013

      I'm a little blown away by the total lack of technical understanding when it comes to MongoDB. This person is obviously very technical, so why would he have chosen MongoDB in the first place? It isn't like MongoDb's technical shortcomings are a secret. The description of how MongoDb does sharding and distributed queries should have immediately raised red flags to any who has even a modicum of CAP understanding. If it sounds complex, a total hack and impossible to manage it probably is.

      For gosh sakes, even the 10gen guys admit MongoDB lost data a year(!) ago. [1] If a database lost data in a single server configuration, why would you trust it as a cluster?

      1: http://www.dbms2.com/2011/04/04/the-mongodb-story/

    • ignoramousMay 2019

      CMU's Andy Pavlo is probably at the forefront in this area, here's what they were working on (they're re-doing it all over again now as a yet to be announced project): https://news.ycombinator.com/item?id=14391523 Also, I'd encourage you to go through their Advanced Database class: https://www.youtube.com/playlist?list=PLSE8ODhjZXjYutVzTeAds8xUt1rcmyT7x

      Michael Stonebreaker, a Turing awardee, called for complete destruction of old database order in 2008: https://www.dbms2.com/2008/02/18/mike-stonebraker-calls-for-the-complete-destruction-of-the-old-dbms-order/ post which, I think, they created VoltDB (H-Store) and SciDB.

      Interesting to note that Andy Pavlo worked on Stonebreaker's H-Store.

    Related

    The Morning Paperblog.acolyer.org606 people

    Local-first software: you own your data, in spite of the cloud

    The Morning Paper
    Personal · last shared 2026
    Coinbase Blogblog.coinbase.com383 people

    Container technologies at Coinbase: Why Kubernetes is not part of our stack

    Coinbase Blog
    Company · last shared 2022
    Cloudera Blogblog.cloudera.com86 people

    Cloudera taken private for $5.3B, acquires Datacoral and Cazena

    Cloudera Blog
    Company · last shared 2023
    Timescale Blogblog.timescale.com95 people

    How percentile approximation works and why it's more useful than averages

    Timescale Blog
    Company · last shared 2022
    Uber Engineering Blogeng.uber.com525 people

    Why Uber Engineering Switched from Postgres to MySQL

    Uber Engineering Blog
    Company · last shared 2025
    Baron Schwartzxaprb.com77 people

    Immutability, MVCC, and garbage collection

    Baron Schwartz
    Personal · last shared 2022
    Oracle Blogsblogs.oracle.com651 people

    “Stop reverse engineering our code”

    Oracle Blogs
    Company · last shared 2026
    Aphyraphyr.com551 people

    Hexing the technical interview (2017)

    Aphyr
    Personal · last shared 2026