Name: The data warehouse ETL toolkit
ISBN: 9780764567575

The data warehouse ETL toolkit

Cowritten by Ralph Kimball, the world's leading data warehousing authority, whose previous books have sold more than 150,000 copies Delivers real-world solutions for the most time- and labor-intensive portion of data warehousing-data staging, or the extract, transform, load (ETL) process Delineates best practices for extracting data from scattered sources, removing redundant and inaccurate data, transforming the remaining data into correctly formatted data structures, and then loading the end product into the data warehouse Offers proven time-saving ETL techniques, comprehensive guidance on building dimensional structures, and crucial advice on ensuring data quality

What Hacker News says

3pt14159Aug 2018

Spark, etc, are great, but honestly if you're just getting started I would forget all about existing tooling that is geared towards people working at 300 person companies and I would read The Data Warehouse ETL Toolkit by Kimball:

https://www.amazon.com/gp/product/1118530802/

I learned from the second edition, but I've heard even better things about the third. As you're working through it, create a project with real data and from-scratch re-implement a data warehouse as you go. It doesn't really matter what you tackle, but I personally like ETLing either data gather from web crawling a single site[0] or push in a weekly gathered wikipedia dump. You'll learn many of the foundational reasons for all the tools the industry uses, which will make it very easy for you to get up to speed on them and to make the right choices about when to introduce them. I personally tend to favour tools that have an API or CLI so I can coordinate tasks without needing to click around, but many others like a giant GUI so they can see data flows graphically. Most good tools have at least some measure of both.

[0] Use something like Scrapy for python (or Mechanize for ruby) with CSS selectors and use the extension Inspector Gadget to quickly generate CSS selectors.

thread

turtleyachtJul 2023

"Fact tables hold the measurements of an enterprise. The relationship between fact tables and measurements are extremely simple. If a measurement exists, it can be modeled as a fact table row. If a fact table row exists, it is a measurement.

"What is a measurement? A common definition is an amount determined by observation with an instrument or scale."

-- The Data Warehouse ETL Toolkit (2004) Wiley Publishing.

Maybe facts need their own "directed graph" of publicly accessible acknowledgements. And to make it more complex, since we can't write one value to a cell, we may need to record a series of events.

thread