Christopher Cabanne
4 HN recommenders · recommended in 6 months, 2017–2023
1:04:36 · 49K views · Aug 7, 2015
J. B. Rainsberger argues that large integrated tests are a scam: they are slow, fail opaquely, and give a false sense of coverage. He advocates focused collaboration and contract tests at boundaries instead.
HN recommends it in testing-strategy debates as the classic case for fewer integration tests and more small, well-contracted unit tests.
6 comments from Hacker News
I have adopted the same philosophy. A few resources on this, part of the so-called London school TDD: - [LINK] (and the rest of the Wiki) - [LINK] - Most of the screencasts and articles at [LINK] (especially this brilliant talk [LINK] ) - Integration Tests Are A Scam: this video All of these basically go the opposite way of the article's philosophy: Not too many integration tests, mostly unit tests. Clearly define a contract between the boundaries of the code, and stub/mock on the contract. You'll be left with mostly pure functions at the leaves, which you'll unit test.
Sure. I'd venture to say that integration tests should be fewer than unit tests, see hexagonal etc. Hopefully those external interfaces are also more stable, so they don't need to be run as often. I tend to use my integration tests also as characterization tests that verify the simulator/test-double I use for any external systems within my unit tests. See also: the testing pyramid[1] and "integrated tests are a scam"[2], which is a tad click-bait, but actually quite good. [1] [LINK] [2] this video
I've tried FitNesse professionally. I think it comes short in many regards. To be clear, I think integrated/integration tests in their own only add value if you have a couple of them (meaning <5). Otherwise, they're red and no one knows why. And they take a whole lot of time to execute. JBRainsberger has an nice talk on them [1]. Pactflow have implemented a Contract/Collaboration framework called Pact [2]. I haven't used it professionally but I love the concept. [1] this video [2] [LINK]
Trying to cover your state space with coarse-grained tests in database-backed applications can lead to slow tests suites that are laborious to set up/tear down, which is a barrier to fast build times in CI. This harms productivity both while writing tests and in deployment. J.B Rainsberger says it best: this video
I have been researching into several testing approaches and so far in my experience yes fakes are better than mocks. But what is even better is not having to fake or mock stuff especially for testing business logic. One of the problems we get ourselves into is abstracting service io code into functions and calling them from our business logic code. This forces us to mock all the io calls, instead we should be doing the opposite i.e abstracting business logic into discrete functions of pure data in and out and then calling those functions from our io code. This way we can test the business logic independently of the io code. Testing the business logic now becomes quite simple and one can leverage property testing to further boost our confidence in such tests. Many times the ramaining io code is just calling other functions and doesn't need to be tested. Several other ideas that relate to this are hexagonal architecture, Domain Driver Design, clean architecture and functional core with an imperative shell. Some resources to further explore these ideas: Clean Architecture in Python - [LINK] Why integrated tests are a scam - this video Boundaries - [LINK]