Google Testing Blog

Your integration test needed the system that you were integrating with, so you'd have to declare that as a dependency.

My philosophy was to always have integration tests run in the normal CI system. This basically meant creating a test binary that happened to link in the systems you were integrating with, and run tests against that. This is easier when everything is written in the same programming language, and for the cases where it wasn't, I was usually happy with "fakes". (https://testing.googleblog.com/2013/06/testing-on-toilet-fake-your-way-to.html)

Other teams really loved the sandbox environment with live instances of everything. They would have some machinery outside the standard CI system to inject their code into this sandbox and run some tests, as well as machinery for keeping their sandbox up to date with production. (And adding test data, etc., etc., which all becomes very complex very quickly.)

Both methodologies have their downsides and upsides.

I generally prefer simplicity and speed; people should be able to run the tests on their workstation 100% of the time without having to set up any external resources. If you have an integration test binary that is built from the build system, this is possible. The downside is that config changes in production can break your system; since you are starting up your own instance of some other team's server, they could theoretically make some config change that breaks your integration. Even if you include their…

Sort of, yeah. See https://testing.googleblog.com/2017/04/code-health-googles-internal-code.html as one of the examples of a type of Engineering Productivity work that I do at Google (although as you have basically pointed out, there are reasons we avoided that word for the work that I do, although I still think it falls under the broader umbrella).

It seems like the thing that you're worried about (I mean, correct me if I'm wrong, this is just the sense I get from your post) is people hiring a sketchy consultant to come in and have everybody follow some process whether or not it has anything to do with their work or making things actually better. I'm definitely also opposed to stuff like that and I've made a few jokes or snide comments about it here and there on the blog, in http://www.codesimplicity.com/book/, and on my Twitter feed.

I believe pretty strongly in the "testing pyramid" - that you should have a broad base of unit tests,
with fewer integration tests and even fewer e2e tests. Source (also from Google):
https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html

I think good unit test coverage is essential for good maintenance, but you can't cover _everything_ with unit tests. You definitely need integration and e2e tests to ensure bigger picture correctness. I have found that value one gets out of the different kinds of tests as you proceed up the pyramid get higher the closer to release you are, and are most valuable post release. During development they become obsolete too quickly due to changes in how things work. But locking in an already-released version to avoid regressions is great.

I think one of the biggest issues is the definitions we use.

Some developers would call mock any kind of double that replaces dependencies. Other developers define mocks as doubles that verify interactions with the dependency (interaction testing [0]).

In this case the author mentions mocks and spies, therefore he's referring to "interaction testing". In other words, tests that verify that the dependency was called in the "right" way. If you own those dependencies, and later you refactor them you'll also need to refactor any test that used "interaction testing".

If every time that you want to refactor a dependency you break a ton of tests, then refactoring becomes a tedious and sometimes difficult task.

[0]https://testing.googleblog.com/2013/03/testing-on-toilet-testing-state-vs.html

This article relates to two classic principles:

1. Avoid long parameter lists [1]

2. Avoid boolean arguments [2]

For #1, long parameter lists should be named (named arguments, options object, etc).

For #2, and booleans should be replaced by meaningful enumerations.

> toggleMenu(true); That’s clear enough. the meaning is obvious

Actually, it's incredible ambiguous. Is that toggling the menu? Or setting it to the open state? Or something else? I have no idea.

setMenuState(MenuState.OPEN)
setMenuState(MenuState.CLOSED)

[1] https://testing.googleblog.com/2024/05/avoid-long-parameter-list.html

[2] https://alexkondov.com/should-you-pass-boolean-to-functions/

One random thing I've been impressed with (that I know is public) is mutant testing: https://testing.googleblog.com/2021/04/mutation-testing.html , https://research.google/pubs/pub46584/

To pull an example from the paper, if your line of code says

if (a == b || b == 1)

you might get a comment that says something like

> "Changing this line to

if (a != b || b == 1)

> does not cause any test to fail."

Page 4 lists other mutations, like replacing logical ANDs/ORs with just `true` or `false`, switching arithmetic plus to minus, etc.

This isn't a larger system design context thing, though, just a testing one.

Disclosure: I work at Google.