James Coplien

> His solution works, but realistically only for small numbers. He never specified an input domain, and during development never tried to identify ambiguities between the API 'promise' that the function would take int32, the expectation by most that it's fast to factor 2,147,483,647 (2^31-1), and the reality that his code would take a long time for that.

I think in one of the videos I watched, he explicitly said, that this is lame implementation and can be optimized (for example by looping until square root of the number). The interesting point, is that one wouldn't have to change tests for that. One would want to write a benchmark for that, to use instead of test. As in: normal tests are tests for logic, while benchmarks are tests for performance. Additionally, it is always true, that premature optimization is not a good idea. If you have an evidence, that current algorithm is not fast enough (knowledge of your dataset sizes + understanding on what would the reasonable speed be + benchmark to verify all of that), then you optimize, without behavior change.

> I think Martin's TDD style works because the problem he uses are "simple" in an algorithm sense, which is why he can talk about a 'Transformation Priority Premise'. Martin says "Refactorings are simple operations that change the structure of code without changing its behavior", but as I wrote before, 'substitute algorithm' is not a simple operation.

Funny thing I have found about 'SubstituteAlgorithm' refactoring, is…

I put together a reading list of the best writing on automated testing I know. Most of this is agnostic wrt the language you use, though of course examples are written in a language.

## Monographs

Test Driven Development By Example (Kent Beck)

Unit Testing: Principles, Practices, and Patterns (Vladimir Khorikov)

The Art of Unit Testing 2nd ed (Roy Osherove)

Working Effectively with Legacy Code (Michael Feathers)

## Various resources

[Unit Testing Best Pratices](https://www.youtube.com/watch?v=k_ItB5btREU&t=2931s&ab_channel=SoftwareCraftsmanshipLuxembourg) (Vladimir Khorikov)

[Testing Without Mocks: A Pattern Language](https://www.jamesshore.com/v2/blog/2018/testing-without-mocks) (James Shore)

[TDD Lunch & Learn](https://www.jamesshore.com/v2/projects/lunch-and-learn) (James Shore)

[Test Driven Development - What? Why? And How?](https://www.youtube.com/watch?v=llaUBH5oayw&ab_channel=ContinuousDelivery) (Dave Farley)

[The 3 Types of Unit Test in TDD](https://www.youtube.com/watch?v=W40mpZP9xQQ&t=22s&ab_channel=ContinuousDelivery) (Dave Farley)

[TDD and the Lump of Coding Fallacy](https://www.geepawhill.org/2018/04/14/tdd-the-lump-of-coding-fallacy/) (Geepaw Hill)

## Against Unit Tests

[Why Most Unit Testing is Waste](https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf) (James O Coplien)

[On unit testing and TDD](https://www.youtube.com/watch?v=21JlBOxgGwY&ab_channel=JeruSanders) (Jonathan Blow)

the core of TDD was to iterate by making a failing test, then making it pass

Yes and when I first heard that 15 years ago (or whenever it was) it sounded like genius and I did it on a few projects.

Its a useful crutch maybe if you're doing something very unfamiliar

But as a dogma that must always be followed? And having a test for every single 'specification' and edge case? No - screw that, you end up faffing around for ages setting up mocks and testing stuff that doesnt deserve it. And you end up with a massive library of tests that get on your nerves.

Also everything got completely bent out of shape for a few years because everything had to be injectable and testable.

https://dhh.dk/2014/tdd-is-dead-long-live-testing.html

http://web.archive.org/web/20180215225218/https://iansommerville.com/systems-software-and-technology/giving-up-on-test-first-development/

http://blog.cleancoder.com/uncle-bob/2016/03/19/GivingUpOnTDD.html

https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf

> Some people involved in the project are talking about "100% unit test coverage" as a target.

100% targets ignore that software engineering is different from programming: costs in engineering hours, resource invested, maintenance burden etc. all should lower the target.

Also think hard if unit test is suitable for your program. Most compilers aren't unit tested. Distributed system unit testing requires an absurd mock infrastructure.

> The project include some fault-tolerant distributed servers, proxies and various API proxies/converters

Make the unit test target ~20% and focus rest of the time you would've spent on writing unit tests to making the system debuggable: all the best distributed systems are extremely debuggable, since there are no unit tests that will help you once there's an RPC (such things subvert any and all invariants you programmed)

- https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf also p. fun read

There was also a follow-up article [1]. My take on the two articles is that he argues that integration tests should be able to replace unit tests in most cases. However, in my own experience, both kinds of tests have their palces.

Why unit tests are good:
- You get well-tested parts that you can use in your integration tests, so that the integration tests truly catch the problesm that couldn't be caught at a lower level. This makes trouble-shooting easier.

- Decoupled design - one of the key advantages of TDD

- Rapid feedback. Not all integration tests can be run as quickly as unit tests.

- Easier to set up a specific context for the tests.

There are more details in the blog post I wrote as a response [2].

[1] http://rbcs-us.com/documents/Segue.pdf

[2] https://henrikwarne.com/2014/09/04/a-response-to-why-most-unit-testing-is-waste/

> Either you're writing tests or you're manually running the software (manual testing)

There's nothing wrong with manual testing. In fact, unless you have good set of E2E tests (most companies don't), you still need manual testing.

At some point manual testing may become tedious and it's can be worthwhile to automate it to certain degree, typically with a few integration tests.

But well-architected systems need very few tests to be reliable. Having a lot of tests is not something you should celebrate, it's a signal your architecture sucks.

And if you find yourself writing a lot of unit tests, please read this: https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf