
This book is about programming for data processing applications, and its purpose is to present a coherent method and procedure for designing systems, programs, and components. The main emphasis is on the structure: on the dissection of a problem into parts and the arrangement of those parts to form a solution.
* A program should correspond 1:1 to its design. "Program structure [should] match the problem structure."
* Avoid flowcharts (flow of control) until the structure is designed. "Flowcharts tend to conceal structure."
* (Reuse of) components (before design) are a premature optimization.
* Write the program by composing primitives, not all of which are actually primitive operations for your machine.
---
(p. 35) In bottom-up design we say instead "this machine is not well-suited to my problem, because the elementary operations are too elementary; I shall therefore use the elementary operations to create a more powerful machine, and the operations of that machine to create one still more powerful, until I have a machine on which my problem can be easily solved".
---
Jackson believes many programs can be described as serial processing of files, so it's a philosophy I'm trying to understand.
Combined with Dubray's State-Action-Model (SAM), Miro Samek's quantum statecharts, and James Shore's nullables testing, it's a heady revitalization of the next couple decades.
> nuances of the system are not clear until I have implemented it
Yes, exactly. Each commit contributes to a graph of discovering each best move (balanced by time or pragmatics). It's hard to draw boxes and scribble when typing things out would surely straighten any gnarls.
But many of the CS pioneers dealt with the issue of reading input, processing it, and giving a meaningful output. Recently I've stumbled upon Jackson's Principles of Program Design and it has really helped me in writing a parser. IMHO general understanding of structuring a program goes a long way than arbitrary design patterns. I now am a propenent of modular programming. For me it resembles functional programming and TDD. Basically it states your program should be made up of numerous self-contained modules that can be independently tested.
For data science workflow it could be an import package, further divided in modules such as general helpers and implementations for different file types; a computation package etc.
Principles of Program Design, Michael Jackson. I found a very good copy with free shipping for less than $5.00. It is absolutely brilliant. It is a conventional technical book style presentation of the many software engineering principles SICP presents using new journalism techniques. One of the cool things is that Jackson dogfoods his methodology to illustrate deeper computer science concepts (e.g. the necessity of a start state for a state machine) with practical examples rather than theory. It does more showing than telling.
</rather late book report>
The relief staff in the bullpen includes:
Programming Clojure, Stuart Halloway.
Elements of ML Programming, Jeff Ullman.
Extreme Programming in Practice, Newkirk and Bob.
---
In top-down design we say, in effect, "this problem is too complex for me; I shall dissect it into a number of smaller problems, and those into smaller problems still, until I have a set of problems simple enough to solve."
In bottom-up design we say instead "this machine is not well-suited to my problem, because the elementary operations are too elementary; I shall therefore use the elementary operations to create a more powerful machine, and the operations of that machine to create one still more powerful, until I have a machine on which my problem can be easily solved."
-- Jackson, M.A. Principles of Program Design (1975)