Pindancing

I have mixed feelings about this.

On the one hand, the peer review system was designed specifically to keep out the low-quality material that blogs let through for free because of their more inherently democractic nature; I can't be the only one who thinks many people who blog (and even some who appear on HN) are not presenting useful enough or clear enough material to be worth reading by any audience. People's lives are finite and time spent reading published work that is useless detracts from time spent reading useful work that helps in one's own line of inquiry.

Smooth talking bullshitters like Ron Jeffries who promote objectively useless or even counterproductive methods abound, taking bandwidth away from more useful ideas. You need only remind yourself of this exchange:

http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-solvers.html

http://pindancing.blogspot.com/2009/09/sudoku-in-coders-at-work.html

It will be sad the day a Ron Jeffries of the academic world wastes as much of other people's time reading his posts.

But has that day come already? On the other hand, it's not like the peer review system is immune to gaming and corruption either. We've got people chopping whatever work they've done into least-publishable-units and objectively good work being ignored because it is not like the other peer-reviewed work out there. And since most people working in a scientific field have not also gone through educating themselves against bullshit techniques (i.e.,…

I got good at mathematics. IMHO, This is the key step in making the shift from enterprise dev to ML. This is also where I've seen many people making the attempt give up.

I also worked through some AI/Machine Learning books. I started with Norvig and Russell's AIMA and open sourced resulting Java code (yes it is a beginner's book and not tough at all, but it is very well written and a great starting point).

Then I worked through more narrowly focuses books (and there are plenty of nice books see lists below). I got involved in the local ML community (mostly defence labs - Bangalore doesn't have many cutting edge startups/ sw companies yet).

The combination of deep math and good programming opens many doors (as Bradford Cross and others point out elsewhere - see Brad's post at http://measuringmeasures.blogspot.com/2010/01/learning-about-statistical-learning.html for a list of resources . I wrote a supplementary blog post on the books I found useful (I have a somewhat different focus from Bradford and so ended up working through a different set of books) at http://pindancing.blogspot.com/2010/01/learning-about-machine-learniing.html).

It helped that I didn't know I was not supposed to do such "crazy" things in my old age and (perhaps rather naively in retrospect) didn't doubt for a second I could teach myself anything I wanted to. I have zero academic background comp sci/math/ML etc.

My proudest moment was when a senior researcher I respect highly at first refused to…

Abstractions aren't merely for duplication removal. Abstractions are for symbolic chunking; for thinking about things at different levels.

We don't use e.g. units-of-measure types [1] because they remove duplication from our code; if anything, they add duplication. They do however clarify our thinking with help from the type checker.

I think a lot in terms of flows / pipes / functional transforms. So I tend to try and express problems in that shape, because I have a lot of mental tools that I can apply, and I know they're extremely easy to test in isolation. Creating a pipe-like thing means reducing it to a simple common push or pull stream pattern. But it's not duplication I'm removing here; I'm actively introducing an abstraction because it has proven power for creating good software and solving problems.

I try and create few, minimal abstractions that can be applied widely. Take a cue from functional languages and split out types from algorithms; if you make your types more general, you increase the reusability of your algorithms. Classical OO design tends to create a lot of types that are specific to the domain model. I happen to think that OO designs are usually poor; they tend to have a high code complexity to implemented logic ratio, and require awkward composition that leaves details hanging out.

I spent 20 years writing OO software and was a big fan, especially in the early 2000s. I still work in OO languages, but most OO code I read makes me wretch now.

I…

Always gonna have to side with Peter Norvig on this one: https://pindancing.blogspot.com/2009/09/sudoku-in-coders-at-work.html

> They said, “Look at the contrast—here’s Norvig’s Sudoku thing and then there’s this other guy, whose name I’ve forgotten, one of these test-driven design gurus. He starts off and he says, “Well, I’m going to do Sudoku and I’m going to have this class and first thing I’m going to do is write a bunch of tests.” But then he never got anywhere. He had five different blog posts and in each one he wrote a little bit more and wrote lots of tests but he never got anything working because he didn’t know how to solve the problem. I actually knew—from AI—that, well, there’s this field of constraint propagation—I know how that works. There’s this field of recursive search—I know how that works. And I could see, right from the start, you put these two together, and you could solve this Sudoku thing. He didn’t know that so he was sort of blundering in the dark even though all his code “worked” because he had all these test cases.

This is much harder than it sounds. Most techniques I’ve seen end up using separate agents to do the planning, implementation, and judging.

The elaborate workarounds you have to build to help an agent which fundamentally doesn’t know what it’s doing reminds me of this old blog post about TDD: https://pindancing.blogspot.com/2009/09/sudoku-in-coders-at-work.html?m=1

IMO present technology is tailored for an experienced developer to give agents manageable tasks that can be one-shot. The marketing right now reminds me of the 90s when AskJeeves promised natural language search when the technology was fundamentally still stuck in keyword search, and learning to craft a search query for Google is today’s prompt engineering

Whenever I hear about searching for a mentor, I imagine some guy that simply
waits for illumination to happen, by osmosis, I suppose. I see searching
specifically for a mentor as a kind of fashion or supersitition of these days.
(Mind you, this is merely an association, a first impression that I have when
I see such questions.)

Do you have a plan how to use this "mentoring" thing? A concrete idea what you
want from a mentor and how your work would look like? Do you have anything to
show him/her, so the mentor has any foothold for comments?

This is a good piece of article about exactly that:
http://pindancing.blogspot.com/2010/12/answer-to-will-you-mentor-me-is.html