Jeff Kaufman

Related essay https://www.jefftk.com/p/yudkowsky-and-miri

>In talking to ML researchers, many were unaware that there was any sort of effort to reduce risks from superintelligence. Others had heard of it before, and primarily associated it with Nick Bostrom, Eliezer Yudkowsky, and MIRI. One of them had very strong negative opinions of Eliezer, extending to everything they saw as associated with him, including effective altruism.

>They brought up the example of So you want to be a seed AI programmer, saying that it was clearly written by a crank. And, honestly, I initially thought it was someone trying to parody him. Here are some bits that kind of give the flavor:

>>First, there are tasks that can be easily modularized away from deep AI issues; any decent True Hacker should be able to understand what is needed and do it. Depending on how many such tasks there are, there may be a limited number of slots for nongeniuses. Expect the competition for these slots to be very tight. ... [T]he primary prerequisite will be programming ability, experience, and sustained reliable output. We will probably, but not definitely, end up working in Java. [1] Advance knowledge of some of the basics of cognitive science, as described below, may also prove very helpful. Mostly, we'll just be looking for the best True Hackers we can find.

>Or:

>>I am tempted to say that a doctorate in AI would be negatively useful, but I am not one to hold someone's reckless youth against them - just because…

Do you track yourself? Do you check, what you are reading, and verify your own progress? Some time ago I started thinking about my problems with Google search, and was inspired by other project to write a program for myself.

Throughout my life I have found many resources worthy of my time, but which I have lost, or forgotten about. I wanted to change that. I wanted to be able to look back.

I have started my project in Sept of 2023. The core functionality is: to store, browse, and search for links. Additionally I added RSS sources as sources, so every hour I ping sources to check if there is anything new, and I expand my database. Does that replace Google search? Nope. On the other hand it expanded my horizons.

I found most of RSS aggregators lacking:
- some provides ability to bookmark entries, but some do not allow to insert links manually
- some do not provide ability to tag links
- some are on-line, and I strongly prefer local NAS web apps
- most do not provide clean ability to search link database (as it may involve heavy operations)

I found Google search lacking:
- 10 pages each with 10 links for a searched keywords is not enough. If I write 'covid' into google search I would be expecting millions of links in results, from variety of sources. Even Fox news, even daily mail
- I have results curated by big tech, I prefer to curate output myself

What I have found about archive.org:
- it is slow and unreliable
- does not store RSS sources often enough
- author…

I love it when anything Caenorhabditis elegans (C. Elegans) related pops up because this little biological organism sits at this beautiful intersection between technology and biology and philosophy. The successful emulation of C. Elegans would represent a concrete step towards whole brain emulation and all the transhuman and ethical and moral quandaries that would bring. The general idea is that the human brain has billions of neurons, Elegans has hundreds (and we've had them mapped since 1986). If one can successfully "upload" Elegans, then humans are just a matter of scale.

However, it should be noted that the field, and specifically this line of research, hasn't produced much in the way of results in 10+ years. University of Oregon planned (though I can't tell if they ever developed) NemaSys[0] ~1997. OpenWorm has been exploring this since 2011. Project Nemaload explored it a bit from 2011-2013.[1] But each project ran into three problems:

- Knowing the connections isn't enough. We also need to know the weights and thresholds. We don't know how to read them from a living worm.[2]

- C. elegans is able to learn by changing the weights. We don't know how weights and thresholds are changed in a living worm.[2]

- Funding [3]

The best we can do is modeling a generic worm - pretraining and running the neural network with fixed weights. Thus, no worm is "uploaded" because we can't read the weights, and these simulations are far from realistic because they are not capable of…

When one of my upstairs toilets started leaking in my house in the US, it finally got so bad that I noticed a stain on the garage ceiling. The insurance inspector said that insurance would not cover a "slow leak". I was very nice and polite and pointed out how soaked the subfloor was and that there was no way to know when the leak started (which was true). She wrote her recommendation that it be covered, which the company apparently accepted since they wrote me a check.

Definitely not $50k (more like $2.5k), but it's still disturbing that the line between insurance covering it and not covering it was so thin.

I wonder if a more thorough inspection would have caught it. Surely houses built around that time are routinely inspected for asbestos, lead-based paint, etc before purchase anyway. (Actually, going back and reading the comments, he actually addresses this point: http://www.jefftk.com/p/mercury-spill#fb-752340840802_752343036402. Perhaps he could have made it a condition of purchase, that the seller pay for a mercury vapor inspection? Perhaps it's not such a common problem that such a requirement would be reasonable. Hindsight, etc.)

It sounds like you're talking about relative numbers, and I have no disagreement there, but I'm talking about absolute numbers. There just weren't that many people online in 1995, so even a large fraction then can easily be less than a very small fraction now.

I know lots of people with websites (ex: my own, https://jefftk.com) and a lot of this is it's so much easier and cheaper now. You don't have to be a sysadmin or be friends with one, domains cost 10x less than they used to, many more people have internet access, etc.

> When was the last time you saw real, true, honest content, without an agenda, without cat videos, without selfies, on a modern content silo?

I think it depends a lot who your friends are, what subcultures you're in, and how you choose who to follow. My Facebook feed is mostly my friends talking about interesting things and I like it a lot.

Yes.

But quite possibly "greater efficiency" according to a fitness function that's not accurately mapped onto "keeping humans alive"...

I wonder if this'll end up in an equivalent state to the "tank detection neural net" which learned with 100% accuracy that the researchers/trainers had always taken pictures of tanks on cloudy days and pictures without tanks on sunny days? ( https://www.jefftk.com/p/detecting-tanks )

Who'd bet against the doctor/insurer neural net training ending up approving all procedures where, say, the doctor ends up with a kickback from a drug company - instead of optimising for maximum human health benefit?