ridiculousfish

The "ack" you reference is apparently faster only in that it's easier to specify exactly which files to search or not to search, which obviously can also be done with grep, e.g. combined with 'find'.

As for grep itself, it has had many incarnations over the decades with various pluses and minuses.

The current GNU grep allows 3 kinds of regex, basic/extended/perl -- the latter being what ack supports.

Note that Perl regexes have extensions beyond regular languages that are inherently slower than the automata specified by basic regexes. Power versus speed.

E.g. grep(1): "Back-references are very slow, and may require exponential time."

For further info:

> why GNU grep is fast
> Mike Haertel mike at ducky.net
> Sat Aug 21 03:00:30 UTC 2010

> Here's a blog post from 2006 about a developer trying to "beat grep" and looking at the algorithms it uses; it goes into a little more detail about the "doesn't need to do the loop exit test at every step" optimization mentioned in this email.

http://ridiculousfish.com/blog/posts/old-age-and-treachery.html

via

https://lists.freebsd.org/pipermail/freebsd-current/2010-August/019310.html

The best writeup is surely by the inimitable Russ Cox, who really really explains clearly when grep as of 2007 was one of the only fast regex implementations:

Regular Expression Matching Can Be Simple And Fast [#1]
(but is slow in Java, Perl, PHP, Python, Ruby, ...)

Russ Cox 2007 jan

http://swtch.com/~rsc/regexp/regexp1.html

(This is a 4 part…

I think very few people equate intelligence with getting any job or making any money at all. Can you point me to someone who has? (Okay, I guess a more general form of 'intelligence' suggests it would be impossible for a slug to apply for and get any job on the market.) Similarly I don't think people equate intelligence with getting a good, high-paying job, but there is a good correlation showing more intelligence matches with higher income, and if you use that correlation in your predictions they will be more accurate.

I watched "Dirty Jobs", great show. I agree a lot of jobs don't require much in terms of IQ, creativity, or rationality, but as you move up the scale of which jobs pay the most, you'll find they do start to require such things that commonly go into 'intelligence' definitions. You'll find counter-examples (like programmers that shouldn't be let anywhere near a computer, or CEOs who don't know a thing about their own business), but fewer and fewer. Bringing this comment back momentarily to the predictive power of intelligence, I would bet that a high IQ compilers programmer can learn to competently do any construction work job (many of such jobs use tools and methods that required high IQ people to come up with...) while a mid or low IQ construction worker couldn't learn to competently do compilers programming. In other words, having higher intelligence makes it a lot easier to change job fields, not just find and work at any job at all. (If you can find me…

No, but I'd love to hear about it.

I see a lot of c++ devs complaining about how after 20 years they still haven't found a common ground on how to properly use c++ [1], its complicated memory management or its poor support for strings [2], things that are taken for granted with objc and Core Foundation. On one side I see an incredible amount of developers dealing with high amounts of complexity with C++, and on the other hand this low-level dynamic language with a powerful [3] open-source standard library [4] that is sadly only commonly used to build Cocoa apps. I'd love to hear about objc+CF for things like game development on other platforms than iOS or OSX, or an explanation to why it's not happening.

I really really don't mean to instigate a flame war here -- I'd just love to hear more about this topic.

[1] http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml

[2] http://benjamin.smedbergs.us/blog/2007-11-05/what-if/

[3] http://ridiculousfish.com/blog/archives/2005/12/23/array/

[4] http://en.wikipedia.org/wiki/Core_Foundation

Yeah, I think calling him the "unenlightened" one is pretty off base here.

For performing the tasks outlined by his examples, Unix utilities are easier for the user as well as executing faster than writing your own code in a general purpose programming language, unless one puts in the time to tune the implementation.

One could rebuild AWK in C and get similar performance, but why not just use some extremely simple AWK? And is anybody going to replicate the amazing speed of grep without a huge time investment? [1]

There is a right tool for the job, and having seen dozens of programmers be exposed to new big data sets, I can tell you that the ones who become productive quickly and stay more productive are the ones who adopt whatever tool is best for the job, not the ones that stick to their one favorite programming language. In fact, a good sign of somebody who will quickly fail is someone who says "forget those Unix tools, I'm just going to write this in X".

[1] http://ridiculousfish.com/blog/posts/old-age-and-treachery.html

the iOS framework lacks a built-in queue

The convention for this sort of thing is to use an NSArray, inserting at position 0 to enqueue and removing the last object to dequeue. The NSArray class is actually written in a way that makes this efficient (for some great analysis see http://ridiculousfish.com/blog/posts/array.html).

or a sorted list (as far as I know)

For sorted lists, I think you're right, you'd have to do it yourself.

The question a Cocoa old-hand might ask would be if you _really_ need it to be always-sorted, or if you can keep your objects in an array, dictionary or set and sort them before using or iterating - or maybe just iterate the collection in a specific order instead of sorting it at all.

If you do need to keep it sorted all the time, note that _inserting_ into an NSArray is also more efficient than you might assume. Keeping a sorted NSArray up to date yourself might not be expensive (it would be a bit more complicated than an auto-sorting class though, of course).

I didn't really want to engage with your broader point. I do disagree with it, but only in a measure of degree, and that's hard to debate because it's perspective and opinion oriented. If performance wasn't as important as you seem to imply, it's very unlikely that ripgrep would have gotten popular given that performance (and filtering) are its two more critical features. That is, that's what I hear from users the most: the like ripgrep because it's fast or because it filters or because of both.

But on the smaller point, it is just factually untrue that GNU grep falls into the "performance will be looked at later" bucket. That was why I responded.

This post is a classic about why GNU grep is fast, demonstrating that GNU grep definitely doesn't look like it wasn't optimized: https://lists.freebsd.org/pipermail/freebsd-current/2010-August/019310.html

And then before that, there's this: https://ridiculousfish.com/blog/posts/old-age-and-treachery.html