Jean Niklas L'orange writes on Clojure internals, including persistent vectors, hash maps, protocols, and event sourcing.
Commenters repeatedly link his persistent vector post as a good explanation of Clojure's tree-of-arrays implementation, plus the companion hash map piece.
On a related note, cache alignment is one of the reasons Clojure's vectors (log32 insertion, access, and deletion) and linked lists (o(index) insertion, access, and deletion) have unintuitive performance characteristics. For instance, constructing lists of either type takes roughly the same amount of time for big lists:
user=> (with-progress-reporting (bench (into '() (range 100000))))
Execution time mean : 9.293932 ms
Execution time std-deviation : 269.771284 µs
user=> (with-progress-reporting (bench (into [] (range 100000))))
Execution time mean : 9.882163 ms
Execution time std-deviation : 359.744662 µs
And with smaller lists, building vectors is ~30-50% slower.
user=> (with-progress-reporting (bench (into '() (range 24))))
Execution time mean : 2.483705 µs
Execution time std-deviation : 71.302962 ns
user=> (with-progress-reporting (bench (into [] (range 24))))
Execution time mean : 3.349080 µs
Execution time std-deviation : 114.007930 ns
However, traversal is significantly faster for vectors, because you can pack 32 references into a cache line at a time. Here's a decent-sized list:
user=> (let [x (apply list (range 100000))] (bench (reduce + x)))
Execution time mean : 8.586510 ms
Execution time std-deviation : 80.923357 µs
Vs a comparable vector:
user=> (let [x (vec (range 100000))] (bench (reduce + x)))
Execution time mean :…
The top level index is a clojure map (a persistent hash-map by default i believe) of strings (stems) to another map of key to weighting. The hash-map is implemented as a Bagwell Trie under the hood. Check out http://blog.higher-order.net/2009/09/08/understanding-clojures-persistenthashmap-deftwice/ for a lot of detail on the implementation, and http://en.wikipedia.org/wiki/Trie for a general overview of how Tries differ from Trees.
There's also a companion piece on PersistentHashMap based also on the idea of chunky bits:
http://blog.higher-order.net/2009/09/08/understanding-clojures-persistenthashmap-deftwice/
There's a good overview of how persistent vectors are implemented in Clojure here - http://blog.higher-order.net/2009/02/01/understanding-clojures-persistentvector-implementation/
Another approach: use a persistent data structure ala clojure's:
http://blog.higher-order.net/2009/02/01/understanding-clojures-persistentvector-implementation
I found a link to the implementation details, you are correct:
http://blog.higher-order.net/2009/02/01/understanding-clojures-persistentvector-implementation/