The engineering team at Yammer writes about its Scala use, deployments, HBase testing, and engineering culture.
HN commenters cite its Scala posts in debates over the language, discuss the HBase availability results, and post Yammer job ads.
@codahale, was this email from you or someone else in your company?
Judging from your blog post a year ago at http://eng.yammer.com/blog/tag/scala, at one point you thought Scala was great. As someone who still thinks Scala is great, I would like to know how long it took you to change your mind and what triggered you to reach this realization. When it comes down to it, I've never developed Scala as part of a team or worked on a project more than a few thousand lines of code, so I definitely lack the experience you probably have with the language and runtime.
As to the email itself, I find the complaints about performance most uninsightful, and I say this as someone who has written CPU-intensive NLP code in Scala. Anyone who writes performant code to run on the JVM knows that the only way to get C-like performance in Java is to, it turns out, write C-like code in Java!
That means operating only on primitives and arrays of primitives when in tight loops. It means avoiding allocations as much as possible. It even means avoiding generics like the plague, including java.util.HashMap. In my NLP code, for instance, I got at least a 100x speed up by switching from HashMaps with string keys to arrays by manually interning each string into an incremented number. One library I used, called Mallet, relied on GNU trove to have hash maps without boxing of primitives, because for high performance, even java.util.HashMap is too slow!
This means that, if you want amazing performance in…
San Francisco, CA - Yammer. Fulltime; H1Bs considered.
Yammer is an awesome tech startup masquerading as an enterprise software vendor. We're building an enterprise social network (an internal communication and collaboration tool. Like Facebook.) and we need your help.
We build our product with insight and wisdom gained from the consumer social networking space, but we charge enterprise prices (and our customers pay them!) We're fighting some big serious competitors (Salesforce/Chatter, Jive, and VMWare/Socialcast) and this space has never been more exciting.
Yammer is really changing the way people get work done. We have actual challenges to overcome and we're doing our best to make a kick-ass product that makes our users happy.
Tech we use: Ruby/Rails, Scala, and JavaScript. We have Obj-C and C# stuff too.
Some bullet points for you to skim:
• Amazing group of smart engineers to work with. Really.
• We hack in Ruby/Rails, Scala/JVM, Javascript/JQuery/Node.js/Adobe AIR, Obj-C for iOS, and some MSFT/Sharepoint stuff.
• Competitive compensation. Enough said.
• Delicious catered lunch and dinner daily, with a 3PM snack cart. Also a fully-stocked beer-and-beverage fridge.
• Fancy Apple hardware of your choice (you can have a PC if you really want one.)
Some links for you to read more:
• Our jobs page: https://www.yammer.com/jobs
• Our Engineering blog: http://eng.yammer.com/
• A blog post about why it's so awesome to work here:…
These are some of the top sites' engineering blogs which I collected over a period of time - I removed those which have already been posted:
Google: http://googledevelopers.blogspot.com/ http://google-engtools.blogspot.com/ http://googleresearch.blogspot.com/
YouTube: http://youtube-eng.blogspot.com/
Baidu: http://usa.baidu.com/category/baidutechblog/
Yahoo: http://yahooeng.tumblr.com/ http://yahoodevelopers.tumblr.com/
Wikipedia (closest I could find): https://blog.wikimedia.org/c/technology/
Amazon: https://aws.amazon.com/blogs/aws/
Twitter: https://engineering.twitter.com/
eBay: http://www.ebaytechblog.com/
Microsoft: http://microsoftengineering.com
FourSquare: http://engineering.foursquare.com/
Apple (closest): https://developer.apple.com/
Paypal: https://www.paypal-engineering.com/ https://devblog.paypal.com/
Dropbox: https://blogs.dropbox.com/tech/
Edit: More (collected from this link: http://www.quora.com/What-are-the-top-startup-engineering-blogs):
Yelp: http://engineeringblog.yelp.com/
Quora: http://engineering.quora.com/
Yammer: http://eng.yammer.com/blog/
AirBnB: http://nerds.airbnb.com/
Square: https://corner.squareup.com/
Flickr: http://code.flickr.net/
Soundcloud: https://developers.soundcloud.com/blog/
Cloudera: http://blog.cloudera.com/blog/
Slideshare: http://engineering.slideshare.net/
Heroku: http://engineering.heroku.com/
Pinterest: http://engineering.pinterest.com/
You are just taking one data point and assuming it proves something other than Yammer had problems with Scala. Check this list of companies using Scala:
http://www.scala-lang.org/old/node/1658.html
Some examples: Twitter, LinkedIn, Foursquare. All of them pretty big and in the same space of Yammer; they seem quite happy and don't plan to move away from Scala.
And here you can find a clarification from Yammer about that issue:
http://eng.yammer.com/scala-at-yammer/
TL;DR: They liked Scala but it has rough edges, every programming language has rough edges (yes, including Java). They found Scala was not the best option for some particular situations and Java was a better fit. End of story.
That's fine and completely reasonable, I think nobody said Scala is the Holy Grail in programming languages. As for your preference, you seem to be looking for a "nicer than Java, but OOP all the way" (don't know about Julia, haven't used it), which is OK as a preference but many of us find it a kludge to work with. I found something like Scala (multi-paradigm, I mean) helps me a lot in reasoning about code. As I said, you use the paradigm that makes more sense.
For complexity and ugliness, you have my previous answer. About compile time, you have a sophisticated type system and type inference which come at a cost, but it's not such a big deal when you run it with fsc (which stands for fast scala compiler).
Yammer - San Francisco, London
At Yammer, our mission is to change the way people work, and that mission starts with us. We use our own product every day to promote and encourage our culture of openness and transparency. Yammer provides each user with a voice, empowering individuals to share ideas, ask questions, and voice concerns. We're passionate about building a great product that people love to use, and we're looking for similarly product-minded engineers to join us.
We work with all kinds of languages (Java, Ruby, Javascript, to name a few) and technologies (anything from Postgres, Riak, and Berkley DB to Rails, Dropwizard, and Backbone). We're also never afraid to try new things but not just for the sake of trying new things.
We've got all the standard perks: free food, free booze, lots of dogs, and amazing benefits.
Check our our blog or shoot me an email (mduncan@yammer-inc.com) - I'd love to talk.
http://eng.yammer.com/blog/
Note that Robert Yokota's addendum[1] points out that HBase "cannot achieve both consistency and availability." His earlier results did not take into account that HBase clients continuously retry failed ops. (According to Nicolas Liochon, upon its death a server's regions are moved to another node.) Failures start rolling in once network partition(s) extend beyond the configured timeout. Kyle [2] was not impressed:
"During the network partition, no requests are successful" is not the best result for a CP system, IMO."
HBase should provide partial availability in the face partitions.
[1] http://eng.yammer.com/call-me-maybe-hbase-addendum/
[2] https://twitter.com/aphyr/status/509841011816665088