Sematext, a monitoring and logging company, writes about Elasticsearch, Solr, Logstash, Docker monitoring, and eBPF.
HN commenters link it for Elasticsearch and Solr comparisons, X-Pack alternatives, and search trend data, especially in discussions of Elasticsearch licensing and security.
This is a good opportunity to update my search reading list.
Books
* Lucene in Action - A bit out of date, but it has a lot of the core concepts of Lucene. 10 years ago, this was the go-to book even if you’re working with Solr or Elasticsearch, to understand the core data structures
* Relevant Search (my book) - Introduction to optimizing the relevance of full text search engines. Getting a bit old now, but still relevant for classic search engines.
* AI Powered Search (disclaimer - I contributed) - Author Trey Grainger is brilliant, and has been a long-time colleague of mine. He managed the search team at Careerbuilder (where we did some work together). This is in some ways his perspective on how machine learning and search work together.
* Elasticsearch The Definitive Guide - online free, very comprehensive, book from Elastic
---
Blogs
* OpenSource Connections (my old company) Blog (http://o19s.com/blog) - lots of meaty search and relevance info
* Query Understanding (https://queryunderstanding.com/) - Daniel Tunkenlang is a long term very smart search person. Has worked at Google, etc
* James Rubenstein’s Blog (https://jamesrubinstein.medium.com/) - I worked closely with James at LexisNexis. He has helped work on search and search evaluation at Ebay, Pinterest, LexisNexis, and Apple
* Sematext (https://sematext.com/blog/) - Sematext are probably best known for being really in the weeds search engine engineers, a fair amount of scaling, etc. But some…
If you look for real FOSS ES security i recommend Search Guard
https://github.com/floragunncom/search-guard . This is a ASL2 licensed repo which provide all kinds of basic security for Elasticsearch including SSL/TLS. So the content in this repo are real fully "open source".
Then there is a second non-FOSS repository https://github.com/floragunncom/search-guard-enterprise-modules which contains all the enterprise/advanced stuff. The code is open but not "open source".
So FOSS and and non-FOSS/proprietary are well separated. I don't understand why elastic mix and merge FOSS and non-FOSS code in the same repo with a weird mixed license applicable for special folders. This all makes no sense to me.
And there are several other FOSS projects which provide x-pack like functionalities described here https://sematext.com/blog/x-pack-alternatives/ like elastalert for alerting or sentinl for reporting.
If one is really comitted and addicted to FOSS then i believe its better to go with the FOSS x-pack alternatives and support and contribute to them.
Just my 2c.
Hi snewman,
I am currently using logdna as my splunk alternative, but it's not very good.
Why should I try scalyr next, over the other alternatives ?
My main constraint is the solution should be at least half as good as splunk, and cost less than $50 a month (I generate less than 1G a day).
Your pricing seems to be reasonable. But how do you compare to splunk on features ?
Here are the other tools I am considering trying
https://www.humio.com/
https://www.sumologic.com/
https://sematext.com/logsene/
https://www.datadoghq.com/log-management/
https://logz.io/pricing/
https://www.chaossearch.io/
https://www.papertrail.com/
https://www.loggly.com/
https://www.graylog.org/products/enterprise
They are both reliable and they both have Elasticsearch outputs (recipe for rsyslog+Elasticsearch+Kibana here: http://blog.sematext.com/2013/07/01/recipe-rsyslog-elasticsearch-kibana/). I would stick with rsyslog if you only need to send syslog to Elasticsearch (maybe upgrade to a recent version, the ones from most distros are ancient). Logstash is more flexible and easy to use, so if I have something that rsyslog can't do or it's too messy, I would replace it with Logstash, or just install Logstash alongside rsyslog (normally, rsyslog is very light)
ES is not really a database :) most databases have some authentication features out of the box ... but true: for those kids who don't know how to block a port, configure nginx proxies etc.: https://sematext.com/blog/2017/01/18/elasticsearch-security-authentication-encryption-backup/
The most valuable part is about backups for ES indices - this could save you some bitcoins ;)
That's interesting to hear. Would love to read about how Vespa compares to Solr and ES.
This may be of interest to you: https://sematext.com/opensee/report/project/trend?q=ElasticSearch,Solr,Vespa,Lucene
Would you happen to know how Vespa compares to ES in terms of memory or CPU footprint? Have you done apples to apples comparison by any chance?