Datadog Blog

I'm a technical writer at Fastly and a published author with No Starch Press. I've worked full time as a technical writer since 2006. It's a great career and I highly recommend it.

If you have a technical background and you've coauthored a book, you already have what it takes to find full time work as a technical writer. There are really two ways into this field. You start with a background in English or professional writing and learn the technology, or you can start with a computer science degree or something similar and learn how to write. It sounds like you've done the latter.

The pay is good if you work for a company full time. Many technical writing jobs these days are remote. (All of the tech writers that work for Fastly and GitHub are remote, for example.) You could probably do freelance work, but I've never been able to make that work. Most companies don't outsource their documentation and I'm not entirely sure why, although I suspect it has something to do with them wanting to build long term relationships with the individual(s) documenting their products.

You will need to do some networking. You may want to consider attending Write the Docs (http://www.writethedocs.org/), a conference for technical writers. Depending on where you live, you might also be able to find an active STC group.

Or just start applying for jobs. :) Really good technical writers are hard to come by, so if you know technology and you can write, you'll probably have the job offers flooding…

Very cool, and very much what seems to be the trend for very high performance data systems: disaggregated storage. I think of Datadog's write-up for Husky, their newest/3rd gen data engine; https://www.datadoghq.com/blog/engineering/introducing-husky/

> By writing data directly to S3-compatible object storage, Bufstream can dispense with expensive disks and nearly eliminate inter-zone networking, slashing costs dramatically.

I am curious what the situation looks like for self-cloud-ers. If you own the servers, its less clear how much advantage there would be using s3 object store versus disk attached storage services. But reciprocally, getting good at Ceph and Ceph Object Gateway (their s3 compat) then being able to use and tune that storage knowledge at the platform level makes sense, versus having a separate storage service for x, y, and z.

Still, I think there is huge potential for something like Pulsar with BookKeeper tablets of data to rise. Our object stores don't seem to be excellent at data-locality, at replicating, and being able to tap that could yield some incredible systems efficiencies & speeds that object storage has abstracted away from us, that object storage has to brute force.

I'm curious what architecture flourishes there are with your latency tuning. Is this still object store based, or something different?

> For particularly latency-sensitive or latency-tolerant workloads, operators can tune how aggressively Bufstream trades latency for cost.

The…

Datadog | Full-time, Permanent Roles: Data Engineer | Software Engineer - Data Infra | Distributed Systems Engineer (ALL AVAILABLE IN REMOTE US/NYC/BOSTON)

1) Data Engineer: Looking for engineers to work on realtime (Kafka + Kafka Connect + custom Go consumers) and batch systems (Spark on EMR + Dataproc + Luigi + Parquet + S3 and GCS) to process 100's of TBs daily - at times PB-scale per day.
Apply here for BOSTON/NYC - https://grnh.se/f4127d5f1 - if REMOTE US, email bryan.hughes@datadoghq.com.

2) Software Engineer - Data Infra: Engineers in our Data Infra team build and own all infrastructure required to transport, process and store data at scale. This team also owns the system that allows teams to schedule and execute their batch jobs on multiple cloud platforms in multiple regions.
Apply here for BOSTON/NYC/REMOTE US - https://grnh.se/ef2ed6f51.

3) Distributed Systems Engineer: Help us build the high-throughput, low-latency systems that power our product. These systems ingest, store, analyze, and query tens of millions of events per second from companies all over the globe. Looking for experience in Go and Python, with bits of C or other languages. Apply here for BOSTON/NYC - https://grnh.se/866fe22c1 - if REMOTE US, email bryan.hughes@datadoghq.com.

More details about Datadog here: https://www.datadoghq.com/

Ilan with Datadog here.

On Friday, September 30th a number of Datadog-AWS shared customers received an erroneous email notification from AWS about compromised AWS access keys associated with Datadog. AWS intended to reach out to shared customers that had been lax in deactivating and deleting access keys associated with the 7/8/2016 Datadog breach (https://www.datadoghq.com/blog/2016-07-08-security-notice/). AWS accidentally sent out a standard compromise notification to the original list of shared customers. False positives abound! You can validate this error by contacting your AWS support contact.

As recommended in our original communication, if you have not you should deactivate and delete any service integration credential shared with Datadog on or prior to 7/8/2016 immediately and for AWS transition to Role Delegation as outlined here:
https://help.datadoghq.com/hc/en-us/articles/210376966-Revoking-AWS-keys-and-enabling-Role-Delegation-for-the-Datadog-AWS-Integration

The AWS communication was made in error. If you received the initial message you should have also received a retraction. Customers that have been lax in deactivating and deleting these access keys will receive additional communication directly from AWS.

Thank you very much for your comment - I've realized that caching DNS should be considered in any "connected" service.

I would add my two cents:

- Calculate checksum and compare it with the latest entry (storing single 32/40-bytes long checksum takes lower amount of memory).

- Respect various HTTP headers like Expires, Cache-control, Accept-Encoding (accepting compressed content may relieve strain on bandwidth), Content-length, Cookie-length etc.

- If you were downloading a web-page and sending it via e-mail (for later usage?) you may gzip it - it may reduce costs of attachments or S3.

- Attach header "User-Agent" and "From" (with an e-mail address to contact developer) - if you don't mind.

- As you wrote, avoiding ioutil.ReadAll should be a good practice - Datadog published a blogpost which is worth reading - https://www.datadoghq.com/blog/crossing-streams-love-letter-gos-io-reader/

- Use logrotate or implement it on your own with i.e. Mutex. Eventually you use fluentd, Kafka, Pub/Sub pipeline or some SaaS offer specialized in storing logs.

Most (or all) of Go packages with "request" in the name that I know doesn't cover aforementioned scenarios automatically - it is up to developer.

I agree, New Relic's pricing model is almost too "enterprise". Per host pricing seems high, and difficult to scale. It's too bad, because I love everything else they offer. I'm even considering the new Insights service. I think they need to introduce a mid-tier pricing level between Free and Pro.

They've got a good foothold in the market as a all-in-one solution. I've attempted to find a drop-in replacement, without much luck. It's difficult to find a one-for-one service, but here's some similar services with entry-level pricing:

DataDog (https://www.datadoghq.com/pricing/): $15/host/month up to 100 hosts.

Scout (https://scoutapp.com/subscriptions): $10/host/month

CopperEgg (http://copperegg.com/pricing/): A la carte depending on service

Stackify (http://www.stackify.com/pricing/): $15/host/month

The biggest feature that I need is application level monitoring for PHP. There's plenty of great server monitoring tools, but not many services that track application performance like New Relic does.

I know there's plenty of ways to replicate New Relic, but who has the time?

Of course, I could be completely wrong, and missing a great alternative service.