The Dagger team, founded by people behind Docker, writes about defining CI/CD pipelines and dev environments as code through language SDKs.
HN commenters praise Dagger as a better approach than Dockerfiles and a Make alternative for CI pipelines. Some comments come from a Dagger employee.
I support the effort to build a platform-agnostic CI/CD pipeline solution, but I don't want it in the form of yet another platform. Rather it needs to be a protocol that any platform can tie in to. I'm especially wary since this is another VC-backed effort that will eventually need to be monetized in some shape or form.
Additionally, as someone else here has already mentioned, my mind first went to Dagger, the dependency injection tool (https://dagger.dev). That tool in particular was named as a play on DAG (directed acyclic graphs), whereas in this case I don't think it would apply since there may be instances where you'd want cycles in a pipeline.
On a whim, I clicked on "Trademark Guidelines" (https://dagger.io/trademark) and from that page alone I would recommend avoiding this based on the aggressive language used to try and claim ownership of generic words. According to their own language, it seems I'm violating their guidelines by writing this comment.
> Our Marks consist of the following registered, unregistered and/or pending trademarks, service marks and logos which are subject to change without notice: Dagger; Blocklayer; and other designs, logos or marks which may be referred to in your specific license agreement or otherwise.
> Blocklayer does not permit using any of our Marks ... to identify non-Blocklayer products, services or technology
Which would include Dagger, the dependency injection tool.
Other sections of note:
> Do Not Use As Nouns
(This one…
This is fascinating. It checks a lot of my boxes. I was recently looking at ActionsFlow [0] which is similar but runs on GitHub Actions.
My thoughts -
1. I don't see proper secret storage being handled. You typically don't want your API keys in your code. What would you recommend instead?
2. "OAuth" based secrets. Many integrations require giving access to an App via OAuth, which involves a flow. I think that's being handled internally from the video [1] and from this project [2], but it's not clear. How is that handled?
A common use case I'd automated once is that when a GitHub project gets starred, the developers public information is scrapped and they are then followed on Twitter, if their twitter handle is found. With Trigger.dev, the twitter part isn't clear.
3. Error Handling - What about when some job fails to run? I understand there is a delay mechanism. But what about injecting custom error handling? Sending a message on slack, for example.
4. Dashboards - They look awesome. And I get the impression that each "action" in the code is mapped to individual blocks in the dashboard. I'd love to be able to see a proper graph of the flow.
I love that I can see the json request / response for each. It'll make debugging easier when some API changes or fails.
5. No Code solutions - In the long run, I can easily picture writing the integration I want in plain text, and having Github CoPilot or ChatGPT generate the code for me, and then I can quickly modify it.
6.…
> These alternatives are far more familiar and easier to work with. You can get most (but not all) of the benefits of NixOS with far less work.
> Worse is better.
There are some potential alternatives out there that seem inspired by Nix but which are less strict. I think a system that offers and manages Nix's purity but doesn't require it could end up the final winner some day. Shea Levy had some ideas for eventually achieving this in Nix that he called 'Nomia' (probably riffing on the name of Sander van der Burg's Dysnomia, I'd guess), but his life circumstances changed before the project really materialized. But Nix still could be the place where that is first achieved.
Meanwhile new container image-building DSLs like Dagger¹ and Bass² look a lot like a Nix which is impure by default, and bash builders are replaced with BuildKit directives. And there's Denxi³, which looks more like a Nix that lets you reduce its purity when you want to.
For now, though, the Nix ecosystem is still vastly more capable than these alternatives because of all the incredible work that Nixpkgs represents. Nix we know it today could still end up the concrete basis for what succeeds it, and not just an inspiration.
1: https://dagger.io/
2: https://bass-lang.org/
3: https://docs.racket-lang.org/denxi-guide/
One primary design goal togomak had from the beginning was concurrency. All tasks run concurrently, unless a `depends_on` argument is mentioned. `just` didn't support that when I was initially building togomak, but there is a feature coming in soon which I am looking forward to: https://github.com/casey/just/pull/1562 .
While I was building togomak, I read through Dagger [1], Earthly [2], Concourse CI [3], Jest and Make along with the stuff I was already working with - Jenkins, GitHub actions and GitLab CI. Dagger [1] is really great, I like its design - it supports writing pipelines in Python, Typescript, Go and a few more languages. togomak tries to abstract away a lot of it. Such as dependency management (in the case of python, the requirement of a python interpreter, and its package managers, etc). togomak is just a single statically-linked binary.
[1]: https://dagger.io/
[2]: https://earthly.dev/
[3]: https://concourse-ci.org/
> GHA has "service containers", but unfortunately the feature is too basic to address real-world use cases: it assumes a container image can just … boot! … and only talk to the code via the network. Real world use cases often require serialized steps between the test & the dependencies, e.g., to create or init database dirs, set up certs, etc.)
My biased recommendation is to write a custom Dagger function, and run it in your GHA workflow. https://dagger.io
If you find me on the Dagger discord, I will gladly write a code snippet summarizing what I have in mind, based on what you explained of your CI stack. We use GHA ourselves and use this pattern to great effect.
Disclaimer: I work there :)
I'd encourage anyone thinking of using make to look at alternatives. Make is great, but is quickly becomes a ball of duct-tape. Make works very well when you spend the time to express your dependency tree, but realistically that never happens and people tend to add hacks upon hacks for Makefiles. Not only that, but they don't scale well as your project adds more components, such as integration testing, documentation, etc.
I found Earthly[0] to be a great replacement. Everything runs in Docker. Your builds are reproducible, cache-able, and parallelizable by default. I've heard Dagger[1] is another good tool in the space.
[0]: https://earthly.dev/
[1]: https://dagger.io/