Andrew Moffat

You have to dig through the documentation a bit on the website but all the info is there[1]

from sh import mysqldump
from sh import gzip

gzip(mysqldump("example_db", _piped=True), _out="dump.sql.gz")

or if you don't want the magic import thing:

import sh
sh.gzip(sh.mysqldump("example_db", _piped=True), _out="dump.sql.gz")

It's definitely foreign to shell scripting languages since the piping syntax is a bit different & you have to remember to do `_piped=True` since it's not parallel by default. But the default behavior is pipefail & exit on the command failing IIRC so it's more of a choose your poison thing (do you want a subtle perf issue or a subtle bug in your script not handling error states correctly). And I find it easier to read. + if you want to customize anything about gzip or not rely on needing the binary in the path, then you can just switch it to Python-native gzip pretty easily.

That's my favorite feature. Conciseness if I'm just translating a script with progressive complexity options to migrate things that need more complexity or different requirements within the same script without having to rewrite it from scratch.

[1] https://amoffat.github.io/sh/

Note that this recommends "sh" package, which made an _interesting_ decision to run processes in tty by default. This goes against unix conventions that batch processes should not be using tty, and breaks a number of utilities like git and systemctl. This is so bad that even their example from the front page is broken(!):

$ python3
>>> import sh
>>> sh.git.show('HEAD').count('\n')
49
# Now resize your terminal window to be 10 lines tall
>>> sh.git.show('HEAD').count('\n')
9

Yep, it truncated "git show" output to the height of your terminal window. Do you want this in your code?

While you can append special flag for this, it is very easy to forget to do so, and the documentation does not help either. Their FAQ states that they are aware of the issue and won't fix it [0].

So my advice is: stay away from "sh" package, as it makes it very simple to generate unreliable code. Instead, use built-in tools which are pretty nice and just a bit more verbose:

>>> import subprocess
>>> subprocess.check_output('git show HEAD', shell=True).decode().count('\n')
49

[0] https://amoffat.github.io/sh/sections/faq.html#why-is-tty-out-true-the-default

On the contrary, I am 100% open to suggestions that BASH sucks. (Personally, I would have picked the inadequacies of `set -e` as my favorite ... unfortunate aspect.) My objection was very narrow: this specific article does a poor job at saying why shell sucks, and a worse job still at presenting Python as a superior alternative. That's not to say that good reasons to use something else don't exist, and it's not even to say that Python can't be that alternative. (Your sibling comments suggesting xonsh and https://amoffat.github.io/sh/ paint a fairly compelling way to actually move to python without losing the advantages of shell, for instance.)

Great, would that work with pipes and redirections too? At that point you might as well use sh[1] which is a more advanced abstraction. My issue with these approaches is that you'll inevitably hit a bug or limitation where just using a shell language would've been easier to maintain and more portable.

Python has its uses, but replacing shell scripts is not one of them IMO. It it was then the problem didn't need to be a shell script to begin with (doesn't shell out often, needs complex data structures, etc.).

[1]: https://amoffat.github.io/sh/

If you mostly want to query and can live without the VFS, dogsheep[1] is your friend. It's a general tool to import lots of different data types into a personal sqlite instance, and dogsheep-photos[2] both extracts image metadata and uploads all the pics to S3 if you'd like.

On my to-try list, there's also supertag[3], a tag-based filesystem that's mounted via FUSE

[1] https://dogsheep.github.io/ [2] https://github.com/dogsheep/dogsheep-photos [3] https://amoffat.github.io/supertag/