Pádraig Brady, a GNU coreutils maintainer, writes about Linux command-line tools, shell scripting, stdio buffering, and C/GCC programming details.
HN commenters point to his pages as useful references on stdio buffering, string buffer and integer overflow checks in GCC, and Linux command lists.
Read man pages.
For every command you run, assume there is a man page, and assume there is quick help, there usually and often are both.
$ man man # The man command has a man page about itself.
$ man -help # The man command has quick help.
A lot of this stuff is 40 years of accumulated and evolving conventions. Quick ref might not be available for a particular command, or might be found with -h, -help or --help.
Read the bash man page. Or whichever shell you're using, but if you haven't set it yourself it's probably bash.
The bash man page is pretty large, so don't read it all at once. Here's table of contents script that will give you a man page's table of contents. Make yourself a bin directory at /home/yourlogin/bin, then put it and other little tools that you write in there. Make them executable, and put that directory on your path. ("bin" is an old convention for "binary" or compiled programs. Ironically, most of what you put in there will not be binary. :)
$ cat ~/bin/toc
#! /usr/bin/env bash
if [[ $# == 2 ]]; then
man -S $2 -P "grep -E '^[^ ]'" $1
else
man -P "grep -E '^[^ ]'" $1
fi
$ toc time
TIME(1) General Commands Manual TIME(1)
NAME
SYNOPSIS
DESCRIPTION
OPTIONS
FORMATTING THE OUTPUT
EXAMPLES
ACCURACY
DIAGNOSTICS
AUTHOR
SEE ALSO
$ toc time 2
TIME(2) Linux Programmer's Manual TIME(2)
NAME
SYNOPSIS
DESCRIPTION
…
Here's my full bash reading list:
- ShellCheck: life-changing BASH linter and testing toolkit https://github.com/koalaman/shellcheck
- 30 interesting commands for the Linux shell – Víctor López Ferrando https://www.lopezferrando.com/30-interesting-shell-commands/
- 7 Surprising Bash Variables https://zwischenzugs.com/2019/05/11/seven-surprising-bash-variables/
- anordal/shellharden https://github.com/anordal/shellharden/blob/master/how_to_do_things_safely_in_bash.md
- barryclark/bashstrap https://github.com/barryclark/bashstrap
- BashPitfalls : Greg's Wiki http://mywiki.wooledge.org/BashPitfalls
- Common shell script mistakes http://www.pixelbeat.org/programming/shell_script_mistakes.html
- Comparison of all the UNIX shells http://hyperpolyglot.org/unix-shells
- Defensive Bash Programming https://www.kfirlavi.com/blog/2012/11/14/defensive-bash-programming/index.html or https://jonlabelle.com/snippets/view/markdown/defensive-bash-programming
- Bash FAQ and Cookbook https://mywiki.wooledge.org/BashFAQ
- Detecting the use of "curl | bash" server side …
Yep, just ran ps_mem.py (http://www.pixelbeat.org/scripts/ps_mem.py) and here's my entire server. Not a busy one, but it lets you see what a running api server and accompanying programs look like.
Private + Shared = RAM used Program
184.0 KiB + 31.5 KiB = 215.5 KiB atd
240.0 KiB + 55.0 KiB = 295.0 KiB cron
240.0 KiB + 68.0 KiB = 308.0 KiB upstart-socket-bridge
304.0 KiB + 72.0 KiB = 376.0 KiB upstart-udev-bridge
392.0 KiB + 79.0 KiB = 471.0 KiB sudo
696.0 KiB + 26.0 KiB = 722.0 KiB dhclient3
604.0 KiB + 189.0 KiB = 793.0 KiB getty (6)
940.0 KiB + 49.0 KiB = 989.0 KiB dbus-daemon
660.0 KiB + 366.0 KiB = 1.0 MiB udevd (3)
1.0 MiB + 71.0 KiB = 1.0 MiB rsyslogd
1.1 MiB + 35.5 KiB = 1.1 MiB redis-server
1.0 MiB + 122.5 KiB = 1.2 MiB init
964.0 KiB + 733.0 KiB = 1.7 MiB polkitd
1.4 MiB + 823.5 KiB = 2.2 MiB console-kit-daemon
2.5 MiB + 1.1 MiB = 3.6 MiB nginx (5)
1.3 MiB + 3.2 MiB = 4.6 MiB sshd (5)
5.4 MiB + 75.5 KiB = 5.5 MiB api_server <==== Go Program
13.8 MiB + 963.0 KiB = 14.7 MiB bash (2)
22.5 MiB + 314.0 KiB = 22.8 MiB mysqld
41.9 MiB + 5.1 MiB = 47.0 MiB python2.7 (2)
79.4 MiB + 542.0 KiB = 79.9 MiB java
---------------------------------
190.3 MiB
=================================
I learnt the same trick some years ago, from an article called Shell Scripts Matter:
https://dev.to/thiht/shell-scripts-matter
So I took some of the advice and tips offered in there, and wrote a template file to be used as a baseline when writing scripts for any project that might need one:
https://github.com/j1elo/shell-snippets/blob/master/template.sh
Other resources that I link in the readme of that repo, because they were a great guide to write better and more robust scripts, are:
- Writing Robust Bash Shell Scripts: https://www.davidpashley.com/articles/writing-robust-shell-scripts/
- Common shell script mistakes: http://www.pixelbeat.org/programming/shell_script_mistakes.html
- Bash Pitfalls: http://mywiki.wooledge.org/BashPitfalls
- The Bash Hackers Wiki: https://wiki.bash-hackers.org/
EDIT: -for anyone who would like to read some actual examples- I have to manage a bunch of scripts so actually a slightly more up to date version of the template is put into practice by means of a common bash.conf file that then gets sourced by all scripts: https://github.com/Kurento/adm-scripts/blob/master/bash.conf.sh
Another maintainer and I follow issues and pull requests on a GitHub mirror. But email works fine for us and many other projects.
Regarding poor UX, it is difficult to dispute with that claim without a specific example. Note that a lot of the features we support are standardized by POSIX. Even if we dislike the behavior, it is better to comply with the standards so the programs don't behave differently than users expect. The sentence you quote isn't meant to put down users. These programs are often much more complex than meets the eye, and there are lots of common gotchas that people have run into (and will continue to do so) [1].
Of course we would love for these programs to be useful for everyone. However, feature requests are often incompatible with existing behavior, incompatible with other feature requests, or have existing functionality elsewhere. For those reasons we cannot accept every feature request.
[1] https://www.pixelbeat.org/docs/coreutils-gotchas.html
I'll start by saying... I'm not a developer by any serious sense anymore. I did C over a decade ago and realized it was just not for me
With that out it the way... This rust infatuation seems like those people using one click tools to interpolate videos with "AI"
The end result is objectively worse and it's done because of subjective opinion: (gleefully) these artifacts are so SMOOTH!
Moving coreutils to rust is similar. The unit test coverage is low, and the problems they're seeing don't inspire confidence
Predictability in the tools is being traded for memory safety. I'd worry about using these behind any scripts, an extra \n from an obscure pipe driven bug could mean portability disaster
Edit:
For example, these quirks
https://www.pixelbeat.org/docs/coreutils-gotchas.html