Technotes

Technotes for future me

Best practices

Bash best practices

An attempt to bring order in good advice on writing Bash scripts I collected from several sources.

General

  • Always use long parameter notation when available. This makes the script more readable, especially for lesser known/used commands that you don’t remember all the options for.

    # Avoid:
rm -rf -- "${dir}"

# Good:
rm --recursive --force -- "${dir}"

  • Don’t use:

    cd "${foo}"
[...]
cd ..

    but

    (
  cd "${foo}"
  [...]
)

Conditionals

Should we use single or double square brackets in conditionals? What’s the difference between if [ "${var}" -le 0 ] and if [[ "${var}" -le 0 ]]?

They’re mostly equal, but double square brackets usually provide cleaner syntax and a few more additional features. Compare this:

[ -f "$file1" -a \( -d "$dir1" -o -d "$dir2" \) ]

and this:

[[ -f $file1 && ( -d $dir1 || -d $dir2 ) ]]

With double square brackets you don’t need to escape parenthesis and unquoted variables work just fine even if they contain spaces (meaning no word splitting or glob expansion).

Variables

Double quotes around every parameter expansion
$ song="My song.mp3"

Bash allows for a limited form of variable annotations. The most important ones are: local (for local variables inside a function) readonly (for read-only variables)

  • Prefer local variables within functions over global variables
  • If you need global variables, make them readonly
  • Variables should always be referred to in the ${var} form (as opposed to $var.
  • Variables should always be quoted, especially if their value may contain a whitespace or separator character: "${var}"
  • Capitalization:
    • Environment (exported) variables: ${ALL_CAPS}
    • Local variables: ${lower_case}
  • Positional parameters of the script should be checked, those of functions should not
  • Some loops happen in subprocesses, so don’t be surprised when setting variabless does nothing after them. Use stdout and greping to communicate status.

Substitution

  • Always use $(cmd) for command substitution (as opposed to backquotes)

  • Prepend a command with \ to override alias/builtin lookup. E.g.:

    $ \time bash -c "dnf list installed | wc -l"
5466
1.32user 0.12system 0:01.45elapsed 99%CPU (0avgtext+0avgdata 97596maxresident)k
0inputs+136outputs (0major+37743minor)pagefaults 0swaps

Output and redirection

  • For various reasons, printf is preferable to echo. printf gives more control over the output, it’s more portable and its behaviour is defined better.

  • Print error messages on stderr. E.g., I use the following function:

    error() {
  printf "${red}!!! %s${reset}\\n" "${*}" 1>&2
}

  • Name heredoc tags with what they’re part of, like:

    cat <<HELPMSG
usage $0 [OPTION]... [ARGUMENT]...

HELPMSG

https://linuxize.com/post/bash-heredoc/

  • Single-quote heredocs leading tag to prevent interpolation of text between them.

    cat <<'MSG'
[...]
MSG

  • When combining a sudo command with redirection, it’s important to realize that the root permissions only apply to the command, not to the part after the redirection operator. An example where a script needs to write to a file that’s only writeable as root:

    # this won't work:
sudo printf "..." > /root/some_file

# this will:
printf "..." | sudo tee /root/some_file > /dev/null

Functions

Bash can be hard to read and interpret. Using functions can greatly improve readability. Principles from Clean Code apply here.

  • Apply the Single Responsibility Principle: a function does one thing.

  • Don’t mix levels of abstraction

  • Describe the usage of each function: number of arguments, return value, output

  • Declare variables with a meaningful name for positional parameters of functions

    foo() {
  local first_arg="${1}"
  local second_arg="${2}"
  [...]
}

  • Create functions with a meaningful name for complex tests

    function name has an underscore as a prefix. It seems like a good idea to always have a special naming convention for your bash functions to avoid any potential clashes with built-in operators or functions you might include from other files

    # Don't do this
if [ "$#" -ge "1" ] && [ "$1" = '-h' ] || [ "$1" = '--help' ] || [ "$1" = "-?" ]; then
  usage
  exit 0
fi

# Do this
_help_wanted() {
  [ "$#" -ge "1" ] && [ "$1" = '-h' ] || [ "$1" = '--help' ] || [ "$1" = "-?" ]
}

if _help_wanted "$@"; then
  usage
  exit 0
fi

Cleanup code

An idiom for tasks that need to be done before the script ends (e.g. removing temporary files, etc.). The exit status of the script is the status of the last statement before the finish function.

finish() {
  result=$?
  # Your cleanup code here
  exit ${result}
}
trap finish EXIT ERR

Source: Aaron Maxwell, How “Exit Traps” can make your Bash scripts way more robust and reliable.

Writing robust scripts and debugging

Bash is not very easy to debug. There’s no built-in debugger like you have with other programming languages. By default, undefined variables are interpreted as empty strings, which can cause problems further down the line. A few tips that may help:

  • Always check for syntax errors by running the script with bash -n myscript.sh

  • Use ShellCheck and fix all warnings. This is a static code analyzer that can find a lot of common bugs in shell scripts. Integrate ShellCheck in your text editor (e.g. Syntastic plugin in Vim)

  • Abort the script on errors and undbound variables. Put the following code at the beginning of each script.

    set -o errexit   # abort on nonzero exitstatus
set -o nounset   # abort on unbound variable
set -o pipefail  # don't hide errors within pipes

    A shorter version is shown below, but writing it out makes the script more readable.

    set -euo pipefail

  • Use Bash’s debug output feature. This will print each statement after applying all forms of substitution (parameter/command substitution, brace expansion, globbing, etc.)

    • Run the script with bash -x myscript.sh
    • Put set -x at the top of the script
    • If you only want debug output in a specific section of the script, put set -x before and set +x after the section.

  • Write lots of log messages to stdout or stderr so it’s easier to drill down to what part of the script contains problematic code. I have defined a few functions for logging, you can find them in my dotfiles repository.

  • Use bashdb

Shell script template

An annotated template for Bash shell scripts:

For now, see https://github.com/bertvv/dotfiles/blob/master/.vim/templates/sh

Resources

Templates

Portable shell scripts

Fun

Last updated on 31 Jan 2021
Published on 24 Mar 2020
Edit on GitHub