# Introduction * Shell scripting is easy, but also risky * So, so easy * So, so risky * Interactive use is off-topic for this talk ----------------------------------------------------------------------------- # Hashbang OK, less powerful than bash, but more portable: ~~~sh #!/bin/sh ~~~ OK on Linux, not everywhere: ~~~sh #!/bin/bash ~~~ Portable, but might invoke a different bash: ~~~sh #!/usr/bin/env bash ~~~ ----------------------------------------------------------------------------- # Error prevention and handling **Always, always, without exception, use this:** ~~~sh set -eu ~~~ If it's OK for a command to fail: ~~~sh this.may.fail || true ~~~ If using bash, optionally also use these: ~~~sh set -o pipefail set -o noclobber ~~~ ----------------------------------------------------------------------------- # No options in hashbang ~~~sh #!/bin/sh set -eu ~~~ **Never, never, ever:** ~~~sh #!/bin/sh -eu ~~~ Because: ~~~sh $ sh -x foo.sh ~~~ -x is a useful debugging option for sh. ----------------------------------------------------------------------------- # Escaping, quoting, variable expansion **Always, always** quote all variable expansions: ~~~{.sh .numberLines} $ foo="/ home/liw" $ print-argv rm -f $foo [0]: /usr/bin/print-argv [1]: rm [2]: -f [3]: / [4]: home/liw $ print-argv rm -f "$foo" [0]: /usr/bin/print-argv [1]: rm [2]: -f [3]: / home/liw $ ~~~ _"If your script needs to use $, switch to Python/Ruby/Perl"_—liw ----------------------------------------------------------------------------- # print-argv ~~~{.python .numberLines} #!/usr/bin/python3 import sys for i, arg in enumerate(sys.argv): sys.stdout.write('[%d]: %s\n' % (i, arg)) ~~~ ----------------------------------------------------------------------------- # Use $(...) instead of backticks. But be careful. ~~~sh $ touch 'foo bar' $ print-argv foo* [0]: /usr/bin/print-argv [1]: foo bar $ print-argv $(ls foo*) [0]: /usr/bin/print-argv [1]: foo [2]: bar $ print-argv "$(ls foo*)" [0]: /usr/bin/print-argv [1]: foo bar $ ~~~ ----------------------------------------------------------------------------- # Parameter expansion: $@ vs @* ~~~{.sh .numberLines} #!/bin/sh set -eu echo dollar-at for x in $@; do echo "<$x>"; done echo; echo dollar-at in quotes for x in "$@"; do echo "<$x>"; done echo; echo dollar-star for x in $@; do echo "<$x>"; done echo; echo dollar-star in quotes for x in "$@"; do echo "<$x>"; done ~~~ ----------------------------------------------------------------------------- ~~~{.sh .numberLines} $ touch 'foo bar' $ ./shell-param.sh foo* dollar-at dollar-at in quotes dollar-star dollar-star in quotes $ ~~~ ----------------------------------------------------------------------------- **Always, always, without exception** use `"$@"` **Never, never, ever** use `$@` or `$*` without quotes exception granted if you can explain why it's safe in a specific case, and you do it in a comment but use Python, Ruby, or Perl instead, seriously ----------------------------------------------------------------------------- # Command line option parsing in shell Don't. Use Python, Ruby, or Perl instead. ----------------------------------------------------------------------------- # Shell functions ~~~sh foo() { local arg1 arg1="$1" } ~~~ Avoid: `local arg1="$1"` because it interacts badly with pipefail. Avoid: `function foo {}` because that's an unnecessary bashism. ----------------------------------------------------------------------------- # Temporary files ~~~{.sh .numberLines} #!/bin/sh set -eu cleanup() { rm -rf "$tempdir" } tempdir="$(mktemp -d)" trap cleanup EXIT echo foo > "$tempdir/foo.txt" echo "$tempdir" ~~~ ----------------------------------------------------------------------------- ~~~{.sh .numberLines} $ mkdir t $ TMPDIR=t ./shell-temp.s t/tmp.EgiOqQ9SAU $ find t t $ ~~~ ----------------------------------------------------------------------------- # Summary * Know the rules. * The rules are intricate and hard to remember all the time. * You _will_ fail. * Give up and use a better language. ----------------------------------------------------------------------------- # Further reading ----------------------------------------------------------------------------- # Legalese Copyright 2020 Wikimedia Foundation This content is licensed under the Creative Commons Attribution-ShareAlike 4.0 International ([CC BY-SA 4.0][]) licence. [CC BY-SA 4.0]: https://creativecommons.org/licenses/by-sa/4.0/ --- title: "Shell scripting" subtitle: "some opinionated dos and dont's" author: "Lars Wirzenius / Wikimedia Foundation" date: "Version for 2020-05-18" ...