Shell variable substitution

Fancy ways to play with shell variables

for sh, bash, and zsh

by Patrick Horgan

(Back to scripting tutorials.)

Introduction

I'm going to talk about the kind of shell variable manipulations that are the bread and butter of shell programmers. I'll tell you what works everywhere and what doesn't. When you're done reading this, you'll be doing things easily that you didn't even know could be done. In particular, you'll know the difference between ${1+"$#"}, ${1:+"$#"}, ${1-"$#"}, and ${1="$#"}.

Who is this for?

This is for people who want to use a shell, either from the command line or in a script (script is what programs for shells are called). If you do, you'll need to know how variables work.

What do you need?

This tutorial assumes that you have access to a POSIX shell command line of some form, either the Bourne Shell, sh, written by Steve Bourne circa 1977, the Bourne Again SHell, bash, written by Brian Fox for the GNU project circa 1989, (my favorite), or zsh, named after the login name of then Princeton TA and now Yale professor Zhong Shao by zsh author Paul Falstad circa 1990 while a student at Princeton. If you want to write/edit scripts, I also assume that you know how to use some text editor.

How will it look?

Most examples will be shown as if typed from the bourne shell command line. It uses a bare $ for the prompt. Any line starting with $ is a line I typed (except for the $) as input to the shell, and any line starting without the $, is the response of the shell.

$ echo Mary had a little lamb <=- I type this Mary had a little lamb <=- The shell types this

Basic variables

We'll start off with basic use of variables.

When to use the $

In shell scripting you give a value to variables using the variable's bare name, and you get the value back out of variables by putting a $ in front of the variable's name. Using the shell command line, we can demonstrate that.

$ foo="value" $ echo $foo value $

First the variable foo is assigned the string "value", and then we use the shell builtin command echo to echo the value back to the screen. The take-away from this, is that we assign to the variable foo without a $ and then use the $ in front of the variable name ($foo) to access the current value of it.

Do we need braces AND suspenders? or: When do we use ${var} instead of $var

We can put a left brace {, and a right brace } around the variable name to let the shell know where the variable ends. Here, I'll show you the problem.

$ foo=truck $ bar=fire$foo $ echo $bar firetruck $

If we want to concatenate a string with the value of a variable like we do here with "fire" and $foo, the shell can tell where the variable starts because it finds the $.

If we try to do it the other way around, it won't work

$ foo=truck $ bar=$foofire $ echo $bar $

The shell can't tell that we're not trying to use a variable named $foofire, and sets the value of bar to the empty string, because $foofire has no value.

The braces can come to our rescue here.

$ foo=truck $ bar=${foo}fire $ echo $bar truckfire $

If we put the braces around the variable name, the shell can see that we meant to use the value of the variable foo.

As we'll see in a minute, there are also other things you can do inside the braces that aren't available to you when you don't use the braces.

(Back to scripting tutorials.)

Parameter Expansion

Technically, everytime you access the value of a variable with $ you are using parameter expansion. The $var expands to the value of the variable. Along the way, other more complicated things can happen, and these things are what are usually referred to as parameter expansion.

Basic parameter expansion 101 - sh parameter options

The original sh supported a limited set of parameter options,

${var-word} # if var is defined, use var; otherwise, "word" ${var+word} # if var is defined, use "word"; otherwise, nothing ${var=word} # if var is defined, use var; otherwise, use "word" AND... # also assign "word" to var ${var?error} # if var is defined, use var; otherwise print "error" and exit