What’s in a Name? Variable Scope and Closure
Let’s start with a riddle, what will the following print?
func main() {
for i := 0; i < 3; i++ {
go func() {
fmt.Print(i, " ")
}()
}
}
This program will won’t print anything since Go does not wait for goroutines before terminating the program. But that’s not I want to ask ☺ Let’s add a time.Sleep and try again:
func main() {
for i := 0; i < 3; i++ {
go func() {
fmt.Print(i, " ")
}()
}
time.Sleep(100 * time.Millisecond)
}
This time, you’ll see an output: 3 3 3. Not the expected random order of 0 1 2.
If you run go vet on the code, it will emit the following error: loop variable i captured by func literal. Most modern IDEs (such as my trusted Vim) will show this as a warning in the code.
But what does “captured by func literal” mean? Before we answer this question, let’s look at how Go resolves name in your program. Say we have the following function:
var scale = 1.1
func scaledMean(values []float64) float64 {
sum := 0.0
for _, val := range values {
sum += val
}
return sum / float64(len(values)) * scale
}
In the last line, we have several variables, each comes from a different scope.
sumis a local variablevaluesis a function argument, which can be seen as a local variablelenandfloat64are pre-declared identifiers (coming from the builtin package)scalecomes from the package scope
If we look at it visually, it might look like:
The Go documentation states:
Go is lexically scoped using blocks:
The scope of a predeclared identifier is the universe block. The scope of an identifier denoting a constant, type, variable, or function (but not method) declared at top level (outside any function) is the package block. The scope of the package name of an imported package is the file block of the file containing the import declaration. The scope of an identifier denoting a method receiver, function parameter, or result variable is the function body. The scope of a constant or variable identifier declared inside a function begins at the end of the ConstSpec or VarSpec (ShortVarDecl for short variable declarations) and ends at the end of the innermost containing block. The scope of a type identifier declared inside a function begins at the identifier in the TypeSpec and ends at the end of the innermost containing block.
When Go start to look for the value of an identifier, it starts from the bottom-most block and goes up. Finally, if it can’t find a block that declares a name, the compilation will failed with undeclared error.
All of this might be interested, but how does it help us? Let’s look at the code again:
func main() {
for i := 0; i < 3; i++ {
go func() {
fmt.Print(i, " ")
}()
}
time.Sleep(100 * time.Millisecond)
}
In the fmt.Print line, the i comes from the for loop. It’s the same i for all goroutines, this i has the value of 3 at the end of the for loop.
How can we fix this? There are two solutions, one easy to understand and one … less easy :)
Let’s start with the easy one:
func main() {
for i := 0; i < 3; i++ {
go func(n int) {
fmt.Print(n, " ")
}(i)
}
time.Sleep(100 * time.Millisecond)
}
We added a parameter to the goroutine. Parameters are evaluated when we call the goroutines, and since in Go everything is passed by value - each goroutines get’s it’s own copy of i as the n parameter. This is the reason that goroutines (and defers) are written as function calls.
For the second solution, you need to recall (or learn?) that every time you write a { you start a new scope. For example:
func main() {
x := 1
{
x := 7
fmt.Println("inner", x)
}
fmt.Println("outer", x) // 2
}
We say the inner x “shadows” the outer x. This might have lead to some interested bugs, but in our case we can use it as another option to solve our captured by func literal issue:
func main() {
for i := 0; i < 3; i++ {
i := i
go func() {
fmt.Print(i, " ")
}()
}
time.Sleep(100 * time.Millisecond)
}
The line i := i is confusing. When you write an assignment, Go will first evaluate the right side of the :=. In this case it’ll look for i and find it in the for loop scope. Then it’ll assign the current value of i to the i declared inside the for loop (which shadows the for loop i). Now when the compiler look for the value of i inside the goroutine, it’ll find the inner i and use it.
Closures
When the goroutine runs, it looks for the value of i and find it in the environment the created the function. This is known as closure. Closure variables are not read only, you can write code like the following:
func makeAccount(balance int) func(int) int {
fn := func(amount int) int {
balance += amount
return balance
}
return fn
}
func main() {
acct := makeAccount(100)
fmt.Println(acct(100))
fmt.Println(acct(-50))
}
This code will print 200 and then 150. This style of programming is not common in Go, but you might see it in HTTP middleware. Some programming circle claim that “objects are merely a poor man’s closures”, I don’t think Go hangs around in these circles :)
In some cases you will mix goroutine parameters and closure variables. For example, let’s replace the time.Sleep in original code with sync.WaitGroup:
func main() {
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func(n int) {
defer wg.Done()
fmt.Print(n, " ")
}(i)
}
wg.Wait()
}
The goroutines gets n as a parameter and uses wg from the closure. Here we use scoping rules to our advantage and create short and concise code.
Named Return Values
Go has named returned values. They are mostly used when handling with panics
func safeDiv(a, b int) (result int, err error) {
defer func() {
// err will be non-nil only if there's a panic
if e := recover(); e != nil {
err = fmt.Errorf("div error: %s\n", e)
}
}()
return a / b, nil
}
In the if statement, we assign a value to err just like any local variable. Try to change err = to err := and see what happens, think why does it happen.
The Sneaky :=
The := operator is very helpful. It declares a variable and assigns a value to it in a single line. However it can lead to some interesting bugs as well. Can you spot the bug in the following code:
func saveJSON(fileName string, val interface{}) error {
file, err := os.Create(fileName)
if err != nil {
return err
}
defer file.Close()
if err := json.NewEncoder(file).Encode(val); err != nil {
log.Printf("error: can't encode - %s", err)
}
return err
}
func main() {
if err := saveJSON("i.json", 1i); err != nil {
log.Printf("main: error - %s", err)
}
}
Running the code will show the log from the saveJSON function (there are no complex numbers in JSON), but not the log from main. The reason is that we use := when calling json.NewEncoder(file).Encode(val) which creates a new err that’s local to the if statement. Remember that every time you write {, you start a new scope.
The solution - Use : instead of :=.
Conclusion
I hope that from now on, whenever you’ll see a variable in your Go code, you will look which scope this variable come from. You should also try to decide if it’s the right scope to use this variable.