Match behaviour of "index" with "." if key does not exist

[robinchrist]

While accessing values via . throws an error if the requested key does not exist, index does not and instead produces <no value> in the output - this is somewhat surprising and not really consistent.

I believe it would be a good thing to make both operators, . and index behave the same in this case, i.e. make index throw an error if the requested key does not exist

This could potentially be hidden behind a flag to make sure no legacy code breaks (as I can imagine that some people may rely on that behaviour)

Related issue: #1362

[hairyhenderson]

Hi @robinchrist, thanks for posting this.

Indeed, this is the behaviour:

$ gomplate -i '{{ .missing }}'
10:33:54 ERR  error="failed to render template <arg>: template: <arg>:1:3: executing \"<arg>\" at <.missing>: map has no entry for key \"missing\""
$ gomplate -i '{{ index . "missing" }}'
<no value>

I'm interested in a bit more clarity on this statement:

[...] this is somewhat surprising and not really consistent.

Why do you think . and index should behave the same way? Is there some documentation that implies the two should behave similarly?

To expand a bit on the background provided in #1362, this Go code can be considered equivalent to the two approaches:

// ".foo" in a template
fmt.Printf("%v", myContext.foo)

// "index . `foo`" in a template
fmt.Printf("%v", myMap["foo"])

Assuming myContext is a data structure in the first example, the code won't even compile unless foo exists. The current error behaviour in gomplate is the closest analogue we have to this (which is why I chose to use that behaviour for the missingkey option).

However, if myMap is a map without a "foo" entry in the second line, the code will compile and run successfully, returning the map's zero value type (i.e. "" if a string, 0 if an integer, etc...).

All this to say - . and index are consistent with the behaviour they're intended to model, even though they're not consistent with each other (which was never the intention).

It's also worth mentioning that index can also be used on arrays or slices, and again, the behaviour matches what happens in the equivalent Go code. If I try to access an index that's out of range, Go will panic:

s := []int{1, 2, 3}

// equivalent to "index $s 4" in a template
fmt.Printf("%v", s[4])

This will panic with index out of range [4] with length 3 (here's a runnable example).

The equivalent gomplate template behaves the same:

$ gomplate -i '{{ $a := slice 0 1 2 }}{{ index $a 4 }}'
10:48:13 ERR  error="failed to render template <arg>: template: <arg>:1:26: executing \"<arg>\" at <index $a 5>: error calling index: index out of range: 4"

As workarounds, as suggested in #1362, using coll.Has is an option, or you could use test.Required instead if you want gomplate to error out.

[robinchrist]

Why do you think . and index should behave the same way? Is there some documentation that implies the two should behave similarly?

Well, they are introduced together in the documentation

For maps, accessing values can be done with the . operator. Given a map $map with a key foo, you could access it like:
...
However, this kind of access is limited to keys which are strings and contain only characters in the set (a-z,A-Z,_,1-9), and which do not begin with a number. If the key doesn’t conform to these rules, you can use the index function (like how arrays are accessed):

implying that they serve the same purpose (which imho they do!) and are interchangeable if the key only has characters in the set a-z,A-Z,_,1-9 - i.e. . just being syntactic sugar

Assuming myContext is a data structure in the first example, the code won't even compile unless foo exists. The current error behaviour in gomplate is the closest analogue we have to this (which is why I chose to use that behaviour for the missingkey option).

However, if myMap is a map without a "foo" entry in the second line, the code will compile and run successfully, returning the map's zero value type (i.e. "" if a string, 0 if an integer, etc...).

All this to say - . and index are consistent with the behaviour they're intended to model, even though they're not consistent with each other (which was never the intention).

It's also worth mentioning that index can also be used on arrays or slices, and again, the behaviour matches what happens in the equivalent Go code. If I try to access an index that's out of range, Go will panic:

From a Go programmers point of view, this is very true. But we need to ask the question, whether this is also true from the point of view of Gomplate users.

Is Gomplate a tool for Go programmers and should thus match the behaviour of Golang? Or is Gomplate more generally a template engine that just happens to be written in Go?

As workarounds, as suggested in #1362, using coll.Has is an option, or you could use test.Required instead if you want gomplate to error out.

Well Yes, but actually no.
While you could do this, it's not really a solution. It results in a massive amount of boilerplate code and makes the templates unnecessarily complex.

My intention behind this issue and discussion is to make it easier to detect typos. I've recently ported a set of configuration files for mechanical stuff to Gomplate, as a lot of data is repeated very often, but sometimes need to be changed if someone spots a mistake and you want to be sure to catch all occasions and not correct anything that happens to be the same value but is not related - Perfect use for a template engine.

However, I've had a couple of typos here and there just because of the sheer amount of data. The keys are deliberately long, expressive and human readable text, they often contain special characters for special units.
The . operator allows me to catch typos of keys very quickly, while the index character does not.

Right now my workaround is to wrap the gomplate call in a script that is followed by a grep through all files

gomplate <snip>

grep --color=always -rni "<no value>" ./out-text/ || GREP_EXIT_CODE=$?

if [ $GREP_EXIT_CODE -eq 0 ]; then
    echo "ERROR"
    exit 1
fi

[hairyhenderson]

Hi @robinchrist, thanks for your thoughtful response.

Well, they are introduced together in the documentation
[...]
implying that they serve the same purpose (which imho they do!) and are interchangeable if the key only has characters in the set a-z,A-Z,_,1-9 - i.e. . just being syntactic sugar

Ah, this is a great point, and I can definitely see how that's implied. I should probably rewrite it to reflect the differences - index for example works with arrays, slices, strings and maps. The . operator however works with maps and data structures (like for example when dealing with time you can call methods on to the time.Time value with . - like (time.Now).Round, etc...).

Still, I understand your perspective and agree that there is indeed a lot of overlap.

Is Gomplate a tool for Go programmers and should thus match the behaviour of Golang? Or is Gomplate more generally a template engine that just happens to be written in Go?

Well put. When I first wrote gomplate it was a tool aimed at people already familiar with the general syntax of Go's text/template language, but it's obviously grown and become popular with lots of different people with lots of different backgrounds.

My intention behind this issue and discussion is to make it easier to detect typos.

I can absolutely empathize with this. You've convinced me that we do need to figure out a good approach for this.

I think the first step would be to add a new function with the stricter behaviour (maybe coll.Index, since it operates on collections?), and then we can explore optionally aliasing index to coll.Index later.

[robinchrist]

I can absolutely empathize with this. You've convinced me that we do need to figure out a good approach for this.

That's great to hear! 🙂

I think the first step would be to add a new function with the stricter behaviour (maybe coll.Index, since it operates on collections?), and then we can explore optionally aliasing index to coll.Index later.

I would simply suggest to call it safeIndex, as it makes the (current) intention a bit clearer

[hairyhenderson]

Almost all gomplate functions are namespaced intentionally, so I think coll.Index makes the most sense. I could alias it to safeIndex too, but given that it probably makes sense for index to be an eventual alias (optionally, or by default in future), I'm not too keen on rushing into an alias.

[hairyhenderson]

FYI I've issued #1588 to clarify the docs - @robinchrist if you're able would you mind having a look and letting me know if it's clear?

[hairyhenderson]

Also see #1589