Add support for tailstrict

[stephenamar-db]

With this PR, I'm implementing tailstrict in sjsonnet.
I'm following the guidance from google/jsonnet#343 (comment):

Quote:

1. If you call a function with tailstrict annotation on the apply AST, e.g. foo(42) tailstrict then the evaluation of the function body happens in "tail strict mode". The annotated AST need not actually be a tail call. Also, the arguments of this function are forced.
2. When another function call (can be a completely different function to the one that was originally called) is made when we're evaluating in "tail strict mode" and it is a tail call, then the current stack frame is re-used for the next call, rather than being pushed on top of.
3. Note that in order to preserve the tail strict mode into the new function, the new call AST has to be tailstrict as well.

Resolves #189.

[JoshRosen]

What is the thread-safety model here?

By having this be a mutable property of the evaluator itself I think we are foregoing safe multi-threaded usage of Evaluator.

A few lines up, I see that we have a non-thread-safe mutable HashMap for cachedImports, so it's possible that this class is de-facto pre-existingly not threadsafe.

Does this match your understanding? And is our current usage in line with this?

I raise this because I've been considering implementing certain performance optimizations which strongly rely on single-threaded evaluation (such as same-thread recycling of certain temporary structures used in string materialization) and it would be convenient to be able to make strong single-threadedness assumptions about evaluator.

[JoshRosen]

I think we might need handling for tailstrict here too? Despite the fact that a zero-argument function cannot itself be tail recursive, don't we still need to propagate the isInTailstrictMode flag for use in child evaluations?

[JoshRosen]

I'm curious about the interaction between tail strict mode, built-in function application, and potential lazy evaluation in built-ins.

For example, it is possible to have a tailstrict user-defined function which invokes a built-in function which returns lazy values which in turn reference user defined functions that could potentially be affected by tail strict mode? i.e. do we need "memory" of whether tail strict mode was in effect at the time of creation of a lazy value?

[JoshRosen]

I'm a little confused about the semantics of having an implicit parameter which has a default value. Where do we override this? I'm wondering if it might be clearer to have it be in its own parameter group if we don't expect to pass it implicitly?

[JoshRosen]

Courtesy of ChatGPT, an example demonstrating that implicits in the caller scope take precedence over the default parameter value:

object DefaultVsImplicitExample extends App {

  // A method with an implicit parameter that has a default value
  def greet(implicit name: String = "DefaultName"): Unit =
    println(s"Hello, $name!")

  // No implicit in scope, so it uses the default value
  greet // prints: "Hello, DefaultName!"

  // Introduce an implicit value in this scope
  {
    implicit val localName: String = "Alice"
    // Now greet() finds the implicit "Alice" instead of using the default
    greet // prints: "Hello, Alice!"
  }

  // Outside that block, the implicit is no longer visible, so greet() falls back to the default
  greet // prints: "Hello, DefaultName!"
}

---

Do we ever actually use the default value of vs here? i.e. do we ever mutate defSiteValScope?

I tried dropping the defaults and then ran into compilation problems in built-in functions, e.g. in uniqArr where we have calls like

val o1Key = keyFFunc.apply1(v, pos.noOffset)(ev)

which won't apply without a vs. But if this is indeed tail rec mode, don't we want to be using the caller's scope?

I find this pretty confusing overall and I'd like to get a better handle on how this works, esp. to make sure that we're not somehow having unexpected interactions between tail recursive functions and built in functions.