Code base overview

.
├── cmd
│   └── arrai    - cli
├── docs         - reference docs
├── engine       - database execution engine
├── examples     - what it says
│   ├── ...
├── rel          - relational type system
├── syntax       - language parser and compiler
├── tools
│   └── module   - module system
└── translate    - external format translators

In greater depth:

engine

Implements a trivial in-memory database system comprising:

• A single arr.ai value that evolves via transform expressions passed in
• Allows subscribers to observe expressions against that value

This feature was the original motivation for arr.ai, but it is not relevant to our current efforts to drive arr.ai towards being a general purpose data transformation language.

rel

• value.go is the core of the rel package. It defines the following interfaces:

  • Value is implemented by all arr.ai values.
  • Expr is implemented by all arr.ai expression objects. Values are also Exprs.
  • Tuple is implemented by all objects that represent arr.ai tuples.
    • Attr is a struct representing the name/value pairs of a tuple.
  • Set is implemented by all objects that represent arr.ai sets.

• value_*.go implement arr.ai value objects.

  An important principle is that there are only three types of values in arr.ai, numbers, tuples and sets. The variety of Go types that implement Value are simply implementation details. Originally the intent was ensure, e.g., that a GenericSet holding {|@,@char| (0,97), (1,98), (2,99)} and a String holding "abc" would compare equal, since they are logically the same value. More recently, the focus has shifted to detecting and automatically canonicalising such situations, such that {|@,@char| (0,97), (1,98), (2,99)} in arr.ai is implemented as a String in Go. This already goes as far as detecting when a transform yields a string-shaped result:

  $ arrai e '[97, 98, 99] => (:.@, @char: .@item)'
  abc
  $ arrai e '([97, 98, 99] => (:.@, @char: .@item)) = "abc"'
  {()}  # = true
  

  Work remains to catch all edge cases.

• expr_*.go implement arr.ai expression objects.

syntax

• parse.go implements the arr.ai language parser. parse_*.go factor out the more complex elements.
• compile.go transforms ASTs produced by parse.go into rel.Expr trees ready to evaluate. compile_*.go factor out the more complex elements.

module

module.go defines the Module interface which is supposed to import modules locally or from the internet. gomodule.go is an implementation of Module which leverages Go Modules strategy.

translate

Contains helper functions to interpret external encodings such as YAML as arr.ai values.