# Values

Generally, everything in Tokay is some kind of value or part of a value. The term "value" refers to both simple atomic values like booleans, numbers, strings but also objects which are partly mutable, recursive or callable.

## Atomics

Atomic values stand on their own and are generally not mutable in the sense of an object. These values are the following:

``````void           # values to representing just nothing
null           # values representing a defined "set to null"
true false     # boolean values
``````

Using the builtin function `bool(v)`, a boolean value can be constructed from any other value `v`, by testing for truth.

## Numbers

Tokay supports `int` and `float` as numeric types.

``````42 -23         # signed 64-bit integer number object
3.1415 -1.337  # signed 64-bit float number object
``````

For numbers, the following methods can be used:

• `int(v)` contructs an int value from any other value `v`
• `float(v)` contructs a float value from any other value `v`
• `float.ceil()` - returns the next integer ceiling of a float
• `float.fract()` - returns only the fractional part of a float
• `float.trunc()` - truncates the fractional part off a float

As of Tokay v0.5, the internal implementation of numbers is unstable, and will be clarified in a future version. See #52 for details.

## Strings

A string ("str") is a unicode-character sequence of arbitrary length.

``````s = "Tokay 🦎"
s = s + " is cool"
s += "!"
``````

Str objects can be concatenated by the `+`- and `+=`-operators, multiplied by the `*`- and `*=`-operators and provide the following methods:

• `str(v)` - constructs a string object from any other value `v`
• `str.byteslen()` - return total length of bytes used by the string
• `str.endswith(s)` - check if string ends with postfix `s`
• `str.join(l)` - create a string delimited by str from a list `l`
• `str.len()` - return number of characters in the string
• `str.lower()` - turns any upper-case characters of a string into lower-case order
• `str.replace(from, to="", n=void)` - replace string `from` by `to` for at least `n`-times
• `str.startswith(s)` - check if string begins with prefix `s`
• `str.substr(start=0, length=void)` - returns a substring from `start` of `length` or to the end
• `str.upper()` - turns any lower-case characters of a string into lower-case order

## Lists

A list is a sequence of arbitrary values in a row. Therefore, a list can also contain further lists, or other complex objects. A list is also mutable, which means items can be extended or removed during runtime.

``````# list of values
(42, true, "yes")
l = (42 true "yes")
l.len()  # 3
``````

Lists can be concatenated by the `+`- and `+=`-operators, and provide the following methods:

• `list(*args)` - constructs a new list from all arguments provided
• `list.len()` - returns number of items in the list
• `list.push(item, index=void)` - either appends an `item` to the list or inserts it at position `index`
• `list.pop(index=void)` - either pops the last item off the list or removes and returns item at position `index`

## Dicts

Dictionaries ("dicts") are hash tables or maps with key-value-pairs, where a value is referenced by using the key as its storage location.

``````# dictionary (dict), a map of key-value-pairs
(i => 42, b => true, status => "success")
d = (i => 42 b => true status => "success")
d.update((x => 23.5))
``````

Currently, as of Tokay v0.5 and below, Dicts in Tokay only support for strings as keys. This behavior is currently discussed and may change in a future version! This is also the reason for a lack of method implementations for dicts currently. See #50 for details.

In Tokay v0.5, dicts provide the following methods:

• `dict()` - creates a new, empty dict
• `dict.len()` - returns number of items in the dict
• `dict.update(other)` - merges two dicts

## Tokens

Tokens are callables consuming input from the stream. They are object values as well. They always return a value parsed from the input stream in case the token matches. Otherwise, tokens usually reject the current block branch or parselet, to try other alternatives.

``````'touch'    # silently touch a string in the input (low severity)
''match''  # verbosely match a string from the input (high severity)
[A-Z0-9]+  # matching a sequence of multiple valid characters
Int        # built-in token for parsing and returning Integer values
Word(3)    # built-in token Word, matching at least words of length 3
``````

In terms of a compiler engineer, tokens in Tokay are the terminal symbols of a context-free grammar.

## Functions and parselets

Functions are sub-programs for a specific task or routine which can be used for multiple tasks. A function can accept arguments with default values.

``````# function that doubles its value
f : @x { x * 2 }
f(9)  # 18

# anonymous function example
@x { x * 3 }(5)  # 15, returned by anonymous function that is called in-place
``````

Parselets are more-specific functions consuming input and used for parsing. They are conceptionally the same, but also they are very distinguishable in their usage.

``````# parselet that parses simple assignments to variables
Assign : @{
variable => Ident _ '=' _ value => Number
}

# called on a given input `n = 42`...
Assign
# ... returns dict `(variable => "n", value => 42)`
``````

In terms of a compiler engineer, parsing functions (in Tokay called "parselets") are considered as non-terminal symbols of a context-free grammar.