Right after releasing Tokay v0.6 as another milestone heavily focusing on internal features, development continues towards v0.7 as a huge milestone with the goals of implementing generic parselets and iterators, which where already made available with Tokay v0.6.1.

All three goals will significantly change Tokay's syntax in some ways, and establish one of the language's most powerful core features.

We're hoping to release this big version during summer 2023.

Generic parselets (generics) will become a fundamental part of Tokay. Generics where already planned from the beginning of the Tokay programming language, but they required some reworks on the compiler and syntax first, which was already laid with Tokay v0.6.1 and Tokay v0.6.2 released in April.

Generics allow to specify constants to a parselet to define specific parsing behaviors once, and use these behavors in combination for specialized tokens, parselets or even entire grammars. The generics are evalualted duing compile-time and by usage.

Here's a very small example which already works with the latest commit of pull request #105. It effectively implements two parselets, one parsing "axa", the other parsing "bxb". The parselets are separately instanciated, but effectively defned only in one place.

# Generic parselet T, serving as a template.
T: @<P> {
    P 'x' P  print($0)
}

# The final parselets are instanciated by its usage.
Parse_axa: T<'a'>
Parse_bxb: T<'b'>

Parse_axa Parse_axa Parse_bxb  # parses "axaaxabxb"

Generic parselets can also make use of defaults for every constant, and allow for named arguments, equally to function calls at runtime.