# The `*.wit` format This is intended to document the `*.wit` format as it exists today. The goal is to provide an overview to understand what features `wit` files give you and how they're structured. This isn't intended to be a formal grammar, although it's expected that one day we'll have a formal grammar for `*.wit` files. If you're curious to give things a spin try out the [online demo](https://bytecodealliance.github.io/wit-bindgen/) of `wit-bindgen` where you can input `*.wit` on the left and see output of generated bindings for languages on the right. If you're looking to start you can try out the "markdown" output mode which generates documentation for the input document on the left. ## Lexical structure The `wit` format is a curly-braced-based format where whitespace is optional (but recommended). It is intended to be easily human readable and supports features like comments, multi-line comments, and custom identifiers. A `wit` document is parsed as a unicode string, and when stored in a file is expected to be encoded as UTF-8. Additionally, wit files must not contain any bidirectional override scalar values, control codes other than newline, carriage return, and horizontal tab, or codepoints that Unicode officially deprecates or strongly discourages. The current structure of tokens are: ```wit token ::= whitespace | comment | operator | keyword | identifier ``` Whitespace and comments are ignored when parsing structures defined elsewhere here. ### Whitespace A `whitespace` token in `*.wit` is a space, a newline, a carriage return, or a tab character: ```wit whitespace ::= ' ' | '\n' | '\r' | '\t' ``` ### Comments A `comment` token in `*.wit` is either a line comment preceded with `//` which ends at the next newline (`\n`) character or it's a block comment which starts with `/*` and ends with `*/`. Note that block comments are allowed to be nested and their delimiters must be balanced ```wit comment ::= '//' character-that-isnt-a-newline* | '/*' any-unicode-character* '*/' ``` There is a special type of comment called `documentation comment`. A `doc-comment` is either a line comment preceded with `///` whichends at the next newline (`\n`) character or it's a block comment which starts with `/**` and ends with `*/`. Note that block comments are allowed to be nested and their delimiters must be balanced ```wit doc-comment ::= '///' character-that-isnt-a-newline* | '/**' any-unicode-character* '*/' ``` ### Operators There are some common operators in the lexical structure of `wit` used for various constructs. Note that delimiters such as `{` and `(` must all be balanced. ```wit operator ::= '=' | ',' | ':' | ';' | '(' | ')' | '{' | '}' | '<' | '>' | '*' | '->' ``` ### Keywords Certain identifiers are reserved for use in `wit` documents and cannot be used bare as an identifier. These are used to help parse the format, and the list of keywords is still in flux at this time but the current set is: ```wit keyword ::= 'use' | 'type' | 'resource' | 'func' | 'u8' | 'u16' | 'u32' | 'u64' | 's8' | 's16' | 's32' | 's64' | 'float32' | 'float64' | 'char' | 'handle' | 'record' | 'enum' | 'flags' | 'variant' | 'union' | 'bool' | 'string' | 'option' | 'list' | 'expected' | 'unit' | 'as' | 'from' | 'static' | 'interface' | 'tuple' | 'async' | 'future' | 'stream' ``` ## Top-level items A `wit` document is a sequence of items specified at the top level. These items come one after another and it's recommended to separate them with newlines for readability but this isn't required. ## Item: `use` A `use` statement enables importing type or resource definitions from other wit documents. The structure of a use statement is: ```wit use * from other-file use { a, list, of, names } from another-file use { name as other-name } from yet-another-file ``` Specifically the structure of this is: ```wit use-item ::= 'use' use-names 'from' id use-names ::= '*' | '{' use-names-list '}' use-names-list ::= use-names-item | use-names-item ',' use-names-list? use-names-item ::= id | id 'as' id ``` Note: Here `use-names-list?` means at least one `use-name-list` term. ## Items: type There are a number of methods of defining types in a `wit` document, and all of the types that can be defined in `wit` are intended to map directly to types in the [interface types specification](https://github.com/WebAssembly/interface-types). ### Item: `type` (alias) A `type` statement declares a new named type in the `wit` document. This name can be later referred to when defining items using this type. This construct is similar to a type alias in other languages ```wit type my-awesome-u32 = u32 type my-complicated-tuple = tuple ``` Specifically the structure of this is: ```wit type-item ::= 'type' id '=' ty ``` ### Item: `record` (bag of named fields) A `record` statement declares a new named structure with named fields. Records are similar to a `struct` in many languages. Instances of a `record` always have their fields defined. ```wit record pair { x: u32, y: u32, } record person { name: string, age: u32, has-lego-action-figure: bool, } ``` Specifically the structure of this is: ```wit record-item ::= 'record' id '{' record-fields '}' record-fields ::= record-field | record-field ',' record-fields? record-field ::= id ':' ty ``` ### Item: `flags` (bag-of-bools) A `flags` statement defines a new `record`-like structure where all the fields are booleans. The `flags` type is distinct from `record` in that it typically is represented as a bit flags representation in the canonical ABI. For the purposes of type-checking, however, it's simply syntactic sugar for a record-of-booleans. ```wit flags properties { lego, marvel-superhero, supervillan, } // type-wise equivalent to: // // record properties { // lego: bool, // marvel-superhero: bool, // supervillan: bool, // } ``` Specifically the structure of this is: ```wit flags-items ::= 'flags' id '{' flags-fields '}' flags-fields ::= id, | id ',' flags-fields? ``` ### Item: `variant` (one of a set of types) A `variant` statement defines a new type where instances of the type match exactly one of the variants listed for the type. This is similar to a "sum" type in algebraic datatypes (or an `enum` in Rust if you're familiar with it). Variants can be thought of as tagged unions as well. Each case of a variant can have an optional type associated with it which is present when values have that particular case's tag. All `variant` type must have at least one case specified. ```wit variant filter { all, none, some(list), } ``` Specifically the structure of this is: ```wit variant-items ::= 'variant' id '{' variant-cases '}' variant-cases ::= variant-case, | variant-case ',' variant-cases? variant-case ::= id | id '(' ty ')' ``` ### Item: `enum` (variant but with no payload) An `enum` statement defines a new type which is semantically equivalent to a `variant` where none of the cases have a payload type. This is special-cased, however, to possibly have a different representation in the language ABIs or have different bindings generated in for languages. ```wit enum color { red, green, blue, yellow, other, } // type-wise equivalent to: // // variant color { // red, // green, // blue, // yellow, // other, // } ``` Specifically the structure of this is: ```wit enum-items ::= 'enum' id '{' enum-cases '}' enum-cases ::= id, | id ',' enum-cases? ``` ### Item: `union` (variant but with no case names) A `union` statement defines a new type which is semantically equivalent to a `variant` where all of the cases have a payload type and the case names are numerical. This is special-cased, however, to possibly have a different representation in the language ABIs or have different bindings generated in for languages. ```wit union configuration { string, list, } // type-wise equivalent to: // // variant configuration { // 0(string), // 1(list), // } ``` Specifically the structure of this is: ```wit union-items ::= 'union' id '{' union-cases '}' union-cases ::= ty, | ty ',' union-cases? ``` ## Item: `func` Functions can also be defined in a `*.wit` document. Functions have a name, parameters, and results. Functions can optionally also be declared as `async` functions. ```wit thunk: func() fibonacci: func(n: u32) -> u32 sleep: async func(ms: u64) ``` Specifically functions have the structure: ```wit func-item ::= id ':' 'async'? 'func' '(' func-args ')' func-ret func-args ::= func-arg | func-arg ',' func-args? func-arg ::= id ':' ty func-ret ::= nil | '->' ty ``` ## Item: `resource` Resources represent a value that has a hidden representation not known to the outside world. This means that the resource is operated on through a "handle" (a pointer of sorts). Resources also have ownership associated with them and languages will have to manage the lifetime of resources manually (they're similar to file descriptors). Resources can also optionally have functions defined within them which adds an implicit "self" argument as the first argument to each function of the same type of the including resource, unless the function is flagged as `static`. ```wit resource file-descriptor resource request { static new: func() -> request body: async func() -> list headers: func() -> list } ``` Specifically resources have the structure: ```wit resource-item ::= 'resource' id resource-contents resource-contents ::= nil | '{' resource-defs '}' resource-defs ::= resource-def resource-defs? resource-def ::= 'static'? func-item ``` ## Types As mentioned previously the intention of `wit` is to allow defining types corresponding to the interface types specification. Many of the top-level items above are introducing new named types but "anonymous" types are also supported, such as built-ins. For example: ```wit type number = u32 type fallible-function-result = expected type headers = list ``` Specifically the following types are available: ```wit ty ::= 'u8' | 'u16' | 'u32' | 'u64' | 's8' | 's16' | 's32' | 's64' | 'float32' | 'float64' | 'char' | 'bool' | 'string' | 'unit' | tuple | list | option | expected | future | stream | id tuple ::= 'tuple' '<' tuple-list '>' tuple-list ::= ty | ty ',' tuple-list? list ::= 'list' '<' ty '>' option ::= 'option' '<' ty '>' expected ::= 'expected' '<' ty ',' ty '>' future ::= 'future' '<' ty '>' stream ::= 'stream' '<' ty ',' ty '>' ``` The `tuple` type is semantically equivalent to a `record` with numerical fields, but it frequently can have language-specific meaning so it's provided as a first-class type. Similarly the `option` and `expected` types are semantically equivalent to the variants: ```wit variant option { none, some(ty), } variant expected { ok(ok-ty) err(err-ty), } ``` These types are so frequently used and frequently have language-specific meanings though so they're also provided as first-class types. Finally the last case of a `ty` is simply an `id` which is intended to refer to another type or resource defined in the document. Note that definitions can come through a `use` statement or they can be defined locally. ## Identifiers Identifiers in `wit` can be defined with two different forms. The first is a lower-case [stream-safe] [NFC] [kebab-case] identifier where each part delimited by '-'s starts with a `XID_Start` scalar value with a zero Canonical Combining Class: ```wit foo: func(bar: u32) red-green-blue: func(r: u32, g: u32, b: u32) ``` This form can't name identifiers which have the same name as wit keywords, so the second form is the same syntax with the same restrictions as the first, but prefixed with '%': ```wit %foo: func(%bar: u32) %red-green-blue: func(%r: u32, %g: u32, %b: u32) // This form also supports identifiers that would otherwise be keywords. %variant: func(%enum: s32) ``` [kebab-case]: https://en.wikipedia.org/wiki/Letter_case#Kebab_case [Unicode identifier]: http://www.unicode.org/reports/tr31/ [stream-safe]: https://unicode.org/reports/tr15/#Stream_Safe_Text_Format [NFC]: https://unicode.org/reports/tr15/#Norm_Forms ## Name resolution A `wit` document is resolved after parsing to ensure that all names resolve correctly. For example this is not a valid `wit` document: ```wit type foo = bar // ERROR: name `bar` not defined ``` Type references primarily happen through the `id` production of `ty`. Additionally names in a `wit` document can only be defined once: ```wit type foo = u32 type foo = u64 // ERROR: name `foo` already defined ``` Names do not need to be defined before they're used (unlike in C or C++), it's ok to define a type after it's used: ```wit type foo = bar record bar { age: u32, } ``` Types, however, cannot be recursive: ```wit type foo = foo // ERROR: cannot refer to itself record bar1 { a: bar2, } record bar2 { a: bar1, // ERROR: record cannot refer to itself } ``` The intention of `wit` is that it maps down to interface types, so the goal of name resolution is to effectively create the type section of a wasm module using interface types. The restrictions about self-referential types and such come from how types can be defined in the interface types section. Additionally definitions of named types such as `record foo { ... }` are intended to map roughly to declarations in the type section of new types.