4. Grammar#
This page defines how the parser puts the words, string literals and other pieces of text of Lexical syntax together into forms, the expressions that the parser produces, the template that every form follows, and the kinds of form that may be written in each body. The signatures of the forms, which list the arguments of each head, are not part of this page.
This page specifies the language.
4.1. The grammar of the parser#
The rules below and the definitions of Lexical syntax are one grammar, mtp.lark. An
alternative written a -> free puts its result in the tree under the name free, and ! before
the name of a rule keeps the quoted text of that rule in the tree.
4.1.1. Forms and sources#
form ::= "(" _whitespace_or_comment? (items | head_items)? RPAR
list_items ::= _whitespace_or_comment? items?
items ::= touching (_whitespace_or_comment items?)?
head_items ::= head_touching (_whitespace_or_comment items?)?
touching ::= plain (form touching?)?
| paren touching?
head_touching ::= head (form touching?)?
expression ::= paren | plain
paren ::= form
| PREFIX form -> prefixed
plain ::= plain_item
| PREFIX plain_item -> prefixed
plain_item ::= list | word | string | interpolated
Rule 4.1
A form is a (, then any number of expressions, its items, then a ). An
expression is a form, a list literal, a word, a broadened type, a
string literal or an interpolated string. The first item of a form is its head, and it may be a head
that ends with _ (Rule 4.9). The form () has no items and no head.
So (circle c (r 1)) is a form with three items, whose head is circle and whose third item is
the form (r 1), and (motif m () …) holds the form (), an empty list of parameters.
start ::=_BOM?_whitespace_or_comment? (form_whitespace_or_comment?)*
Rule 4.2
A source is any number of forms, with or without whitespace and comments between them. Anything else at the top level of a source is an error. A source may hold no form at all.
Rule 4.3
Among the items of a form, a list literal or brackets, whitespace or a comment separates every two
expressions, except where one of the two is a form. More exactly, an expression that ends with the
) of a form, which is a form or a prefixed form such as free@(g), may be followed
directly by any expression that does not start with . or [ (Rule 3.6),
and every other expression may be followed directly only by a form. So (f a(b)c) holds three
expressions after f, (f (a)b) and (f 'x'(b)) hold two, and (f 'a''b') and (f [1]x) are
errors.
4.1.2. Lists and brackets#
list ::= "["list_items"]"list_brackets* list_brackets ::= "["_whitespace_or_comment?expression_whitespace_or_comment? "]" brackets ::= "["_whitespace_or_comment?items"]"
Rule 4.4
A list literal is written in square brackets, with its items separated as in
Rule 4.3: [], [0 1], [(f x) 'a']. A [ right after the ] of a list literal
starts brackets after the list literal, which hold exactly one expression: [a b c][h].
Rule 4.5
A [ right after a name, with no whitespace between them, starts brackets after the name,
which hold one expression or more: xs[0], xs[(+ k 1)], g['m1'], func[real real]. Several
pairs of brackets may follow each other, and after brackets, . and more
name parts may follow: xs[0].pos.x, m[0][1], a[0].b[1]. A [ after
whitespace starts a list literal: xs [0] is the name xs and the list literal [0].
4.1.3. Words#
word ::=one_word("|"one_word)* |name_stringone_word ::=NAMEafter_brackets? |number|bool|NONE|operatorafter_brackets ::= (brackets("."PARTS)?)+ name_string ::=name_string_text("."PARTS|list_brackets)*
number ::=INT|RATIO|REAL|IMAG|COMPLEXbool ::=TRUE|FALSE!operator ::= "+" | "-" | "*" | "/" | "^" | "<" | "<=" | ">" | ">=" | "=" | "!=" | "."
4.1.4. Heads#
head ::=HEAD|one_word("|"one_word)* "|"HEAD
A mtp:head is the head of a form that has no name: a name that ends with _, or a
broadened type whose last word is such a name. Rule 4.9 gives its meaning.
4.1.5. Strings#
string ::=STRING|TEXinterpolated ::=DOLLAR_QUOTE(TEXT|braces)*QUOTEname_string_text ::=HASH_QUOTETEXT?braces(TEXT|braces)*QUOTEbraces ::= "{"_WHITESPACE?expression_WHITESPACE?FORMAT? "}"
4.2. The expressions that the parser produces#
Rule 4.7
The parser produces the forms of a source as expressions, one for each form, in the order of the source. Each expression is one of these:
Expression |
Written |
Holds |
|---|---|---|
form |
|
its items, in order |
list literal |
|
its items, in order |
name |
|
its name parts, in order. Each name part is the text of a part between dots, or what a pair of brackets holds: its one expression, or, when it holds more than one, the brackets |
brackets |
|
its expressions, in order |
broadened type |
|
its words, as expressions, in order |
string literal |
|
its text, and whether it is a TeX string |
interpolated string |
|
its pieces: text, the expression of each |
|
|
the expression and the format spec |
|
the value (Rule 3.12) |
|
bool literal |
|
the value |
none |
|
nothing |
In an interpolated string, each piece of text is as long as possible, so two pieces of text are never next to each other.
An operator is a name with one name part (Rule 3.9).
Rule 4.8
The parser produces each spelling on the left as the form on the right:
Written |
Is the form |
|---|---|
|
|
|
|
|
|
|
|
|
|
The form has the position of the first character of its spelling. A (. X Y) that follows a name
string has the position of its . or [.
Rule 4.9
When the head of a form is a mtp:head, the parser removes the _ and puts none in as
the second item: (circle_ (r 1)) is (circle none (r 1)), and (marks.grid_ …) is
(marks.grid none …). When the head is a broadened type, this rule applies to its last word, so
(line|none_ …) is (line|none none …). The head none_ becomes the literal none:
(none_ a) is (none none a).
The parser produces a name with several name parts, such as p.r or xs[0].y, as one name
(Rule 4.7). The rewrite step rewrites it into (. X Y) forms: p.r is
(. p r), and xs[0].y is (. (. xs 0) y) (Rule 5.4). The meaning of (. X Y)
is given in Rule 5.8.
4.3. The form template#
Rule 4.10
Every form is written (head name? positional… option… body…):
The head is the first item. In a declaration, it names the kind of thing declared. It is a name (
circle,real,motif,dial), a name followed by brackets (list[color],func[real]), a broadened type (real|none), or a form that gives a type (((real (>= 0)) r 1)).The name, for a form that has one, is the second item (Rule 4.11).
The positional arguments come next. Each kind of form has a fixed number of them, and some of them may be left out.
The optional arguments come next (Rule 4.13).
The docstring, for a form that has one, comes next (Rule 4.15).
The body comes last: any number of items, or none. The kinds of item in a body depend on the form (Rule 4.16).
So a call (+ a b c), a special form (if c a b) and an import (import math dot) follow the
template, with no name.
Rule 4.11
The name of a declaration is a bare name with one name part, none, a name string such as
#'dot-{k}', or (name s). The string of a name string and of (name s) is known when compiling.
none as the name means that the declaration has no name, as the head head_ does
(Rule 4.9). A name in quotes is an error (“a name is written bare, not in quotes”),
and so is a name that Rule 6.7 does not allow.
Rule 4.12
These forms have no name, so their second item is their first positional argument or their first
optional argument: a listener, (move …), an (animation …) inside a control or a
listener, (foreach …), (let …), (exists-when …), (import …), (term …), (like …), every
optional argument, every call and every special form. A view and a control always have a
name, because their elements on the page give it, as <mtp-canvas name="v">.
Rule 4.13
An optional argument is a form, after the name and the positional arguments, whose head names one
of the optional arguments that the form allows, such as (r 1) in (circle c (r 1)). In every
form, the optional arguments come before the body, in any order, except that a (like …) comes
before every other optional argument. An optional argument after the first item of the body is an
error, such as “(x …) is an optional argument, so
it comes before the canvas’s body”.
Code differs
The compiler checks this rule in the bodies of the scene, of groups, rows and columns, of
animations, of at-once and in-order, of canvases, of controls and of listeners. It allows the
(defaults …) of a formula after its other items (compiler.py:4905, formula_decl), the
(font f) of a readout after its listeners, and the (kind k) of a display after its listeners.
Bead mtp-ydd4.7.11 makes these errors.
Rule 4.14
Each optional argument appears at most once in a form: a second one is an error, such as “‘fill’ is given twice”.
Since (pos P) is (x …) (y …) (Rule 5.4), a form that gives pos and also x
or y gives x or y twice, and is an error. The forms that allow a (like …) are given in
Rule 5.4.
Rule 4.15
Every declaration and every listener may have a docstring, doc@'text', and at most one. The
docstring comes after all the positional arguments and all the optional arguments, and before the
body. In a form that has no body, the docstring is the last item. Its string literal is plain: an
interpolated string is an error.
So a docstring comes after the name, the positional arguments, the parameters of a function or a
motif included, and the optional arguments. A variable in the body of a motif may have one, as any
other declaration. (real k 1 doc@'d' (>= 0)), (func f ((real x)) doc@'d' (* 2 x)) and
(circle c (r 1) doc@'d') are allowed. (real k 1 (>= 0) doc@'d'),
(func f ((real x)) (* 2 x) doc@'d'), (circle c doc@'d' (r 1)) and
(button b doc@'d' (animation a)) are errors.
Code differs
The compiler removes a docstring from any place after the name (compiler.py:7322, _take_doc),
so a docstring before an optional argument or after the body compiles. About 22 forms of the
examples have a docstring before an optional argument, such as a button, a play, an on-ready or
an on-click with a docstring before its (animation …), and three formulas with a docstring
before their (defaults …). Bead mtp-ydd4.7.34 makes these errors and moves those docstrings. A
variable in the body of a motif may not have a docstring: (motif m () (real k 1 doc@'d') …) is the error
“unknown constraint ‘doc’”, because compiler.py:4822 (body_variable) does not remove the
docstring. Bead mtp-qiro fixes this.
4.4. Bodies#
Rule 4.16
Each body holds only the kinds of item that Table 4.1 gives for it, in the numbers
that it gives, and the let
and foreach forms that Rule 4.17 allows. Any other item in a body is an error.
Body |
Optional arguments |
Other items |
let, foreach |
|---|---|---|---|
The top level of a source |
none |
|
yes |
The top level of a file to import |
none |
Only |
yes |
|
|
Shapes: every shape, |
yes |
|
the optional arguments of a group: |
Shapes. |
yes |
Any other shape, such as |
the optional arguments of the shape |
The string of a text, as a positional argument |
no body |
A motif instance, |
its parameters, and the optional arguments of an instance: |
None: “an instance of M takes no members” |
no body |
A variant instance |
the optional arguments of its base shape, and the parameters of the variant |
None |
no body |
|
none |
Shapes. Variables that are bound or known when compiling. |
yes |
|
none |
Exactly one shape, which is not a group |
no |
|
none |
Exactly one expression |
no |
|
none |
yes |
|
|
none |
Items |
yes |
|
none |
Variables, namespaces, palettes. Not: functions, types, motifs, variants, shapes, components, transitions, animations, imports, interfaces, colormaps, choreos |
yes |
|
none |
Items |
yes |
|
|
Exactly one string, plain or TeX, known when compiling. |
yes |
|
|
None |
no body |
A variable, |
none |
Constraints, such as |
yes |
|
|
None |
no body |
|
|
Steps: |
yes |
|
|
The steps of an animation. Not: |
yes |
|
|
Its content, at least one item: names of shapes or of the scene, list literals, and expressions that give a shape, |
yes |
|
|
The listeners |
yes |
|
|
Listeners on the terms of its formula ( |
yes |
A control, such as |
its optional arguments, among them |
Its positional argument: the move of a slider, the target of a toggle. Conditions, such as |
yes |
A listener, inside the body of a view |
its optional arguments, its animation among them. Not: |
Its positional arguments: the shape or term that it listens to, and for some listeners a target. Conditions |
yes |
|
none |
Items |
yes |
|
none |
|
yes |
|
none |
Items |
yes |
|
none |
Exactly one circle, rect, ellipse, triangle, polygon or ring |
no |
|
none |
The file, then the names to import, or no names to import the file whole |
no |
So a source need not hold a scene: (real a 1) (readout r $'{a}') is a world model with a
variable and a readout, and an empty file is a world model with nothing in it. A source with two
scenes is an error. (canvas v) is an error, because a canvas holds at least one item of content,
and (canvas v s) and (canvas v c) are allowed.
Code differs
The error for a second scene is “a file has exactly one scene” (compiler.py:4854), and
(canvas v) compiles, to a canvas that draws nothing. Bead mtp-ydd4.7.59.
Rule 4.17
(let ((name value) …) item…), (foreach i n item…), (foreach x list item…) and
(foreach ((i n) (x list) …) item…), written in a body, are replaced by their items, with their names bound: once
for let, and once for each value for foreach. A let and a foreach may be written in every
body that holds declarations or any number of items, the top level of a source and of a file to
import included, and nowhere else. Their items are of the kinds that the body around them holds.
The column “let, foreach” of Table 4.1 follows from this rule. So a let and a
foreach may never be written among the positional arguments or the optional arguments of a form,
in a form that has no body, or in a body that holds a fixed number of items, such as the body of a
function or a variant. (let ((k 2)) (real a k)) may be written at the top level, a let may be
written among the steps of an animation, and (circle c (let ((k 2)) (r k))) is an error, because
(r k) is an optional argument of the circle.
Code differs
compiler.py:947(body_let) makes aletat the top level an error, and aletinside aforeachat the top level too. For aforeachat the top level of a file to import, the error is “holds a variable that is not const” (compiler.py:7257,_not_importable), which names the wrong form (bead mtp-inhr). Bead mtp-ydd4.7.7 allows both at both top levels.A
letis an error in the body of an animation, a choreo, anat-onceand anin-order(compiler.py:3018,steps), though aletinside aforeachthere compiles.letandforeachare errors in a namespace and a formula, and the body of a canvas, a readout or a display allows them only around listeners. Bead mtp-ydd4.7.8 allows them in every body marked “yes” in Table 4.1.compiler.py:5158(shape_here) expands every item of a shape, so(circle c (let ((k 2)) (r k)))and(circle c (foreach i 1 (r 2)))compile, and give the circle a radius of 2. Bead mtp-ydd4.7.9 makes aletor aforeachamong the arguments of a form an error.
By Rule 6.3, an exists-when names a shape, a term or a member that is
declared before it, in every body.
Code differs
In the scene and in the body of a motif, the compiler follows this. In a group,
compiler.py:5166 (shape_here) keeps every exists-when until all the members are declared, so
(group g (exists-when c true) (circle|none c)) compiles. Bead mtp-ydd4.7.10 makes it an error.
Rule 4.18
The string of a formula declares the terms that it marks with \term{name}{…}.
The (defaults …) of a formula is an optional argument, so it comes before the body
(Rule 4.13).
The body holds the string, the terms, the exists-when forms and the variables. By
Rule 6.3, a (term name …) and an exists-when of a term come after the
string. So (formula f (defaults (padding 3)) tex'\term{a}{x}' (term a (fill 'red'))) is allowed,
and the same formula with (defaults …) last, or with (term a …) before the string, is an error.
Code differs
compiler.py:4905 (formula_decl) looks for the string, the (defaults …), the terms, the
exists-when and the variables at every place, so the string may come last, and (defaults …)
may come after a (term …). Bead mtp-ydd4.7.11 makes the compiler follow the rule.
Rule 4.19
A listener acts only on what happens in the view in whose body it is written. A listener on its
own view leaves out the name of the view: (on-view (animation …)), (on-pointermove target).
Table 4.1 gives the view in whose body each listener is written: a listener on a
shape, on-show, on-hide, on-view, on-ready, on-live, on-quiet and on-pointermove in a
canvas, a listener on a term of a formula, on-live and on-quiet in a display, and on-live and
on-quiet in a readout. A listener is never written at the top level and has no (in …), while a
control is written at the top level and names the views that it belongs to with (in …).
4.5. Imports#
Where an import may be written is given in Table 4.1. Everything else about
imports is in Imports.
4.6. An example#
This file has each form in a body that allows it: a variable, its docstring and its constraints at
the top level, the optional arguments of the scene before its shapes, a foreach of shapes in the
scene, and a canvas and a slider at the top level.
(real k 1 doc@'The radius of the circle.' (>= 0.2) (<= 2))
(scene s
(background 'white')
(circle c (r k) (fill 'crimson'))
(foreach i 3
(circle #'dot-{i}' (x (+ 3 i)) (r 0.2) (fill 'gray'))))
(canvas v (x -3) (y -3) (width 9) (height 6) s)
(slider size (move k (from 0.2) (to 2)) (label 'radius'))