Expressions#

You write every expression: in one arithmetic grammar: in a constraint, the objective, a named expression and a macro template.

expression  ::= arithmetic | arithmetic COMPARATOR arithmetic
arithmetic  ::= atom | unary_op arithmetic | arithmetic binary_op arithmetic
             |  function_call | "(" arithmetic ")"
atom        ::= NUMBER | NAME
unary_op    ::= "+" | "-"       binary_op ::= "+" | "-" | "*" | "/" | "**"
COMPARATOR  ::= "<=" | ">=" | "=="
function_call ::= NAME "(" [pos_arg ("," pos_arg)*] ["," kwarg ("," kwarg)*] ")"
kwarg       ::= NAME "=" (arithmetic | QUOTED | "[" NAME ("," NAME)* "]" | COLUMNS)
COLUMNS     ::= NAME "[" NAME ("," NAME)* "]"
NAME        ::= [a-zA-Z_][a-zA-Z0-9_]*
NUMBER      ::= integer | float | "inf" | ".inf"

• Operators bind in this order, highest first: **, then unary + and -, then * and /, then binary + and -. So -x ** 2 is -(x ** 2), as in Python.
• A float may carry an exponent, as in 1e5 or 2.5e-3.
• The same keyword twice in one call is an error.
• An expression and a where: string nest at most 100 levels deep, and at most 300 with every named expression they read written in.

Where a product of two variables is allowed#

The objective and the constraints take variable * variable. A quadratic cost is sum(p * p * wear, over=g), and a quadratic row is p * q >= floor. Three rules limit it:

• At most one factor may be a sum of terms, so sum(p, over=g) * sum(q, over=g) is refused. Multiply before you reduce, or constrain a variable to equal the reduction. Factors on different dimensions are allowed: x * y * link broadcasts, and the table link says which pairs exist.
• The degree stops at 2, so p * p * p is refused.
• A bound and a piecewise: link stay affine. A bound is one number per column.

A reported expression is not held to these.

/ needs a divisor that carries no variable, and ** needs a base and an exponent that carry no variable. Those places accept any arithmetic over numbers and parameters, so you write a discount factor where you use it:

dimensions:
  period: { dtype: int }
parameters:
  rate: { dims: [] }
  years: { dims: [period] }
  price: { dims: [period] }
variables:
  build: { dims: [period], bounds: { lower: 0 } }
constraints:
  enough: { dims: [], expression: "sum(build, over=period) >= 1" }
objective:
  sense: minimize
  expression: sum(build * price / (1 + rate) ** years, over=period)

years is a parameter over period. The exponent cannot be period itself, because an expression reads no dimension. Write x * x for a square.

Name resolution#

One flat namespace covers dimensions, relations, parameters, variables, named expressions, masks, macros and the built-in operators. Two entries with one name are a load error, and the message names both, so no name hides another.

The place where a name stands decides which kinds of name are legal:

A bare word in the value of a keyword argument is a name to resolve, but a keyword's key is never a name.

Four sections sit outside the flat namespace: constraints, assumptions, sos and piecewise. No expression reads them by a bare name, so a name in one of them may be the name of an entry in the flat namespace, or of an entry in one of the other three. dual() is the one reader of a constraint, by its name. A term adds_to a named expression, and never to a constraint of the same name.

How dimensions combine#

to_spec knows the dimension set of every expression before any data is attached:

An outer product is allowed. The dimensions an entry ranges over are its frame, and an entry may not disagree with its expression:

• A constraint requires dims(lhs) ∪ dims(rhs) to equal its dims.
• An objective must carry no dimensions, so write the sums that reduce it.
• A where predicate and a bound parameter must not exceed the frame they sit in.

Each of these is a load error.

where strings#

A where: is a boolean mask, and true means "this coordinate exists".

where_expr ::= atom | "NOT" where_expr | where_expr ("AND"|"OR") where_expr
            |  "(" where_expr ")"
atom       ::= NAME | (NAME | COLUMN) COMPARATOR value | expression COMPARATOR expression
            |  POSITION COMPARATOR INTEGER | COUNT COMPARATOR INTEGER | TRANSLATED
            |  READ | "True" | "False"
COMPARATOR ::= "<=" | ">=" | "==" | "!=" | "<" | ">"
value      ::= NUMBER | QUOTED | NAME_OR_STRING | COLUMN
expression ::= the arithmetic grammar above, with no variable and no dual in it
POSITION   ::= "position" "(" NAME [ "," "within" "=" COLUMNS ] ")"
COUNT      ::= "count" "(" where_expr "," "over" "=" NAME ")"
TRANSLATED ::= "shift" "(" where_expr "," "along" "=" NAME "," "offset" "=" INTEGER ")"
READ       ::= "at" "(" where_expr "," "by" "=" COLUMNS ")"
COLUMN     ::= NAME "[" NAME "]"
COLUMNS    ::= NAME "[" NAME { "," NAME } "]"
QUOTED     ::= "'" chars "'" | '"' chars '"'

A bare name that is not declared is a load error.

At a missing row, a comparison and a bare bool name read the value that the missing: of the parameter names. A bare numeric name asks whether the data has a row, whatever the missing:.

Counting what a predicate admits#

count(<where_expr>, over=<dimension>) is how many coordinates along that dimension the predicate is true at:

dimensions:
  bp: { dtype: int }
  generator: { dtype: str }
parameters:
  bp_x: { dims: [generator, bp] }
  points: { dims: [generator, bp], dtype: bool }
variables:
  p:
    dims: [generator]
    where: "count(points, over=bp) >= 2"
    bounds: { lower: 0 }
constraints:
  cap:
    dims: [generator]
    expression: p <= 1
objective:
  sense: minimize
  expression: sum(p, over=generator)

The dimension counted over is removed, as a sum(over=) removes it: one count per generator above. Counting along a dimension the predicate does not read is a load error.

The right-hand side is a whole number. A fraction, a parameter, and a comparison a count can never fail or never meet (>= 0, < 0, or any negative number) are load errors.

Reading a predicate at the previous coordinate#

shift(<where_expr>, along=<dimension>, offset=<integer>) reads the predicate offset coordinates back. It is false where the translation vacates, and it takes no edge=. With count, it names the start of a run:

where: "count(points AND NOT shift(points, along=bp, offset=1), over=bp) == 1"

That where is true when the marked breakpoints are one consecutive run.

A negative offset reads forwards. This form accepts neither within= nor edge='wrap'. For a grouped or cyclic translation, compare the arithmetic shift.

Reading a predicate through a relation#

at(<where_expr>, by=<relation>[<column>]) reads a predicate over coarse coordinates at fine ones, as at reads an array. It is true where the relation has a row and the predicate holds at the coordinate that row maps to, and false where the relation has no row.

dimensions:
  converter: { dtype: str }
  flow: { dtype: str }
relations:
  converter_of: { key: flow, values: converter }
parameters:
  has_curve: { dims: [converter], dtype: bool }
  cap: { dims: [flow] }
variables:
  rate:
    dims: [flow]
    where: "at(has_curve, by=converter_of[converter])"
    bounds: { lower: 0, upper: cap }
objective:
  sense: minimize
  expression: sum(rate, over=flow)

The read removes the column it reads, converter, and adds the key, flow, so the mask above is over flow alone. The rules are those of at in an expression. by= names a value column, or several as relation[a, …]. The read groups by the key of the relation, and the predicate carries every dimension that the read joins on.

The right-hand side of a comparison#

A bare name on the right is read as a string label when the spec does not declare it, and a declared name there is a load error.

Quote a label that is not an identifier, such as 'combined-cycle'. A quoted word is never read as an entry.

A comparison is checked against the declared dtype. A datetime dimension is compared against a quoted ISO date such as '2030-01-01' or '2030-01-01T06:00', and a number against it is a load error.

String labels compare bytewise, whatever the dimension's order, and a label the dimension does not carry compares equal to nothing.

The language cannot compare two dimensions, so precompute a boolean parameter instead.

Arithmetic in a comparison#

Either side of a comparison may be an expression over parameters: p_min <= 0.5 * p_max, or p_max <= at(bus_cap, by=bus_of[bus]). The side is read as an expression is, macros and named expressions included. A variable and a dual() are refused. A relation column and a quoted label are compared on their own, and are not read in arithmetic.

A side absent at a coordinate compares false there, but under a summing operator the absent term is one summand fewer. To compare against the previous row, give the shift an edge=, and add a position() term to keep the first row out:

dimensions:
  snapshot: { dtype: int, ordered: true }
parameters:
  load: { dims: [snapshot] }
  ramp: { dims: [] }
variables:
  shed: { dims: [snapshot], bounds: { lower: 0 } }
constraints:
  shed_when_load_jumps:
    dims: [snapshot]
    where: "load - shift(load, along=snapshot, offset=1, edge=0) > ramp AND position(snapshot) > 0"
    expression: shed >= load - ramp

A case when: may compare expressions only where cases: holds one case. A comparison with a number on both sides, such as 2 < 1, is refused everywhere.

position()#

position(dim) counts along the order shift steps along, not the label:

dimensions:
  snapshot: { dtype: int, ordered: true }
parameters:
  soc_initial: { dims: [] }
variables:
  soc: { dims: [snapshot], bounds: { lower: 0 } }
constraints:
  soc_start:
    dims: [snapshot]
    where: "position(snapshot) == 0" # not: snapshot == 0
    expression: soc == soc_initial

A position that no coordinate occupies is an error when the data is attached.

within= counts inside each group a partition makes, so each period gets one seeded row:

dimensions:
  snapshot: { dtype: int, ordered: true }
  period: { dtype: int }
relations:
  period_of: { key: snapshot, values: period }
parameters:
  soc_initial: { dims: [period] }
variables:
  soc: { dims: [snapshot], bounds: { lower: 0 } }
constraints:
  soc_start:
    dims: [snapshot]
    where: "position(snapshot, within=period_of[period]) == 0"
    expression: soc == at(soc_initial, by=period_of[period])