Skip to main content
A shape is a JSON object sent in the request body to POST /v1/org/employees/shape. It describes how an Employee should be projected into a custom output payload, instead of returning Twine’s default representation. By default, every employee field is exposed as an array of dated property records. That structure preserves history but is awkward when the integrating system needs only a few specific fields, flattened, renamed, or reshaped to match its own model. A shape removes that friction: each output key is described inline, the response comes back in exactly that form, and no post-processing is needed on the caller’s side. Shapes are inline only. There is no library of saved shapes on the server, and shapes cannot be referenced by name. Every request carries its own shape in the body. (Stored, reusable shapes are on the roadmap - see Roadmap.)

Background: dated properties, tombstones, and streams

Most fields on a Twine entity are not single scalars but lists of dated property values. A dated property is {value, valid_from, id}, and several of them per field encode the history of that field over time. By convention, a dated property with value: null and a non-null valid_from is a tombstone: it signals that the field’s value ended on valid_from - 1 day. Tombstones are markers, not data. The model is append-only, so an end-of-validity is recorded by adding a tombstone rather than mutating the prior entry. The id field on a dated property acts as a stream identifier. Most properties use a single stream, with id: null on every entry. Stream ids only appear for properties whose values need to coexist in parallel at the same point in time - salary supplements, benefit allocations, and, in the rarer cases where they occur, parallel employments. The id may be a source-system primary key (an employment row’s id, a salary record’s id) or a plain discriminator such as "main" or "bonus" - whatever the source uses to keep the streams apart. Same-key entries with different ids belong to independent timelines and must not be merged. By default the shape endpoint treats every entry under a given key as a single timeline (newest valid_from wins regardless of id). The $per_id directive is what projects per-stream rows. These conventions are what $historical and $per_id blocks rely on, so they are worth keeping in mind throughout the rest of this page. See Dated Properties for the full data model.

Shape values

Every value in a shape is one of the following forms. Output keys with no $ prefix become keys in the response. Keys with a $ prefix are directives, and only the names listed below are recognised; anything else is rejected at parse time. $per_id and $historical may be combined in the same block (a flat (id × valid_from) array) or nested with $per_id on the outside and $historical on the inside (per-stream history grouping). The reverse nesting - $historical on the outside, $per_id on the inside - is rejected at parse time.

@<key> - latest property value

Reads the newest applicable dated property value for <key>. The “newest” entry is the one with the largest valid_from (entries with valid_from: null sort last). When <key> carries multiple streams, all entries are merged and the single newest one wins, regardless of stream id.
To read an org-defined custom property, prefix the name with custom_properties., for example "@custom_properties.favorite_color".

$id, $inserted_at, $updated_at - source metadata

These three directives read identifying metadata from the source record itself, rather than any of its properties. $id is the source entity’s primary key (a UUID for an Employee); it has nothing to do with the per-stream stream id used by $per_id.

Field specs ($field)

A flat field value can be written either as the "@<key>" shorthand or as a $field object that adds optional behaviours. The two forms are interchangeable when no extra keys are set: "@salary_amount" is identical to { "$field": "@salary_amount" }.
  • $field (required) - the property reference, exactly like the "@<key>" shorthand. A static field or custom_properties.<name>. It must be a @-prefixed string.
  • $at (optional) - resolve the value as of a date instead of taking the newest one. See Point-in-time.
  • $type, $coerce, $on_error (optional) - validate or coerce the value to a scalar type. See Type validation.

Point-in-time ($at)

By default a flat field returns the value of the newest dated property, which may be a tombstone (null) if the field has ended. Supplying a date instead resolves the value in effect on that date: the newest entry whose valid_from is less than or equal to the date. If no entry applies (the date precedes every entry), the result is null. The moment can be set in three places, listed from most to least specific. The most specific one present wins:
  1. Per-field - $at inside a $field spec.
  2. Request body - a top-level $at key in the shape root. It applies to every flat field that has no per-field $at.
  3. Query string - ?at=YYYY-MM-DD on the request URL.
A per-field $at overrides the body root $at, which overrides the ?at= query parameter. If none is set, the newest value is returned. The body root $at is a directive, not an output key - it does not appear in the response. The moment applies only to flat reads (@<key> and $field). It does not reach into $all (which always returns the full value list) or $historical blocks (whose rows govern their own moments). For an employee whose salary_amount is 2000 from 2021-01-01 and 1000 from 2020-01-01, a request to /v1/org/employees/shape?at=2020-06-01 with the shape { "salary": "@salary_amount" } returns 1000 - the value in effect on that date. The same result comes from a body root $at or a per-field $at.

Type validation ($type)

$type declares the scalar type the caller expects. The allowed types are string, number, integer, float, boolean, date, datetime, time, and enum.
  • With $coerce: false (the default) the value is only validated: if it already matches the type it passes through unchanged, otherwise it is a type mismatch.
  • With $coerce: true Twine attempts to convert the value to the type (for example the string "42" becomes the integer 42).
  • null always passes (an absent value is not a type error). List-valued properties pass through unchanged - type checks apply to scalar values only.
A few representative coercions: "5" to integer 5; 2.4 to integer 2; "1.5" to float 1.5; "2024-01-02" to date; "yes"/"1" to boolean true and "no"/"0" to boolean false; 100000 to string "100000". A value that cannot be converted (for example "abc" to integer, or "maybe" to boolean) is a mismatch. When a value fails validation or coercion, $on_error decides the output value: In all cases the request still returns 200, and a warning is recorded - the endpoint never fails the whole request because one value of one employee was the wrong type. (set_to_nil and error produce the same null output; they differ only in the action recorded on the warning.) A successful coercion is also recorded as a warning, so callers can see what was changed. Warnings are returned in a top-level warnings array, alongside data and pagination. Each entry carries:
If employee_no holds the string "E-1001", the value cannot be validated as an integer, so the output key becomes null and a warning is appended:
Declaring $type with an unknown type, a non-boolean $coerce, or an unrecognised $on_error mode is a parse-time 400 (see Parse-time rejections). Only a runtime value mismatch produces a warning. $at is not allowed inside a $historical block, where the row’s own valid_from already governs the moment.

$all - every value of a property

Returns every dated property value for the referenced key as a flat array, dropping the valid_from annotations. Useful when only the value history is needed, not the dates.
$all cannot have sibling keys. The object must contain only $all and nothing else.

Plain nested object

Any object that is not a directive block ($field, $all, $relation, $historical, $per_id) is treated as a plain nested object and shaped recursively.

Historical blocks

A $historical block expands a single output key into an array of rows, one per unique valid_from across all @property references inside the block. The output key for each row’s valid_from value is given by $valid_from_field. If $valid_to_field is present, each row also carries an end-of-validity date. At each row’s valid_from, every referenced property is resolved by carry-forward: the latest dated property whose valid_from is less than or equal to the row’s valid_from is used. If no such entry exists for a given property, that property’s output key is omitted from the row entirely. This is distinct from “key present, value null” - a null value comes from a tombstone or an explicit nil entry, while an absent key means no entry was applicable at all. Rows are returned in descending order by valid_from (newest first). Rows whose valid_from is null sort last.

Example A - carry-forward, no end date

Source:
Shape:
Response:
The last row has no amount key because no salary_amount entry has valid_from: null - there is nothing to carry forward to that row. frequency, on the other hand, has an entry with valid_from: null and so applies to every row.

Example B - with $valid_to_field, no tombstone

When $valid_to_field is present, each row’s valid_to is computed from the next-newer row’s valid_from - 1 day. The newest row’s valid_to is null, meaning open-ended. Source:
Shape:
Response:

Example C - joint tombstone consumed

If the newest row in a historical block is a joint tombstone - every referenced @property resolves to null at that valid_from - the row is dropped from the output and consumed as the end marker for the row immediately before it. The prior row’s valid_to becomes tombstone.valid_from - 1 day. Source:
Shape:
Without consumption, the output would carry a leading 2022-06-01 row with both amount and currency set to null:
Because both referenced properties tombstone at the same date, that row is consumed and the response is:

Example D - partial-nil row survives

A row where some referenced properties are null but at least one is non-null is not a joint tombstone. It survives in the output as a partial-value row, and its valid_to follows the standard rule. Source:
Same shape as Example C. Response:
salary_amount tombstones at 2022-06-01, but salary_currency carries "USD" forward, so the row is kept.
Per-property end dates are not tracked inside multi-property historical blocks. The $valid_to_field annotation is row-level only. A historical block referencing a single property is what to use when per-property resolution is needed.

Streams ($per_id)

Several has-many properties (employments, salaries, rates, competences, and others) produce multiple parallel streams under a single property key, each tagged with a stream id. Without $per_id, all streams collapse: @employment_start_date returns the newest start date across every employment, regardless of which employment it belongs to. With $per_id, the block becomes an array, one element per unique non-null stream id observed across the block’s referenced properties. Scoping inside a $per_id row. Every @<key> reference in the block is filtered to dated property entries whose id equals the row’s id. A property with no entry under that id is omitted from the row entirely (the output key is not present), rather than being emitted as null. Entries with id: null are not visible inside $per_id rows. Output ordering. Rows are sorted by each stream’s most recent valid_from, newest first. Streams whose newest valid_from is null sort last. $id_field is optional. When set, each row carries the stream id under that output key. When omitted, the id is still used to partition rows but does not appear in the response.

Example - per-stream projection

Response:

Example - $per_id and $historical on the same block

A flat array of (id × valid_from) rows. Tombstones are consumed within their own stream and never bridge across stream ids.

Example - nested form (per-stream history)

$per_id on the outside and $historical on the inside groups history under each stream:
Each element in employments is { "id": ..., "history": [...] }, where history is scoped to that one stream’s dated property entries. The reverse nesting - $historical on the outside, $per_id on the inside - is rejected at parse time, because per-valid_from rows have no useful per-stream sub-structure.

Relations ($relation)

A $relation block projects a related collection of the employee as an array of rows. The block names the relation kind and maps output keys to the related record’s own fields:
Response:
Key points:
  • The relation kinds mirror the relations parameter of the regular employees endpoint: organizational_unit_mappings and employee_mappings.
  • Inside a $relation block, @<field> references the related record’s own fields - a different namespace from the employee’s dated property fields. Only the relation’s whitelisted fields are referenceable:
    • organizational_unit_mappings: id, org_id, inserted_at, updated_at, organizational_unit_id, external_organizational_unit_id, employee_id, external_employee_id.
    • employee_mappings: id, org_id, inserted_at, updated_at, external_id, status, status_updated_at, system_integration_id, employee_id.
  • Relation blocks are flat. $historical, $per_id, $all, nested objects, and nested $relation are not allowed inside them.
  • A relation that is empty or not loaded yields an empty array.
  • Relations expose only identifiers and metadata, so they are not subject to the per-field authorization that applies to @property references. They are gated by the :list permission alone, matching the regular endpoint.

Custom properties

Organizations can define their own custom fields, stored under a separate custom_properties bag on the employee. Reference one by prefixing its name with custom_properties.:
Custom properties behave exactly like static @property references. They work in $all, $historical, $per_id, and $field specs (including $at and $type). The name after the prefix is the custom property’s name as defined for the organization; an unknown name is rejected at parse time. Like the regular employees endpoint, custom properties are returned regardless of the role’s authorized field set - they are not part of per-field authorization.

Parse-time rejections

A shape is validated in full before evaluation. Errors are aggregated, so all problems are reported at once rather than one at a time. Each rejection returns HTTP 400 with a structured errors array; each entry carries domain: "DataShape", a reason from the table below, and a message containing the path inside the shape where the problem was found.

Authorization

Field-level authorization is enforced on the parsed shape, before evaluation:
  • The role calling /v1/org/employees/shape needs the :list action on :employee.
  • Every static @property reference - including the @property inside a $field spec and inside a { "$all": "@property" } block - is checked against the role’s authorized_fields. Any reference to a field outside that set fails the request with HTTP 403 and a structured error (reason: "forbidden_field") listing the offending property name(s).
  • Not checked against authorized_fields: custom_properties.<name> references and fields inside $relation blocks. These mirror the index endpoint, which returns custom properties and relations without per-field narrowing, so they are gated by the :list permission alone. This is intentional, not an inconsistency.
This is stricter than the regular employee index endpoint, which silently narrows the response to the fields the role is allowed to see. The shape endpoint fails loudly instead, because partial silent payloads would not match the shape the caller asked for and would be hard to diagnose.

End-to-end example

A complete request and response, combining a top-level @property, the $id and $inserted_at directives, a nested object, a $historical block with $valid_to_field, an $all projection, a $field spec with $at and $type, a $relation block, and a @custom_properties.<key> reference. The ?at=2020-06-01 query parameter sets the request-level moment for flat reads.
Request body (shape.json):
Response:
A few things to notice in the response:
  • salaryAsOf reads @salary_amount as a flat field, so the ?at=2020-06-01 moment resolves it to 1000 (the value in effect on that date). salaryAmounts ($all) and salaries ($historical) are unaffected by the moment and still show the full history.
  • employeeNo declared $type: "integer" but the source value "E-1001" is not an integer, so the output is null and a warning is recorded. The request still returns 200.
  • warnings is an empty array ([]) when nothing failed validation.
The endpoint paginates the same way as the regular employee list endpoint. Each item in data is the result of evaluating the shape against one employee record, in the same order the underlying list endpoint would return.

Roadmap: stored validation schemas

This capability is planned and not yet available. It is described here only so integrators can anticipate it - the request and response formats are not final.
Today every shape is inline: it travels in the request body and is not stored. A future addition will let an organization save a named shape (a validation schema) in Twine and:
  1. Reference it by name from the shape endpoint instead of sending the body inline.
  2. Reuse the $type validation rules after each sync to automatically raise data-quality anomalies when a synced value does not match the declared type, resolving once the value conforms again.
Until then, shapes are inline-only and $type validation applies only to the response of the request that declares it.