Home /  Mappings /  How to / 

Use the Mappings API

The information on this page is written for software developers. We assume you're familiar with calling HTTP API endpoints.

Authentication

Before you can get started with the API, you need an API key. If you already have an API key for another Stedi product, you can use that here as well; you don't need a separate key for Mappings.

Once you have the key, include it in every call to the API as part of the Authorization header.

Authorization: Key $YOUR_API_KEY

If you'd like more information about this, take a look at our Authentication guide.

Anatomy of a mapping definition

A mapping definition describes how to turn input JSON into output JSON. It consists of several parts.

  • A set of mapping expressions, describing how to transform input fields to output fields.
  • A mapping type, specifying how output fields are determined.
  • A target example, providing default values for output fields.
  • A set of schemas, describing what the input and output JSON should look like.
  • Lookup tables, to make converting values in mapping expressions easier.

To help you get started quickly, we’ve published a Postman/Insomnia collection at our GitHub starter-kit repo, which you can import in seconds and get started.

Mapping expressions

A mapping expression is a piece of code that specifies how to turn input fields into an output field. For example, if you want to output a total price from a quantity and a unit price, it would look something like this.

{ "total": item.quantity * item.unit_price }

On the left is the name of the output field, on the right is the mapping expression, written in JSONata. You can add more output fields, separated by commas.

{ "total": item.quantity * item.unit_price, "currency": item.currency, "product": product.id }

The result looks just like JSON, even though it technically isn't, because of the syntax of JSONata.

Mapping types

The mapping type determines which fields show up in the output. All mapping types produce output fields based on the mapping expressions. Where they differ, is what happens to the output field if the mapping expression doesn't produce a value. This can happen when one or more of the input fields that the mapping expression depends on aren't present.

Only mapped keys

  • An output field will be added for every mapping expression that returns a result.
  • No output fields will be added based on the target example.
  • No output fields will be added based on the input.

For example, say you have the following mapping expression.

{ "currency": item.currency, "total": item.quantity * item.unit_price }

You provide the following input.

{
  "item": {
    "unit_price": 2.5,
    "currency": "USD"
  }
}

This will give the following output.

{
  "currency": 5
}

The output will not contain any fields, for the following reasons.

  • The field currency is added to the output, because there is a mapping expression for currency that produces a result.
  • The field total is not added to the output. There is a mapping expression for total, but it doesn't produce a result because item.quantity is missing from the input.
  • The fields item.unit_price and item.currency are not added to the output, because there are no mapping expressions for them.

Merge with target example

  • An output field will be added for every mapping expression that returns a result.
  • An output field will be copied from the target example, unless that output field was already added because of a mapping expression.
  • No output fields will be added based on the input.

For example, say you have the following mapping expressions.

{ "unit_price": item.price, "quantity": item.quantity, "total": item.quantity * item.price }

You also have the following target example.

{
  "currency": "USD",
  "quantity": 1,
  "unit_price": 0
}

You provide the following input.

{
  "item": {
    "price": 2.5
  }
}

This will give the following output.

{
  "currency": "USD",
  "unit_price": 2.5,
  "quantity": 1
}
  • The field currency is copied from the target example, because there is no mapping expression for currency.
  • The field unit_price is not copied from the target example, because there is a mapping expression for unit_price that produces a result.
  • The field quantity is copied from the target example. There is a mapping expression for quantity, but that one doesn't produce a result, because item.quantity is missing from the input.
  • The field total doesn't end up in the output. There's a mapping expression for total, but it doesn't produce a result and the target example doesn't contain a default value for total.

Pass through

  • An output field will be added for every mapping expression that returns a result.
  • An output field will be copied from the input, unless that output field was already added because of a mapping expression.
  • No output fields will be added based on the target example.

For example, say you have the following mapping expressions.

{
  "item": { "quantity": item.quantity * items_per_unit },
  "tax_rate": tax_rate < 1 ? tax_rate + 1 : tax_rate,
  "total": item.quantity * item.price
}

You provide the following input.

{
  "item": {
    "quantity": 5,
    "price": 2.5,
    "currency": "USD"
  },
  "tax_rate": 0.06
}

This will give the following output.

{
  "item": {
    "quantity": 5,
    "price": 2.5,
    "currency": "USD"
  },
  "tax_rate": 1.06,
  "total": 12.5
}
  • All fields are copied from the input.
  • The field tax_rate is overwritten, because there is a mapping expressions for tax-rate that produces a result.
  • The field item.quantity is not overwritten. There is a mapping expressions for item.quantity , but it doesn't produce a result, because items_per_unit is missing from the input.
  • The field total is added, because there's a mapping expression for total that produces a result.

Target example

The target example is a JSON object that provides default values in case a mapping expression doesn't return a result. For example, the following target example sets the default currency to USD.

{
  "default": {
    "currency": "USD"
  }
}

If the mapping expression can't determine the currency—for example because the input document doesn't contain currency information—then the default from the target example will be used. For this to work, the mapping type must be set to Merge with target example. Any other mapping type completely ignores the target example. Also, if you set the mapping type to Merge with target example, you must provide a target example. It can be empty, but it must be part of the mapping definition.

The target example is actually part of the target schema. The example above is a valid, albeit minimal JSON Schema.

Schemas

The source and target schemas are used by the Mappings UI. If you don't plan on using the Mappings UI—i.e. you'll use the API exclusively—then you generally don't need to provide schemas. The only exception is when you want to provided default values for your output fields.

Any schema you specify must be a valid JSON Schema (version 2020-12). Also, each schema needs to be self-contained, so you can't include references to external schemas.

Source schema

The Mappings UI uses the source schema to show an example of an input document. You can use this example to find and select the input fields you need in your mapping expressions. The following schema specifies three input fields: quantity , unit_price and currency.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "quantity": {
      "type": "number"
    },
    "unit_price": {
      "type": "number"
    },
    "currency": {
      "type": "string",
      "minLength": 3,
      "maxLength": 3
    }
  }
}

Just the schema isn't enough, though; you also need to provide some values for those fields. The Mappings UI will use the values to show what the output of a mapping expression will be like while you are crafting that expression, which is a useful feature. You can add values to the example by extending the source schema with a default document.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "quantity": {
      "type": "number"
    },
    "unit_price": {
      "type": "number"
    },
    "currency": {
      "type": "string",
      "minLength": 3,
      "maxLength": 3
    }
  },
  "default": {
    "quantity": 15,
    "unit_price": 2.5,
    "currency": "CAD"
  }
}

The Mappings UI will now show the target schema, with the provided values.

An example of the target schema rendered in the UI.

Target schema

The Mappings UI uses the target schema to show an example of an output document. You can only create mapping expressions for fields that are included in the target schema, so without a target schema, you can't use the Mappings UI. The following schema specifies two output fields: total and currency.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "total": {
      "type": "number"
    },
    "currency": {
      "type": "string",
      "minLength": 3,
      "maxLength": 3
    }
  }
}

As with the source schema, you can add a default document to your target schema. Unlike with the source schema, this doesn't help with writing mapping expressions. However, if you want to provide default values for your output fields, adding a default document to your target schema is the way to do that. The following example sets the default currency to USD.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "total": {
      "type": "number"
    },
    "currency": {
      "type": "string",
      "minLength": 3,
      "maxLength": 3
    }
  },
  "default": {
    "currency": "USD"
  }
}

Default values only work if you set the mapping type to Merge with target example.

Lookup tables

A lookup table makes it easy to convert a value from one representation to another. For example, the following table allows you to translate country names.

englishspanishgerman
United StatesEstados UnidosVereinigte Staaten
MexicoMéxicoMexiko
GermanyAlemaniaDeutschland

There's no primary key in a lookup table. You can freely translate from English to Spanish, from Spanish to German, from German to English; any direction you want. You specify this in the mapping expression using the $lookupTable function. The following example uses the lookup table called countries to translate the input field land (which is German for country) from German to English.

{
  "country": $lookupTable($tables.countries, "german", land).english
}

In JSON, the above lookup table looks like this.

[
  {
    "english": "United States",
    "spanish": "Estados Unidos",
    "german": "Vereinigte Staaten"
  },
  {
    "english": "Mexico",
    "spanish": "México",
    "german": "Mexiko"
  },
  {
    "english": "Germany",
    "spanish": "Alemania",
    "german": "Deutschland"
  }
]

Managing mapping definitions

Before you can map documents from one structure into another, you need a mapping definition.

Creating a mapping definition

You create a mapping definition by POSTing to https://mappings.us.stedi.com/2021-06-01/mappings. At the very least, your mapping definition should have a name, a mapping types, and a few mapping expressions.

POST https://mappings.us.stedi.com/2021-06-01/mappings HTTP/1.1
Authorization: Key $YOUR_API_KEY
Content-Type: application/json

{
  "name": "A minimal example",
  "type": "only_mapped_keys",
  "mapping": "{ \"total\": item.quantity * item.unit_price }"
}

The mapping expressions look like JSON, but they're not. JSONata has all kinds of syntax that isn't valid JSON. That's why you need to provide the mapping expressions inside of a string.

Including a target example

If you want to use the mapping type Merge with target example, then you need to provide a target example. You do this by including a target schema that contains the target example. The following mapping definition provides a default value of 0 for the output field total.

POST https://mappings.us.stedi.com/2021-06-01/mappings HTTP/1.1
Authorization: Key $YOUR_API_KEY
Content-Type: application/json

{
  "name": "A target example example",
  "type": "merge_with_target_example",
  "mapping": "{ \"total\": item.quantity * item.unit_price }",
  "target": {
    "name": "target.json",
    "type": "jsonschema@2020-12",
    "content": "{\"$schema\": \"https://json-schema.org/draft/2020-12/schema\", \"default\": {\"total\": 0 }}"
  }
}

The target schema needs to be passed inside a string. That makes the above example a bit hard to read, so here's the target schema with more sane formatting.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "default": {
    "total": 0
  }
}

Including a lookup table

You can add one or more lookup tables by providing the lookup_tables field. Every lookup table must have a name and a set of values.

POST https://mappings.us.stedi.com/2021-06-01/mappings HTTP/1.1
Authorization: Key $YOUR_API_KEY
Content-Type: application/json

{
  "name": "A lookup table example",
  "type": "only_mapped_keys",
  "mapping": "{ \"country\": $lookupTable($tables.countries, \"german\", land).english }",
  "lookup_tables": [
    {
      "name": "countries",
      "values": [
        {
          "english": "United States",
          "spanish": "Estados Unidos",
          "german": "Vereinigte Staaten"
        },
        {
          "english": "Mexico",
          "spanish": "México",
          "german": "Mexiko"
        },
        {
          "english": "Germany",
          "spanish": "Alemania",
          "german": "Deutschland"
        }
      ]
    }
  ]
}

Response

The response contains a copy of the mapping definition you just created, plus a couple of extra fields.

FieldDescription
idThe unique identifier for your mapping definition. You need this if you ever want to update, delete, or use your mapping definition.
created_atA timestamp indicating when you created this mapping definition.
updated_atA timestamp indicating when you last updated this mapping definition. If you've never updated the mapping definition, this is the same as created_at.
HTTP/1.1 200 OK
Content-Type: application/json

{
  "id": "01AB3DEF4GHIJ5KL67MNOP8QR9",
  "name": "A minimal example",
  "type": "only_mapped_keys",
  "mapping": "{ \"total\": item.quantity * item.unit_price }"
  "created_at": "2022-01-01T20:22:01.01Z",
  "updated_at": "2022-01-01T20:22:01.01Z"
}

Retrieving a mapping definition

You retrieve a mapping definition by GETting it from https://mappings.us.stedi.com/2021-06-01/mappings/mapping_id. The response is identical to the one you received when you created the mapping definition. In other words, it's the mapping definition as you specified it, supplemented with the fields id, created_at, and updated_at.

Updating a mapping definition

You update a mapping definition by PUTting a new version at https://mappings.us.stedi.com/2021-06-01/mappings/mapping_id. You need to specify an entire new mapping definition, so when we talk about updating the mapping definition, we really mean replacing it.

Other than that, updating a mapping definition is the same as creating one: the request body is the same, the response body is the same.

Deleting a mapping definition

You delete a mapping definition by DELETEing it at https://mappings.us.stedi.com/2021-06-01/mappings/mapping_id. That's all there is to it, really. You don't need to provide a request body and the response will be empty as well.

Listing mapping definitions

You retrieve a list of all your mapping definitions by GETting it from https://mappings.us.stedi.com/2021-06-01/mappings.

GET https://mappings.us.stedi.com/2021-06-01/mappings HTTP/1.1
Authorization: Key $YOUR_API_KEY

The response has a field mappings which contains a list with mapping definitions.

HTTP/1.1 200 OK
Content-Type: application/json

{
  "mappings": [
    {
      "id": "01AB3DEF4GHIJ5KL67MNOP8QR9",
      "name": "A minimal example",
      "type": "only_mapped_keys",
      "created_at": "2022-01-01T20:22:01.01Z",
      "updated_at": "2022-01-01T20:22:01.01Z"
    },
    {
      "id": "02JK3LMN4OPQR5ST67UVWX8YZ9",
      "name": "A target example example",
      "type": "merge_with_target_example",
      "created_at": "2022-01-01T20:22:01.01Z",
      "updated_at": "2022-01-01T20:22:01.01Z"
    }
  ]
}

Metadata

The list in the response doesn't contain complete mapping definitions; only metadata. It includes the fields id, name, type, created_at, and updated_at, but not the mapping expressions or the schema. If you're only displaying a list of mapping definitions, this is typically what you want, but should you need the full mapping definitions, you can set the query parameter metadata_only to false.

GET https://mappings.us.stedi.com/2021-06-01/mappings?metadata_only=false HTTP/1.1
Authorization: Key $YOUR_API_KEY

The response will now include the mapping expressions and the schemas.

HTTP/1.1 200 OK
Content-Type: application/json

{
  "mappings": [
    {
      "id": "01AB3DEF4GHIJ5KL67MNOP8QR9",
      "name": "A minimal example",
      "type": "only_mapped_keys",
      "mapping": "{ \"total\": item.quantity * item.unit_price }"
      "created_at": "2022-01-01T20:22:01.01Z",
      "updated_at": "2022-01-01T20:22:01.01Z"
    },
    {
      "id": "02JK3LMN4OPQR5ST67UVWX8YZ9",
      "name": "A target example example",
      "type": "merge_with_target_example",
      "mapping": "{ \"total\": item.quantity * item.unit_price }",
      "target": {
        "name": "target.json",
        "type": "jsonschema@2020-12",
        "content": "{\"$schema\": \"https://json-schema.org/draft/2020-12/schema\", \"default\": {\"total\": 0 }}"
      },
      "created_at": "2022-01-01T20:22:01.01Z",
      "updated_at": "2022-01-01T20:22:01.01Z"
    }
  ]
}

Paging

If you have a lot of mapping definitions, you may not receive them all in one response. In that case, the response will contain a field next_page_token.

HTTP/1.1 200 OK
Content-Type: application/json

{
  "mappings": [
    ...
  ],
  "next_page_token": "0t1H23ER4e5ArEMORE6REsU78l_TSyET"
}

You can make another request passing the value of that field to the query parameter page_token and you will receive the next few mapping definitions.

GET https://mappings.us.stedi.com/2021-06-01/mappings?page_token=0t1H23ER4e5ArEMORE6REsU78l_TSyET HTTP/1.1
Authorization: Key $YOUR_API_KEY

As long as the field next_page_token shows up in the response, there are more mapping definitions to retrieve.

Mapping documents

Once you have a mapping definition, you can map your JSON documents from one shape to another.

Mapping a single document

You can map a JSON document by POSTing it to https://mappings.us.stedi.com/2021-06-01/mappings/mapping_id/map. That URL includes the ID of the mapping definition you want to use, and the body of the request is the JSON you want to map.

POST https://mappings.us.stedi.com/2021-06-01/mappings/01AB3DEF4GHIJ5KL67MNOP8QR9/map HTTP/1.1
Authorization: Key $YOUR_API_KEY
Content-Type: application/json

{
  "quantity": 15,
  "unit_price": 2.50
}

The body of the response is the mapped document.

HTTP/1.1 200 OK
Content-Type: application/json

{
  "total": 37.50
}

Mapping multiple documents

There is no endpoint that allows you to map multiple documents at once, so if you want to map multiple documents, you have to make a separate request for each document. This is easy enough to do for a small set of documents, but if you want to map, say, thousands of documents in a short time, you need to take a little bit more care.

Mappings allows you to do up to 200 concurrent requests, so if you need to map 200 documents or less, you can send them all at once. If you need to map more than 200 documents, you should make sure that you don't exceed 1000 requests per second. Also keep in mind that there's a maximum of one million requests per day.

If you make more requests than Mappings is able to handle, you will receive a 429 Too Many Requests response.

  • Keep track of how many outstanding requests you have. Don't send any new requests if you have 200 or more.
  • Keep track of how many total requests you've sent in a day and make sure it doesn't exceed 1,000,000.
  • Keep a timestamp for every successful response you receive. When sending a new request, first check the timestamp for 1000 requests ago. If it's less than a second ago, wait until the second has passed before making a new request.
  • If you receive a 429 Too Many Requests response, wait for one second before sending new requests. If you still get back 429 after waiting for one second, then wait for two seconds. If that's not enough wait for four, then eight, etc.
  • If you receive a 429 Too Many Requests response, add the document back into the list of documents that you need to map. Don't resend the request immediately.
AuthenticationAnatomy of a mapping definitionManaging mapping definitionsMapping documents