Skip to main content

Target Data Model API

Target data models define the structure of the output data in all Ingestro pipelines. They ensure that the mapped and transformed data is organized according to the desired format and schema required by the destination system.

Within a target data model, you can define the name of each column at both a UI-based level (label) and a technical level (key). Additionally, you can apply required, unique, and regex validations, as well as specify the column type (e.g., float, int, date, currency, etc.).

This ensures that the pipeline’s output not only has the correct structure but also that the values are in the correct format.

Use this base URL and add the corresponding endpoint respectively:

Base URL

api-gateway.ingestro.com/dp/api/v1/

Create​

Endpoint

POST /tdm/

Payload​

Attributes

name

The name of the TDM

columns

List of all columns in your TDM

key

The column key used to uniquely identify each column

label

The name of the column as displayed to users in the interface

columnType

The type of the column, defining which data type and format is expected for this column. Learn more about column types here

optionConfiguration

Configuration for category columns that defines available options and their mapping behavior

options

List of available options for the dropdown

label

The display name of the option shown to users

value

The actual value stored when this option is selected

type

The data type of the value

  • STRING: Text value
  • NUMBER: Numeric value

alternativeMatches

Alternative text values that should map to this option during data import

description

Optional description providing additional context about the option

mappingConfiguration

Configuration for how imported values are mapped to options

layers

Mapping strategies to use

  • EXACT: Exact match
  • SMART: AI-based matching
  • FUZZY: Similarity-based matching

threshold

Minimum similarity score (0.0-1.0)

optionSource

Source of the options

  • STATIC: Predefined options
  • DYNAMIC: Fetched from external source

dynamicOptionFetch

Configuration for fetching options dynamically from an external API

url

The endpoint URL to fetch options from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Authentication configuration for the dynamic option fetch endpoint

method

The authentication method to use

refresh_url

The endpoint to retrieve the access token from

headers

The list of key-value pairs that define the headers for the authentication request

multiSelect

Whether multiple options can be selected

  • true: Multiple selection allowed
  • false: Single selection only

mappingValidation

A validation rule to indicate under which conditions the column has to be mapped. Learn more about validations here

logic

Set a pre-defined validation rule. Refer to this page here for more info

errorMessage

Set a custom error message for one of the validation rules above

dataValidations

A list of value based validation rules applied to the column. Learn more about validations here

logic

Set a pre-defined validation rule. Refer to this page here for more info

errorMessage

Set a custom error message for one of the validation rules above

advancedValidations

url

The endpoint that recevies the data

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Defines your refresh token endpoint to obtain a new access token after the previous one has expired. This mechanism is more secure and allows you to re-authenticate every time you need to obtain a new access token

refresh_url

The endpoint to retrieve the access token from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

payloadConfiguration

Defines which columns' data is sent to the endpoint and the size of the batches the data is sent in

columns

The list of columns that are sent to the endpoint

batchSize

The size of the badges that are sent to the endpoint. The default is 10.000 rows and the maximum is 25.000 rows per badge.

triggerConfiguration

Defines which columns trigger sending the data to the endpoint for validation

columns

The list of columns that trigger sending the data to the endpoint

advancedCleanings

url

The endpoint that recevies the data

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Defines your refresh token endpoint to obtain a new access token after the previous one has expired. This mechanism is more secure and allows you to re-authenticate every time you need to obtain a new access token

refresh_url

The endpoint to retrieve the access token from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

payloadConfiguration

Defines which columns' data is sent to the endpoint and the size of the batches the data is sent in

columns

The list of columns that are sent to the endpoint

batchSize

The size of the badges that are sent to the endpoint. The default is 25.000 rows per badge.

triggerConfiguration

Defines which columns trigger sending the data to the endpoint for validation

columns

The list of columns that trigger sending the data to the endpoint

permissions

Defines whether the TDM should be available to all your sub-organizations or only for internal use

level

  • PUBLIC: The TDM can also be used by sub-organizations
  • PRIVATE: The TDM can only be used by users within your organization

Payload

{
  "name": "string",
  "columns": [
    {
      "key": "string",
      "label": "string",
      "columnType": "string",
      "optionConfiguration": {
        "options": [
          {
            "label": "string",
            "value": "string",
            "type": "STRING" or "NUMBER",
            "alternativeMatches": ["string", ...],
            "description": "string"
          },
        ],
        "mappingConfiguration": {
          "layers": ["EXACT", "SMART", "FUZZY"],
          "threshold": 0.6
        },
        "optionSource": "STATIC" | "DYNAMIC",
        "dynamicOptionFetch": {
          "url": "string",
          "method": "string",
          "headers": {},
          "authentication": {
            "method": "string",
            "refresh_url": "string",
            "headers": {}
          },
        "multiSelect": false,
	    },
      "mappingValidation": {
        "logic": "string",
        "errorMessage": "string"
      },
      "dataValidations": [
        {
          "logic": "string",
          "errorMessage": "string"
        }
      ]
    }
  ],
  "advancedValidations": [
    {
      "url": "string",
      "method": "string",
      "headers": {},
      "authentication":{
        "refresh_url": "string",
        "method": "string",
        "headers": {}
      },
      "payloadConfiguration":{
        "columns": [],
        "batchSize": 10000
      }
      "triggerConfiguration":{
        "columns": []
      }
    }
  ],
  "advancedCleanings": [
    {
      "url": "string",
      "method": "string",
      "headers": {},
      "authentication":{
        "refresh_url": "string",
        "method": "string",
        "headers": {}
      },
      "payloadConfiguration":{
        "columns": [],
        "batchSize": 10000
      }
      "triggerConfiguration":{
        "columns": []
      }
    }
  ],
  "permissions": {
    "level": "PUBLIC"
  }
}

Response​

Attributes

id

The ID of the TDM

name

The name of the TDM

columns

List of all columns in your TDM

key

The column key used to uniquely identify each column

label

The name of the column as displayed to users in the interface

columnType

The type of the column, defining which data type and format is expected for this column. Learn more about column types here

optionConfiguration

Configuration for category columns that defines available options and their mapping behavior

options

List of available options for the dropdown

label

The display name of the option shown to users

value

The actual value stored when this option is selected

type

The data type of the value

  • STRING: Text value
  • NUMBER: Numeric value

alternativeMatches

Alternative text values that should map to this option during data import

description

Optional description providing additional context about the option

mappingConfiguration

Configuration for how imported values are mapped to options

layers

Mapping strategies to use

  • EXACT: Exact match
  • SMART: AI-based matching
  • FUZZY: Similarity-based matching

threshold

Minimum similarity score (0.0-1.0)

optionSource

Source of the options

  • STATIC: Predefined options
  • DYNAMIC: Fetched from external source

dynamicOptionFetch

Configuration for fetching options dynamically from an external API

url

The endpoint URL to fetch options from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Authentication configuration for the dynamic option fetch endpoint

method

The authentication method to use

refresh_url

The endpoint to retrieve the access token from

headers

The list of key-value pairs that define the headers for the authentication request

multiSelect

Whether multiple options can be selected

  • true: Multiple selection allowed
  • false: Single selection only

mappingValidation

A validation rule to indicate under which conditions the column has to be mapped. Learn more about validations here

logic

Set a pre-defined validation rule. Refer to this page here for more info

errorMessage

Set a custom error message for one of the validation rules above

dataValidations

A list of value based validation rules applied to the column. Learn more about validations here

logic

Set a pre-defined validation rule. Refer to this page here for more info

errorMessage

Set a custom error message for one of the validation rules above

advancedValidations

url

The endpoint that recevies the data

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Defines your refresh token endpoint to obtain a new access token after the previous one has expired. This mechanism is more secure and allows you to re-authenticate every time you need to obtain a new access token

refresh_url

The endpoint to retrieve the access token from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

payloadConfiguration

Defines which columns' data is sent to the endpoint and the size of the batches the data is sent in

columns

The list of columns that are sent to the endpoint

batchSize

The size of the badges that are sent to the endpoint. The default is 10.000 rows and the maximum is 25.000 rows per badge.

triggerConfiguration

Defines which columns trigger sending the data to the endpoint for validation

columns

The list of columns that trigger sending the data to the endpoint

advancedCleanings

url

The endpoint that recevies the data

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Defines your refresh token endpoint to obtain a new access token after the previous one has expired. This mechanism is more secure and allows you to re-authenticate every time you need to obtain a new access token

refresh_url

The endpoint to retrieve the access token from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

payloadConfiguration

Defines which columns' data is sent to the endpoint and the size of the batches the data is sent in

columns

The list of columns that are sent to the endpoint

batchSize

The size of the badges that are sent to the endpoint. The default is 25.000 rows per badge.

triggerConfiguration

Defines which columns trigger sending the data to the endpoint for validation

columns

The list of columns that trigger sending the data to the endpoint

permissions

Defines whether the TDM should be available to all your sub-organizations or only for internal use

level

  • PUBLIC: The TDM can also be used by sub-organizations
  • PRIVATE: The TDM can only be used by users within your organization

created_at

The date and time when the TDM was first created

created_by

Information about whom created the TDM

id

The ID of the user or sub-organization who created the TDM

name

The name of the user or sub-organization who created the TDM

identifier

The identifier of the user or sub-organization who created the TDM

type

Defines the type of user who created the TDM:

  • USER: A user of your organization
  • SUB_ORG: A sub-organization that is part of your organization

updated_at

The date and time when the TDM was last updated

updated_by

Information about whom last updated the TDM

id

The ID of the user or sub-organization who last updated the TDM

name

The name of the user or sub-organization who last updated the TDM

identifier

The identifier of the user or sub-organization who last updated the TDM

type

Defines the type of user who last updated the TDM:

  • USER: A user of your organization
  • SUB_ORG: A sub-organization that is part of your organization

Response

{
  "data": {
    "id": "string",
    "name": "string",
    "columns": [
      {
        "key": "string",
        "label": "string",
        "columnType": "string",
        "optionConfiguration": {
          "options": [
            {
              "label": "string",
              "value": "string",
              "type": "STRING" or "NUMBER",
              "alternativeMatches": ["string", ...],
              "description": "string"
            },
          ],
          "mappingConfiguration": {
            "layers": ["EXACT", "SMART", "FUZZY"],
            "threshold": 0.6
          },
          "optionSource": "STATIC" | "DYNAMIC",
          "dynamicOptionFetch": {
            "url": "string",
            "method": "string",
            "headers": {},
            "authentication": {
              "method": "string",
              "refresh_url": "string",
              "headers": {}
            },
          "multiSelect": false,
        },
        "mappingValidation": {
          "logic": "string",
          "errorMessage": "string"
        },
        "dataValidations": [
          {
            "logic": "string",
            "errorMessage": "string"
          }
        ]
      }
    ],
    "advancedValidations": [
      {
        "url": "string",
        "method": "string",
        "headers": {},
        "authentication":{
          "refresh_url": "string",
          "method": "string",
          "headers": {}
        },
        "payloadConfiguration":{
          "columns": [],
          "batchSize": 10000
        }
        "triggerConfiguration":{
          "columns": []
        }
      }
    ],
    "advancedCleanings": [
      {
        "url": "string",
        "method": "string",
        "headers": {},
        "authentication":{
          "refresh_url": "string",
          "method": "string",
          "headers": {}
        },
        "payloadConfiguration":{
          "columns": [],
          "batchSize": 10000
        }
        "triggerConfiguration":{
          "columns": []
        }
      }
    ],
    "permissions": {
      "level": "PUBLIC"
    },
    "created_at": "2022-03-07 12:48:28.653",
    "created_by": {
      "id": "string",
      "name": "string",
      "identifier": "string",
      "type": "USER"
    },
    "updated_at": "2022-03-07 12:48:28.653",
    "updated_by": {
      "id": "string",
      "name": "string",
      "identifier": "string",
      "type": "USER"
    }
  }
}

Example​

cURL
Javascript
curl -X 'POST' 'https://api-gateway.ingestro.com/dp/api/v1/tdm' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
  "name": "New TDM name",
  "columns": [
    {
      "key": "company_name",
      "label": "Company name",
      "columnType": "string",
      "optionConfiguration": {
        "options": [
          {
            "label": "option",
            "value": "option",
            "type": "STRING",
            "alternativeMatches": ["Option"],
            "description": "option"
          },
        ],
        "mappingConfiguration": {
          "layers": ["EXACT", "SMART", "FUZZY"],
          "threshold": 0.6
        },
        "optionSource": "STATIC",
        "multiSelect": false
	    },
      "mappingValidation": {
        "logic": "mapped('company_name')",
        "errorMessage": "Column has to be mapped."
      },
      "dataValidations": [
        {
          "logic": "row.company_name != ''",
          "errorMessage": "Column is required."
        },
        {
          "logic": "regex(row.company_name, { expression: '^[a-zA-Z ]*$' })",
          "errorMessage": "Invalid format."
        },
        {
          "logic": "unique(['company_name'])",
          "errorMessage": "The company name must be unique across all entries."
        }
      ]
    }
  ],
  "advancedValidations": [
    {
      "url": "https://your-endpoint.com",
      "method": "POST",
      "headers": {},
      "authentication":{
        "refresh_url": "https://your-authentication-endpoint.com",
        "method": "POST",
        "headers": {}
      },
      "payloadConfiguration":{
        "columns": ['company_name'],
        "batchSize": 10000
      }
      "triggerConfiguration":{
        "columns": ['company_name']
      }
    }
  ],
  "advancedCleanings": [
    {
      "url": "https://your-endpoint.com",
      "method": "POST",
      "headers": {},
      "authentication":{
        "refresh_url": "https://your-authentication-endpoint.com",
        "method": "POST",
        "headers": {}
      },
      "payloadConfiguration":{
        "columns": ['company_name'],
        "batchSize": 10000
      }
      "triggerConfiguration":{
        "columns": ['company_name']
      }
    }
  ],
  "permissions": {
    "level": "PUBLIC"
  }
}'

Update​

Endpoint

PUT /tdm/{id}

Payload​

Attributes

name

The name of the TDM

columns

List of all columns in your TDM

key

The column key used to uniquely identify each column

label

The name of the column as displayed to users in the interface

columnType

The type of the column, defining which data type and format is expected for this column. Learn more about column types here

optionConfiguration

Configuration for category columns that defines available options and their mapping behavior

options

List of available options for the dropdown

label

The display name of the option shown to users

value

The actual value stored when this option is selected

type

The data type of the value

  • STRING: Text value
  • NUMBER: Numeric value

alternativeMatches

Alternative text values that should map to this option during data import

description

Optional description providing additional context about the option

mappingConfiguration

Configuration for how imported values are mapped to options

layers

Mapping strategies to use

  • EXACT: Exact match
  • SMART: AI-based matching
  • FUZZY: Similarity-based matching

threshold

Minimum similarity score (0.0-1.0)

optionSource

Source of the options

  • STATIC: Predefined options
  • DYNAMIC: Fetched from external source

dynamicOptionFetch

Configuration for fetching options dynamically from an external API

url

The endpoint URL to fetch options from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Authentication configuration for the dynamic option fetch endpoint

method

The authentication method to use

refresh_url

The endpoint to retrieve the access token from

headers

The list of key-value pairs that define the headers for the authentication request

multiSelect

Whether multiple options can be selected

  • true: Multiple selection allowed
  • false: Single selection only

mappingValidation

A validation rule to indicate under which conditions the column has to be mapped. Learn more about validations here

logic

Set a pre-defined validation rule. Refer to this page here for more info

errorMessage

Set a custom error message for one of the validation rules above

dataValidations

A list of value based validation rules applied to the column. Learn more about validations here

logic

Set a pre-defined validation rule. Refer to this page here for more info

errorMessage

Set a custom error message for one of the validation rules above

advancedValidations

url

The endpoint that recevies the data

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Defines your refresh token endpoint to obtain a new access token after the previous one has expired. This mechanism is more secure and allows you to re-authenticate every time you need to obtain a new access token

refresh_url

The endpoint to retrieve the access token from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

payloadConfiguration

Defines which columns' data is sent to the endpoint and the size of the batches the data is sent in

columns

The list of columns that are sent to the endpoint

batchSize

The size of the badges that are sent to the endpoint. The default is 10.000 rows and the maximum is 25.000 rows per badge.

triggerConfiguration

Defines which columns trigger sending the data to the endpoint for validation

columns

The list of columns that trigger sending the data to the endpoint

advancedCleanings

url

The endpoint that recevies the data

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Defines your refresh token endpoint to obtain a new access token after the previous one has expired. This mechanism is more secure and allows you to re-authenticate every time you need to obtain a new access token

refresh_url

The endpoint to retrieve the access token from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

payloadConfiguration

Defines which columns' data is sent to the endpoint and the size of the batches the data is sent in

columns

The list of columns that are sent to the endpoint

batchSize

The size of the badges that are sent to the endpoint. The default is 25.000 rows per badge.

triggerConfiguration

Defines which columns trigger sending the data to the endpoint for validation

columns

The list of columns that trigger sending the data to the endpoint

permissions

Defines whether the TDM should be available to all your sub-organizations or only for internal use

level

  • PUBLIC: The TDM can also be used by sub-organizations
  • PRIVATE: The TDM can only be used by users within your organization

Payload

{
  "name": "string",
  "columns": [
    {
      "key": "string",
      "label": "string",
      "columnType": "string",
      "optionConfiguration": {
        "options": [
          {
            "label": "string",
            "value": "string",
            "type": "STRING" or "NUMBER",
            "alternativeMatches": ["string", ...],
            "description": "string"
          },
        ],
        "mappingConfiguration": {
          "layers": ["EXACT", "SMART", "FUZZY"],
          "threshold": 0.6
        },
        "optionSource": "STATIC" | "DYNAMIC",
        "dynamicOptionFetch": {
          "url": "string",
          "method": "string",
          "headers": {},
          "authentication": {
            "method": "string",
            "refresh_url": "string",
            "headers": {}
          },
        "multiSelect": false,
      },
      "mappingValidation": {
        "logic": "string",
        "errorMessage": "string"
      },
      "dataValidations": [
        {
          "logic": "string",
          "errorMessage": "string"
        }
      ]
    }
  ],
  "advancedValidations": [
    {
      "url": "string",
      "method": "string",
      "headers": {},
      "authentication":{
        "refresh_url": "string",
        "method": "string",
        "headers": {}
      },
      "payloadConfiguration":{
        "columns": [],
        "batchSize": 10000
      }
      "triggerConfiguration":{
        "columns": []
      }
    }
  ],
  "advancedCleanings": [
    {
      "url": "string",
      "method": "string",
      "headers": {},
      "authentication":{
        "refresh_url": "string",
        "method": "string",
        "headers": {}
      },
      "payloadConfiguration":{
        "columns": [],
        "batchSize": 10000
      }
      "triggerConfiguration":{
        "columns": []
      }
    }
  ],
  "permissions": {
    "level": "PUBLIC"
  }
}

Response​

Attributes

id

The ID of the TDM

name

The name of the TDM

columns

List of all columns in your TDM

key

The column key used to uniquely identify each column

label

The name of the column as displayed to users in the interface

columnType

The type of the column, defining which data type and format is expected for this column. Learn more about column types here

optionConfiguration

Configuration for category columns that defines available options and their mapping behavior

options

List of available options for the dropdown

label

The display name of the option shown to users

value

The actual value stored when this option is selected

type

The data type of the value

  • STRING: Text value
  • NUMBER: Numeric value

alternativeMatches

Alternative text values that should map to this option during data import

description

Optional description providing additional context about the option

mappingConfiguration

Configuration for how imported values are mapped to options

layers

Mapping strategies to use

  • EXACT: Exact match
  • SMART: AI-based matching
  • FUZZY: Similarity-based matching

threshold

Minimum similarity score (0.0-1.0)

optionSource

Source of the options

  • STATIC: Predefined options
  • DYNAMIC: Fetched from external source

dynamicOptionFetch

Configuration for fetching options dynamically from an external API

url

The endpoint URL to fetch options from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Authentication configuration for the dynamic option fetch endpoint

method

The authentication method to use

refresh_url

The endpoint to retrieve the access token from

headers

The list of key-value pairs that define the headers for the authentication request

multiSelect

Whether multiple options can be selected

  • true: Multiple selection allowed
  • false: Single selection only

mappingValidation

A validation rule to indicate under which conditions the column has to be mapped. Learn more about validations here

logic

Set a pre-defined validation rule. Refer to this page here for more info

errorMessage

Set a custom error message for one of the validation rules above

dataValidations

A list of value based validation rules applied to the column. Learn more about validations here

logic

Set a pre-defined validation rule. Refer to this page here for more info

errorMessage

Set a custom error message for one of the validation rules above

advancedValidations

url

The endpoint that recevies the data

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Defines your refresh token endpoint to obtain a new access token after the previous one has expired. This mechanism is more secure and allows you to re-authenticate every time you need to obtain a new access token

refresh_url

The endpoint to retrieve the access token from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

payloadConfiguration

Defines which columns' data is sent to the endpoint and the size of the batches the data is sent in

columns

The list of columns that are sent to the endpoint

batchSize

The size of the badges that are sent to the endpoint. The default is 10.000 rows and the maximum is 25.000 rows per badge.

triggerConfiguration

Defines which columns trigger sending the data to the endpoint for validation

columns

The list of columns that trigger sending the data to the endpoint

advancedCleanings

url

The endpoint that recevies the data

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Defines your refresh token endpoint to obtain a new access token after the previous one has expired. This mechanism is more secure and allows you to re-authenticate every time you need to obtain a new access token

refresh_url

The endpoint to retrieve the access token from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

payloadConfiguration

Defines which columns' data is sent to the endpoint and the size of the batches the data is sent in

columns

The list of columns that are sent to the endpoint

batchSize

The size of the badges that are sent to the endpoint. The default is 25.000 rows per badge.

triggerConfiguration

Defines which columns trigger sending the data to the endpoint for validation

columns

The list of columns that trigger sending the data to the endpoint

permissions

Defines whether the TDM should be available to all your sub-organizations or only for internal use

level

  • PUBLIC: The TDM can also be used by sub-organizations
  • PRIVATE: The TDM can only be used by users within your organization

created_at

The date and time when the TDM was first created

created_by

Information about whom created the TDM

id

The ID of the user or sub-organization who created the TDM

name

The name of the user or sub-organization who created the TDM

identifier

The identifier of the user or sub-organization who created the TDM

type

Defines the type of user who created the TDM:

  • USER: A user of your organization
  • SUB_ORG: A sub-organization that is part of your organization

updated_at

The date and time when the TDM was last updated

updated_by

Information about whom last updated the TDM

id

The ID of the user or sub-organization who last updated the TDM

name

The name of the user or sub-organization who last updated the TDM

identifier

The identifier of the user or sub-organization who last updated the TDM

type

Defines the type of user who last updated the TDM:

  • USER: A user of your organization
  • SUB_ORG: A sub-organization that is part of your organization

Response

{
  "data": {
    "id": "string",
    "name": "string",
    "columns": [
      {
        "key": "string",
        "label": "string",
        "columnType": "string",
        "optionConfiguration": {
          "options": [
            {
              "label": "string",
              "value": "string",
              "type": "STRING" or "NUMBER",
              "alternativeMatches": ["string", ...],
              "description": "string"
            },
          ],
          "mappingConfiguration": {
            "layers": ["EXACT", "SMART", "FUZZY"],
            "threshold": 0.6
          },
          "optionSource": "STATIC" | "DYNAMIC",
          "dynamicOptionFetch": {
            "url": "string",
            "method": "string",
            "headers": {},
            "authentication": {
              "method": "string",
              "refresh_url": "string",
              "headers": {}
            },
          "multiSelect": false,
        },
        "mappingValidation": {
          "logic": "string",
          "errorMessage": "string"
        },
        "dataValidations": [
          {
            "logic": "string",
            "errorMessage": "string"
          }
        ]
      }
    ],
    "advancedValidations": [
      {
        "url": "string",
        "method": "string",
        "headers": {},
        "authentication":{
          "refresh_url": "string",
          "method": "string",
          "headers": {}
        },
        "payloadConfiguration":{
          "columns": [],
          "batchSize": 10000
        }
        "triggerConfiguration":{
          "columns": []
        }
      }
    ],
    "advancedCleanings": [
      {
        "url": "string",
        "method": "string",
        "headers": {},
        "authentication":{
          "refresh_url": "string",
          "method": "string",
          "headers": {}
        },
        "payloadConfiguration":{
          "columns": [],
          "batchSize": 10000
        }
        "triggerConfiguration":{
          "columns": []
        }
      }
    ],
    "permissions": {
      "level": "PUBLIC"
    },
    "created_at": "2022-03-07 12:48:28.653",
    "created_by": {
      "id": "string",
      "name": "string",
      "identifier": "string",
      "type": "USER"
    },
    "updated_at": "2022-03-07 12:48:28.653",
    "updated_by": {
      "id": "string",
      "name": "string",
      "identifier": "string",
      "type": "USER"
    }
  }
}

Example​

cURL
Javascript
curl -X 'PUT' 'https://api-gateway.ingestro.com/dp/api/v1/tdm/${tdm_id}' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN' \
  -d '{
  "name": "NEW TDM name",
  "columns": [
    {
      "key": "company_name",
      "label": "Company name",
      "columnType": "string",
      "optionConfiguration": {
        "options": [
          {
            "label": "option",
            "value": "option",
            "type": "STRING",
            "alternativeMatches": ["Option"],
            "description": "option"
          }, ...
        ],
        "mappingConfiguration": {
          "layers": ["EXACT", "SMART", "FUZZY"],
          "threshold": 0.6
        },
        "optionSource": "STATIC",
        "multiSelect": false
	    },
      "mappingValidation": {
        "logic": "mapped('company_name')",
        "errorMessage": "Column has to be mapped."
      },
      "dataValidations": [
        {
          "logic": "row.company_name != ''",
          "errorMessage": "Column is required."
        },
        {
          "logic": "regex(row.company_name, { expression: '^[a-zA-Z ]*$' })",
          "errorMessage": "Invalid format."
        },
        {
          "logic": "unique(['company_name'])",
          "errorMessage": "The company name must be unique across all entries."
        }
      ]
    }
  ],
  "advancedValidations": [
    {
      "url": "https://your-endpoint.com",
      "method": "POST",
      "headers": {},
      "authentication":{
        "refresh_url": "https://your-authentication-endpoint.com",
        "method": "POST",
        "headers": {}
      },
      "payloadConfiguration":{
        "columns": ['company_name'],
        "batchSize": 10000
      }
      "triggerConfiguration":{
        "columns": ['company_name']
      }
    }
  ],
  "advancedCleanings": [
    {
      "url": "https://your-endpoint.com",
      "method": "POST",
      "headers": {},
      "authentication":{
        "refresh_url": "https://your-authentication-endpoint.com",
        "method": "POST",
        "headers": {}
      },
      "payloadConfiguration":{
        "columns": ['company_name'],
        "batchSize": 10000
      }
      "triggerConfiguration":{
        "columns": ['company_name']
      }
    }
  ],
  "permissions": {
    "level": "PRIVATE"
  }
}'

Read by ID​

Endpoint

GET /tdm/{id}

Response​

Attributes

id

The ID of the TDM

name

The name of the TDM

columns

List of all columns in your TDM

key

The column key used to uniquely identify each column

label

The name of the column as displayed to users in the interface

columnType

The type of the column, defining which data type and format is expected for this column. Learn more about column types here

optionConfiguration

Configuration for category columns that defines available options and their mapping behavior

options

List of available options for the dropdown

label

The display name of the option shown to users

value

The actual value stored when this option is selected

type

The data type of the value

  • STRING: Text value
  • NUMBER: Numeric value

alternativeMatches

Alternative text values that should map to this option during data import

description

Optional description providing additional context about the option

mappingConfiguration

Configuration for how imported values are mapped to options

layers

Mapping strategies to use

  • EXACT: Exact match
  • SMART: AI-based matching
  • FUZZY: Similarity-based matching

threshold

Minimum similarity score (0.0-1.0)

optionSource

Source of the options

  • STATIC: Predefined options
  • DYNAMIC: Fetched from external source

dynamicOptionFetch

Configuration for fetching options dynamically from an external API

url

The endpoint URL to fetch options from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Authentication configuration for the dynamic option fetch endpoint

method

The authentication method to use

refresh_url

The endpoint to retrieve the access token from

headers

The list of key-value pairs that define the headers for the authentication request

multiSelect

Whether multiple options can be selected

  • true: Multiple selection allowed
  • false: Single selection only

mappingValidation

A validation rule to indicate under which conditions the column has to be mapped. Learn more about validations here

logic

Set a pre-defined validation rule. Refer to this page here for more info

errorMessage

Set a custom error message for one of the validation rules above

dataValidations

A list of value based validation rules applied to the column. Learn more about validations here

logic

Set a pre-defined validation rule. Refer to this page here for more info

errorMessage

Set a custom error message for one of the validation rules above

advancedValidations

url

The endpoint that recevies the data

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Defines your refresh token endpoint to obtain a new access token after the previous one has expired. This mechanism is more secure and allows you to re-authenticate every time you need to obtain a new access token

refresh_url

The endpoint to retrieve the access token from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

payloadConfiguration

Defines which columns' data is sent to the endpoint and the size of the batches the data is sent in

columns

The list of columns that are sent to the endpoint

batchSize

The size of the badges that are sent to the endpoint. The default is 10.000 rows and the maximum is 25.000 rows per badge.

triggerConfiguration

Defines which columns trigger sending the data to the endpoint for validation

columns

The list of columns that trigger sending the data to the endpoint

advancedCleanings

url

The endpoint that recevies the data

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

authentication

Defines your refresh token endpoint to obtain a new access token after the previous one has expired. This mechanism is more secure and allows you to re-authenticate every time you need to obtain a new access token

refresh_url

The endpoint to retrieve the access token from

method

The REST API method to use for the request

headers

The list of key-value pairs that define the headers for the request

payloadConfiguration

Defines which columns' data is sent to the endpoint and the size of the batches the data is sent in

columns

The list of columns that are sent to the endpoint

batchSize

The size of the badges that are sent to the endpoint. The default is 25.000 rows per badge.

triggerConfiguration

Defines which columns trigger sending the data to the endpoint for validation

columns

The list of columns that trigger sending the data to the endpoint

permissions

Defines whether the TDM should be available to all your sub-organizations or only for internal use

level

  • PUBLIC: The TDM can also be used by sub-organizations
  • PRIVATE: The TDM can only be used by users within your organization

created_at

The date and time when the TDM was first created

created_by

Information about whom created the TDM

id

The ID of the user or sub-organization who created the TDM

name

The name of the user or sub-organization who created the TDM

identifier

The identifier of the user or sub-organization who created the TDM

type

Defines the type of user who created the TDM:

  • USER: A user of your organization
  • SUB_ORG: A sub-organization that is part of your organization

updated_at

The date and time when the TDM was last updated

updated_by

Information about whom last updated the TDM

id

The ID of the user or sub-organization who last updated the TDM

name

The name of the user or sub-organization who last updated the TDM

identifier

The identifier of the user or sub-organization who last updated the TDM

type

Defines the type of user who last updated the TDM:

  • USER: A user of your organization
  • SUB_ORG: A sub-organization that is part of your organization

Response

{
  "data": {
    "id": "string",
    "name": "string",
    "columns": [
      {
        "key": "string",
        "label": "string",
        "columnType": "string",
        "optionConfiguration": {
          "options": [
            {
              "label": "string",
              "value": "string",
              "type": "STRING" or "NUMBER",
              "alternativeMatches": ["string", ...],
              "description": "string"
            },
          ],
          "mappingConfiguration": {
            "layers": ["EXACT", "SMART", "FUZZY"],
            "threshold": 0.6
          },
          "optionSource": "STATIC" | "DYNAMIC",
          "dynamicOptionFetch": {
            "url": "string",
            "method": "string",
            "headers": {},
            "authentication": {
              "method": "string",
              "refresh_url": "string",
              "headers": {}
            },
          "multiSelect": false,
        },
        "mappingValidation": {
          "logic": "string",
          "errorMessage": "string"
        },
        "dataValidations": [
          {
            "logic": "string",
            "errorMessage": "string"
          }
        ]
      }
    ],
    "advancedValidations": [
      {
        "url": "string",
        "method": "string",
        "headers": {},
        "authentication":{
          "refresh_url": "string",
          "method": "string",
          "headers": {}
        },
        "payloadConfiguration":{
          "columns": [],
          "batchSize": 10000
        }
        "triggerConfiguration":{
          "columns": []
        }
      }
    ],
    "advancedCleanings": [
      {
        "url": "string",
        "method": "string",
        "headers": {},
        "authentication":{
          "refresh_url": "string",
          "method": "string",
          "headers": {}
        },
        "payloadConfiguration":{
          "columns": [],
          "batchSize": 10000
        }
        "triggerConfiguration":{
          "columns": []
        }
      }
    ],
    "permissions": {
      "level": "PUBLIC"
    },
    "created_at": "2022-03-07 12:48:28.653",
    "created_by": {
      "id": "string",
      "name": "string",
      "identifier": "string",
      "type": "USER"
    },
    "updated_at": "2022-03-07 12:48:28.653",
    "updated_by": {
        "id": "string",
        "name": "string",
        "identifier": "string",
        "type": "USER"
      }
  }
}

Example​

cURL
Javascript
curl -X 'GET' 'https://api-gateway.ingestro.com/dp/api/v1/tdm/${tdmId}' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN'

Read all​

To further refine the response you can use query parameters like sort, filters, pagination and options. Look at a more detailed explanation here.

Endpoint

GET /tdm/

Response​

Attributes

id

The ID of the TDM

name

The name of the TDM

permissions

Defines whether the TDM should be available to all your sub-organizations or only for internal use

level

  • PUBLIC: The TDM can also be used by sub-organizations
  • PRIVATE: The TDM can only be used by users within your organization

created_at

The date and time when the TDM was first created

created_by

Information about whom created the TDM

id

The ID of the user or sub-organization who created the TDM

name

The name of the user or sub-organization who created the TDM

identifier

The identifier of the user or sub-organization who created the TDM

type

Defines the type of user who created the TDM:

  • USER: A user of your organization
  • SUB_ORG: A sub-organization that is part of your organization

updated_at

The date and time when the TDM was last updated

updated_by

Information about whom last updated the TDM

id

The ID of the user or sub-organization who last updated the TDM

name

The name of the user or sub-organization who last updated the TDM

identifier

The identifier of the user or sub-organization who last updated the TDM

type

Defines the type of user who last updated the TDM:

  • USER: A user of your organization
  • SUB_ORG: A sub-organization that is part of your organization

pagination

An object containing metadata about the result

total

The number of entries in the data array

offset

The offset set in the request parameters

limit

The limit set in the request parameters

Response

{
  "data": [
    {
      "id": "string",
      "name": "string",
      "permissions": {
        "level": "PUBLIC"
      },
      "created_at": "2022-03-07 12:48:28.653",
      "created_by": {
        "id": "string",
        "name": "string",
        "identifier": "string",
        "type": "USER"
      },
      "updated_at": "2022-03-07 12:48:28.653",
      "updated_by": {
        "id": "string",
        "name": "string",
        "identifier": "string",
        "type": "USER"
      }
    }
  ],
  "pagination": {
    "total": 0,
    "offset": 0,
    "limit": 0
  }
}

Example​

cURL
Javascript
curl -X 'GET' 'https://api-gateway.ingestro.com/dp/api/v1/tdm' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN'

Delete​

Endpoint

DELETE /tdm/{id}

Response​

Attributes

message

Message confirming the deletion of the TDM or providing an error message

Response

{
  "data": {
    "message": "string"
  }
}

Example​

cURL
Javascript
curl -X 'DELETE' 'https://api-gateway.ingestro.com/dp/api/v1/tdm/${tdmId}' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN'