Pipeline Templates API

A pipeline template is a blueprint for building a pipeline. You can predefine all components, such as input and output connectors, the pipeline name, and the target data model, or only some. Templates provide full flexibility.

Components of a pipeline template:

Template name
Pipeline name
Input connector
Output connector
Target data model
Developer mode
Schedule
Error threshold

Use this base URL and add the corresponding endpoint respectively:

Base URL

api-gateway.ingestro.com/dp/api/v1/

Create

Endpoint

POST /pipeline-template/

Attributes

name

The name of the pipeline template

pipeline_name

The name of the pipeline created using the template

input_connectors

The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.

output_connectors

The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.

tdm

The ID of the set target data model

error_config

Defines how the pipeline created using the template should handle errors that might occur during pipeline execution

error_threshold

A float between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail.

schedule_config

Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed

frequency

Sets how often the pipeline is executed. It is intertwined with interval. For example, if frequency is set to HOURLY and interval is set to 2, the pipeline is executed every 2 hours:

HOURLY
DAILY
WEEKLY
MONTHLY

interval

Sets the interval based on the frequency at which the pipeline is executed. For example, if interval is set to 2 and frequency is set to HOURLY, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time.

starts_on

The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). The date and time cannot be in the past.

ends_on

The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time.

header_config

Defines how the header row is determined

type

Specifies whether Ingestro's header detection is applied or if the set row_index is used to determine the header row:

SMART: Ingestro's header detection is used to define the header row
STATIC: The row at the specified row_index is used as the header row

row_index

The index of the row that should be used as the header row if type is set to STATIC

sheet_config

Defines which sheet to process when working with multi-sheet file types (XLS, XLSX, XML). This configuration is only applicable for multi-sheet files and is ignored for single-sheet file types (CSV, TSV, JSON). Pipelines created from this template will inherit this sheet selection configuration.

When to use:

Your input connector uses XLS, XLSX, or XML files with multiple sheets
You want to specify which sheet contains the data to process
You need consistent sheet selection across multiple pipeline executions

Behavior:

For single-sheet file types (CSV, TSV, JSON): sheet_config is ignored
For multi-sheet files with only one sheet: sheet_config is optional (defaults to first sheet)
For multi-sheet files with multiple sheets: sheet_config determines which sheet to use

selection_type

Defines the strategy for selecting which sheet to process:

INDEX: Select sheet by its position (0-indexed). Use this when you always want to process the sheet at a specific position (e.g., always the 2nd sheet)
NAME: Select sheet by its name. Use this when you always want to process a sheet with a specific name (e.g., always the "CRM" sheet)

sheet_index

The 0-based index of the sheet to process. Required when selection_type is INDEX.

Examples:

0 = First sheet
1 = Second sheet
2 = Third sheet

Important: Backend uses 0-indexing. The first sheet is at index 0, not 1.

sheet_name

The name of the sheet to process. Required when selection_type is NAME.

Examples:

"Sheet1" = Process the sheet named "Sheet1"
"CRM" = Process the sheet named "CRM"
"Data" = Process the sheet named "Data"

Important: Sheet names are case-sensitive. The execution will fail if a sheet with the specified name is not found in the file.

developer_mode

Defines if the pipeline created using the template is executed in developer mode (true) or not (false). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows.

Payload

{
  "name": "string",
  "pipeline_name": "string",
  "input_connectors": [
    "string"
  ],
  "output_connectors": [
    "string"
  ],
  "tdm": "string",
  "error_config": {
    "error_threshold": 0
  },
  "schedule_config": {
    "frequency": "HOURLY",
    "interval": 0,
    "starts_on": "2024-09-02T15:38:58.450Z",
    "ends_on": "2024-09-02T15:38:58.450Z"
  },
  "header_config": {
    "type": "SMART",
    "row_index": 0
  },
  "sheet_config": {
    "selection_type": "INDEX",
    "sheet_index": 0,
    "sheet_name": null
  }
  "developer_mode": false
}

Response

Attributes

The ID of the pipeline template

name

The name of the pipeline template

pipeline_name

The name of the pipeline created using the template

input_connectors

The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.

output_connectors

The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.

tdm

The ID of the set target data model

error_config

Defines how the pipeline created using the template should handle errors that might occur during pipeline execution

error_threshold

schedule_config

Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed

frequency

Sets how often the pipeline is executed. It is intertwined with interval. For example, if frequency is set to HOURLY and interval is set to 2, the pipeline is executed every 2 hours:

HOURLY
DAILY
WEEKLY
MONTHLY

interval

starts_on

The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). The date and time cannot be in the past

ends_on

The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time

header_config

Defines how the header row is determined

type

Specifies whether Ingestro's header detection is applied or if the set row_index is used to determine the header row:

SMART: Ingestro's header detection is used to define the header row
STATIC: The row at the specified row_index is used as the header row

row_index

The index of the row that should be used as the header row if type is set to STATIC

sheet_config

When to use:

Your input connector uses XLS, XLSX, or XML files with multiple sheets
You want to specify which sheet contains the data to process
You need consistent sheet selection across multiple pipeline executions

Behavior:

For single-sheet file types (CSV, TSV, JSON): sheet_config is ignored
For multi-sheet files with only one sheet: sheet_config is optional (defaults to first sheet)
For multi-sheet files with multiple sheets: sheet_config determines which sheet to use

selection_type

Defines the strategy for selecting which sheet to process:

INDEX: Select sheet by its position (0-indexed). Use this when you always want to process the sheet at a specific position (e.g., always the 2nd sheet)
NAME: Select sheet by its name. Use this when you always want to process a sheet with a specific name (e.g., always the "CRM" sheet)

sheet_index

The 0-based index of the sheet to process. Required when selection_type is INDEX.

Examples:

0 = First sheet
1 = Second sheet
2 = Third sheet

Important: Backend uses 0-indexing. The first sheet is at index 0, not 1.

sheet_name

The name of the sheet to process. Required when selection_type is NAME.

Examples:

"Sheet1" = Process the sheet named "Sheet1"
"CRM" = Process the sheet named "CRM"
"Data" = Process the sheet named "Data"

Important: Sheet names are case-sensitive. The execution will fail if a sheet with the specified name is not found in the file.

developer_mode

created_at

The date and time when the pipeline template was first created

created_by

Information about whom created the pipeline template

The ID of the user or sub-organization who created the pipeline template

name

The name of the user or sub-organization who created the pipeline template

identifier

The identifier of the user or sub-organization who created the pipeline template

type

Defines the type of user who created the pipeline template:

USER: A user of your organization
SUB_ORG: A sub-organization that is part of your organization

updated_at

The date and time when the pipeline templatewas last updated

updated_by

Information about whom last updated the pipeline template

The ID of the user or sub-organization who last updated the pipeline template

name

The name of the user or sub-organization who last updated the pipeline template

identifier

The identifier of the user or sub-organization who last updated the pipeline template

type

Defines the type of user who last updated the pipeline template:

USER: A user of your organization
SUB_ORG: A sub-organization that is part of your organization

Response

{
  "data": {
    "id": "string",
    "name": "string",
    "pipeline_name": "string",
    "input_connectors": [
      "string"
    ],
    "output_connectors": [
      "string"
    ],
    "tdm": "string",
    "error_config": {
      "error_threshold": 0
    },
    "schedule_config": {
      "frequency": "HOURLY",
      "interval": 0,
      "starts_on": "2024-09-02T15:44:41.375Z",
      "ends_on": "2024-09-02T15:44:41.375Z"
    },
    "header_config": {
      "type": "SMART",
      "row_index": 0
    },
    "sheet_config": {
      "selection_type": "INDEX",
      "sheet_index": 0,
      "sheet_name": null
    },
    "developer_mode": false,
    "created_at": "2022-03-07 12:48:28.653",
    "created_by": {
      "id": "string",
      "name": "string",
      "identifier": "string",
      "type": "USER"
    },
    "updated_at": "2022-03-07 12:48:28.653",
    "update_by": {
      "id": "string",
      "name": "string",
      "identifier": "string",
      "type": "USER"
    }
  }
}

Example

cURL

Javascript

curl -X 'POST' 'https://api-gateway.ingestro.com/dp/api/v1/pipeline-template' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
  "name": "NEW template name",
  "pipeline_name": "New pipeline name",
  "input_connectors": [
    "INPUT_CONNECTOR_ID"
  ],
  "output_connectors": [
    "OUTPUT_CONNECTOR_ID"
  ],
  "tdm": "TARGET_DATA_MODEL_ID",
  "error_config": {
    "error_threshold": 10
  },
  "schedule_config": {
    "frequency": "WEEKLY",
    "interval": 2,
    "starts_on": "2025-03-17T18:47:33.864Z"
  },
  "developer_mode": true
}
'

Update

Endpoint

PUT / pipeline - template / { id };

Attributes

name

The name of the pipeline template

pipeline_name

The name of the pipeline created using the template

input_connectors

The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.

output_connectors

The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.

tdm

The ID of the set target data model

error_config

Defines how the pipeline created using the template should handle errors that might occur during pipeline execution

error_threshold

schedule_config

Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed

frequency

Sets how often the pipeline is executed. It is intertwined with interval. For example, if frequency is set to HOURLY and interval is set to 2, the pipeline is executed every 2 hours:

HOURLY
DAILY
WEEKLY
MONTHLY

interval

starts_on

The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). The date and time cannot be in the past.

ends_on

The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time.

header_config

Defines how the header row is determined

type

Specifies whether Ingestro's header detection is applied or if the set row_index is used to determine the header row:

SMART: Ingestro's header detection is used to define the header row
STATIC: The row at the specified row_index is used as the header row

row_index

The index of the row that should be used as the header row if type is set to STATIC

sheet_config

When to use:

Your input connector uses XLS, XLSX, or XML files with multiple sheets
You want to specify which sheet contains the data to process
You need consistent sheet selection across multiple pipeline executions

Behavior:

For single-sheet file types (CSV, TSV, JSON): sheet_config is ignored
For multi-sheet files with only one sheet: sheet_config is optional (defaults to first sheet)
For multi-sheet files with multiple sheets: sheet_config determines which sheet to use

selection_type

Defines the strategy for selecting which sheet to process:

INDEX: Select sheet by its position (0-indexed). Use this when you always want to process the sheet at a specific position (e.g., always the 2nd sheet)
NAME: Select sheet by its name. Use this when you always want to process a sheet with a specific name (e.g., always the "CRM" sheet)

sheet_index

The 0-based index of the sheet to process. Required when selection_type is INDEX.

Examples:

0 = First sheet
1 = Second sheet
2 = Third sheet

Important: Backend uses 0-indexing. The first sheet is at index 0, not 1.

sheet_name

The name of the sheet to process. Required when selection_type is NAME.

Examples:

"Sheet1" = Process the sheet named "Sheet1"
"CRM" = Process the sheet named "CRM"
"Data" = Process the sheet named "Data"

Important: Sheet names are case-sensitive. The execution will fail if a sheet with the specified name is not found in the file.

developer_mode

Payload

{
  "name": "string",
  "pipeline_name": "string",
  "input_connectors": [
    "string"
  ],
  "output_connectors": [
    "string"
  ],
  "tdm": "string",
  "error_config": {
    "error_threshold": 0
  },
  "schedule_config": {
    "frequency": "HOURLY",
    "interval": 0,
    "starts_on": "2024-09-02T15:38:58.450Z",
    "ends_on": "2024-09-02T15:38:58.450Z"
  },
  "header_config": {
    "type": "SMART",
    "row_index": 0
  },
  "sheet_config": {
    "selection_type": "INDEX",
    "sheet_index": 0,
    "sheet_name": null
  },
  "developer_mode": false
}

Response

Attributes

The ID of the pipeline template

name

The name of the pipeline template

pipeline_name

The name of the pipeline created using the template

input_connectors

The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.

output_connectors

The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.

tdm

The ID of the set target data model

error_config

Defines how the pipeline created using the template should handle errors that might occur during pipeline execution

error_threshold

schedule_config

Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed

frequency

Sets how often the pipeline is executed. It is intertwined with interval. For example, if frequency is set to HOURLY and interval is set to 2, the pipeline is executed every 2 hours:

HOURLY
DAILY
WEEKLY
MONTHLY

interval

starts_on

The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). The date and time cannot be in the past.

ends_on

The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time.

header_config

Defines how the header row is determined

type

Specifies whether Ingestro's header detection is applied or if the set row_index is used to determine the header row:

SMART: Ingestro's header detection is used to define the header row
STATIC: The row at the specified row_index is used as the header row

row_index

The index of the row that should be used as the header row if type is set to STATIC

sheet_config

When to use:

Your input connector uses XLS, XLSX, or XML files with multiple sheets
You want to specify which sheet contains the data to process
You need consistent sheet selection across multiple pipeline executions

Behavior:

For single-sheet file types (CSV, TSV, JSON): sheet_config is ignored
For multi-sheet files with only one sheet: sheet_config is optional (defaults to first sheet)
For multi-sheet files with multiple sheets: sheet_config determines which sheet to use

selection_type

Defines the strategy for selecting which sheet to process:

INDEX: Select sheet by its position (0-indexed). Use this when you always want to process the sheet at a specific position (e.g., always the 2nd sheet)
NAME: Select sheet by its name. Use this when you always want to process a sheet with a specific name (e.g., always the "CRM" sheet)

sheet_index

The 0-based index of the sheet to process. Required when selection_type is INDEX.

Examples:

0 = First sheet
1 = Second sheet
2 = Third sheet

Important: Backend uses 0-indexing. The first sheet is at index 0, not 1.

sheet_name

The name of the sheet to process. Required when selection_type is NAME.

Examples:

"Sheet1" = Process the sheet named "Sheet1"
"CRM" = Process the sheet named "CRM"
"Data" = Process the sheet named "Data"

Important: Sheet names are case-sensitive. The execution will fail if a sheet with the specified name is not found in the file.

developer_mode

created_at

The date and time when the pipeline template was first created

created_by

Information about whom created the pipeline template

The ID of the user or sub-organization who created the pipeline template

name

The name of the user or sub-organization who created the pipeline template

identifier

The identifier of the user or sub-organization who created the pipeline template

type

Defines the type of user who created the pipeline template:

USER: A user of your organization
SUB_ORG: A sub-organization that is part of your organization

updated_at

The date and time when the pipeline templatewas last updated

updated_by

Information about whom last updated the pipeline template

The ID of the user or sub-organization who last updated the pipeline template

name

The name of the user or sub-organization who last updated the pipeline template

identifier

The identifier of the user or sub-organization who last updated the pipeline template

type

Defines the type of user who last updated the pipeline template:

USER: A user of your organization
SUB_ORG: A sub-organization that is part of your organization

Response

{
  "data": {
    "id": "string",
    "name": "string",
    "pipeline_name": "string",
    "input_connectors": [
      "string"
    ],
    "output_connectors": [
      "string"
    ],
    "tdm": "string",
    "error_config": {
      "error_threshold": 0
    },
    "schedule_config": {
      "frequency": "HOURLY",
      "interval": 0,
      "starts_on": "2024-09-02T15:44:41.375Z",
      "ends_on": "2024-09-02T15:44:41.375Z"
    },
    "header_config": {
      "type": "SMART",
      "row_index": 0
    },
    "sheet_config": {
      "selection_type": "INDEX",
      "sheet_index": 0,
      "sheet_name": null
    },
    "developer_mode": false,
    "created_at": "2022-03-07 12:48:28.653",
    "created_by": {
      "id": "string",
      "name": "string",
      "identifier": "string",
      "type": "USER",
      "organization": "string"
    },
    "updated_at": "2022-03-07 12:48:28.653",
    "update_by": {
      "id": "string",
      "name": "string",
      "identifier": "string",
      "type": "USER",
      "organization": "string"
    }
  }
}

Example

cURL

Javascript

curl -X 'PUT' 'https://api-gateway.ingestro.com/dp/api/v1/pipeline-template/${templateId}' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN' \
  -d '{
  "name": "NEW template name",
  "pipeline_name": "new pipeline name",
  "input_connectors": [
    "INPUT_CONNECTOR_ID"
  ],
  "output_connectors": [
    "OUTPUT_CONNECTOR_ID"
  ],
  "tdm": "TARGET_DATA_MODEL_ID",
  "error_config": {
    "error_threshold": 10
  },
  "schedule_config": {
    "frequency": "WEEKLY",
    "interval": 2,
    "starts_on": "2025-03-17T18:42:34.161Z"
  },
  "developer_mode": true
}'

Example

cURL

Javascript

curl -X 'PUT' 'https://api-gateway.ingestro.com/dp/api/v1/pipeline-template/${templateId}' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN' \
  -d '{
  "name": "NEW template name",
  "pipeline_name": "new pipeline name",
  "input_connectors": [
    "INPUT_CONNECTOR_ID"
  ],
  "output_connectors": [
    "OUTPUT_CONNECTOR_ID"
  ],
  "tdm": "TARGET_DATA_MODEL_ID",
  "error_config": {
    "error_threshold": 10
  },
  "schedule_config": {
    "frequency": "WEEKLY",
    "interval": 2,
    "starts_on": "2025-03-17T18:42:34.161Z"
  },
  "developer_mode": true
}'

Sheet Selection Examples

Multi-Sheet File Support

When working with Excel (XLS, XLSX) or XML files that contain multiple sheets, use the sheet_config property to specify which sheet to process. You can select sheets either by their position (INDEX) or by their name (NAME). For single-sheet file types like CSV, TSV, or JSON, the sheet_config property is not needed and will be ignored if provided.

Example 1: Select Sheet by Position (INDEX)

This example shows how to create a pipeline template that always processes the second sheet (index 1) in a multi-sheet Excel file.

cURL

Javascript

curl -X 'PUT' 'https://api-gateway.ingestro.com/dp/api/v1/pipeline-template/{id}' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "CRM Template",
    "pipeline_name": "CRM Data Import",
    "input_connectors": ["connector_id"],
    "output_connectors": ["connector_id"],
    "tdm": "tdm_id",
    "header_config": {
      "type": "STATIC",
      "row_index": 0
    },
    "sheet_config": {
      "selection_type": "INDEX",
      "sheet_index": 1
    }
  }'

Use case: When your data is always in the second sheet, regardless of what the sheet is named. This is useful when sheet names might change but the position remains constant. All pipelines created from this template will inherit this sheet selection.

Example 2: Select Sheet by Name (NAME)

This example shows how to create a pipeline template that always processes the sheet named "CRM" in a multi-sheet Excel file.

cURL

Javascript

curl -X 'PUT' 'https://api-gateway.ingestro.com/dp/api/v1/pipeline-template/{id}' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "CRM Template",
    "pipeline_name": "CRM Data Import",
    "input_connectors": ["connector_id"],
    "output_connectors": ["connector_id"],
    "tdm": "tdm_id",
    "header_config": {
      "type": "STATIC",
      "row_index": 0
    },
    "sheet_config": {
      "selection_type": "NAME",
      "sheet_name": "CRM"
    }
  }'

Use case: When your data is always in a sheet with a specific name (e.g., "CRM", "Data", "Sales"). This is useful when the sheet name is consistent but its position might change. All pipelines created from this template will inherit this sheet selection.

Example 3: Pipeline Template Without Sheet Selection (Single-Sheet File)

For CSV, TSV, or JSON files, you don't need to specify sheet_config:

cURL

Javascript

curl -X 'PUT' 'https://api-gateway.ingestro.com/dp/api/v1/pipeline-template/{id}' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "CSV Template",
    "pipeline_name": "CSV Data Import",
    "input_connectors": ["connector_id"],
    "output_connectors": ["connector_id"],
    "tdm": "tdm_id",
    "header_config": {
      "type": "STATIC",
      "row_index": 0
    }
  }'

info

Important Notes About Sheet Selection:

File Type Support:
- Multi-sheet file types: XLS, XLSX, XML
- Single-sheet file types: CSV, TSV, JSON
Selection Type Guidelines:
- Use INDEX when sheet position is consistent but names may vary
- Use NAME when sheet names are consistent but positions may vary
Indexing:
- Sheet indices are 0-based (first sheet = 0, second sheet = 1, etc.)
- When displaying to users, convert to 1-based (1st, 2nd, 3rd, etc.)
Error Handling:
- If selection_type is INDEX and the specified index doesn't exist, the execution will fail
- If selection_type is NAME and the specified sheet name doesn't exist, the execution will fail
- Sheet names are case-sensitive
Default Behavior:
- If sheet_config is not provided for multi-sheet files, the first sheet (index 0) is used
- For single-sheet files, sheet_config is ignored even if provided
Template Inheritance:
- Pipelines created from a template inherit the sheet_config from the template
- Users can override the inherited sheet_config during pipeline creation if needed

Read by ID

Endpoint

GET /pipeline-template/{id}

Response

Attributes

The ID of the pipeline template

name

The name of the pipeline template

pipeline_name

The name of the pipeline created using the template

input_connectors

The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.

output_connectors

The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.

tdm

The ID of the set target data model

error_config

Defines how the pipeline created using the template should handle errors that might occur during pipeline execution

error_threshold

schedule_config

Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed

frequency

Sets how often the pipeline is executed. It is intertwined with interval. For example, if frequency is set to HOURLY and interval is set to 2, the pipeline is executed every 2 hours:

HOURLY
DAILY
WEEKLY
MONTHLY

interval

starts_on

The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). The date and time cannot be in the past

ends_on

The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time

header_config

Defines how the header row is determined

type

Specifies whether Ingestro's header detection is applied or if the set row_index is used to determine the header row:

SMART: Ingestro's header detection is used to define the header row
STATIC: The row at the specified row_index is used as the header row

row_index

The index of the row that should be used as the header row if type is set to STATIC

sheet_config

When to use:

Your input connector uses XLS, XLSX, or XML files with multiple sheets
You want to specify which sheet contains the data to process
You need consistent sheet selection across multiple pipeline executions

Behavior:

For single-sheet file types (CSV, TSV, JSON): sheet_config is ignored
For multi-sheet files with only one sheet: sheet_config is optional (defaults to first sheet)
For multi-sheet files with multiple sheets: sheet_config determines which sheet to use

selection_type

Defines the strategy for selecting which sheet to process:

INDEX: Select sheet by its position (0-indexed). Use this when you always want to process the sheet at a specific position (e.g., always the 2nd sheet)
NAME: Select sheet by its name. Use this when you always want to process a sheet with a specific name (e.g., always the "CRM" sheet)

sheet_index

The 0-based index of the sheet to process. Required when selection_type is INDEX.

Examples:

0 = First sheet
1 = Second sheet
2 = Third sheet

Important: Backend uses 0-indexing. The first sheet is at index 0, not 1.

sheet_name

The name of the sheet to process. Required when selection_type is NAME.

Examples:

"Sheet1" = Process the sheet named "Sheet1"
"CRM" = Process the sheet named "CRM"
"Data" = Process the sheet named "Data"

Important: Sheet names are case-sensitive. The execution will fail if a sheet with the specified name is not found in the file.

developer_mode

created_at

The date and time when the pipeline template was first created

created_by

Information about whom created the pipeline template

The ID of the user or sub-organization who created the pipeline template

name

The name of the user or sub-organization who created the pipeline template

identifier

The identifier of the user or sub-organization who created the pipeline template

type

Defines the type of user who created the pipeline template:

USER: A user of your organization
SUB_ORG: A sub-organization that is part of your organization

updated_at

The date and time when the pipeline templatewas last updated

updated_by

Information about whom last updated the pipeline template

The ID of the user or sub-organization who last updated the pipeline template

name

The name of the user or sub-organization who last updated the pipeline template

identifier

The identifier of the user or sub-organization who last updated the pipeline template

type

Defines the type of user who last updated the pipeline template:

USER: A user of your organization
SUB_ORG: A sub-organization that is part of your organization

Response

{
  "data": {
    "id": "string",
    "name": "string",
    "pipeline_name": "string",
    "input_connectors": [
      "string"
    ],
    "output_connectors": [
      "string"
    ],
    "tdm": "string",
    "error_config": {
      "error_threshold": 0
    },
    "schedule_config": {
      "frequency": "HOURLY",
      "interval": 0,
      "starts_on": "2024-09-02T15:44:41.375Z",
      "ends_on": "2024-09-02T15:44:41.375Z"
    },
    "header_config": {
      "type": "SMART",
      "row_index": 0
    },
    "sheet_config": {
      "selection_type": "INDEX",
      "sheet_index": 0,
      "sheet_name": null
    },
    "developer_mode": false,
    "created_at": "2022-03-07 12:48:28.653",
    "created_by": {
      "id": "string",
      "name": "string",
      "identifier": "string",
      "type": "USER"
    },
    "updated_at": "2022-03-07 12:48:28.653",
    "update_by": {
      "id": "string",
      "name": "string",
      "identifier": "string",
      "type": "USER"
    }
  }
}

Example

cURL

Javascript

curl -X 'GET' 'https://api-gateway.ingestro.com/dp/api/v1/pipeline-template/${templateId}' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN'

Example

cURL

Javascript

curl -X 'GET' 'https://api-gateway.ingestro.com/dp/api/v1/pipeline-template/${templateId}' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN'

Read all

To further refine the response you can use query parameters like sort, filters, pagination and options. Look at a more detailed explanation here.

Endpoint

GET /pipeline-template/

Response

Attributes

The ID of the pipeline template

name

The name of the pipeline template

pipeline_name

The name of the pipeline created using the template

input_connectors

The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here

output_connectors

The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here

tdm

The ID of the set target data model

error_config

Defines how the pipeline created using the template should handle errors that might occur during pipeline execution

error_threshold

schedule_config

Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed

frequency

Sets how often the pipeline is executed. It is intertwined with interval. For example, if frequency is set to HOURLY and interval is set to 2, the pipeline is executed every 2 hours:

HOURLY
DAILY
WEEKLY
MONTHLY

interval

starts_on

The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g 2024-09-02T13:26:13.642Z). The date and time cannot be in the past

ends_on

The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time

header_config

Defines how the header row is determined

type

Specifies whether Ingestro's header detection is applied or if the set row_index is used to determine the header row:

SMART: Ingestro's header detection is used to define the header row
STATIC: The row at the specified row_index is used as the header row

row_index

The index of the row that should be used as the header row if type is set to STATIC

sheet_config

When to use:

Your input connector uses XLS, XLSX, or XML files with multiple sheets
You want to specify which sheet contains the data to process
You need consistent sheet selection across multiple pipeline executions

Behavior:

For single-sheet file types (CSV, TSV, JSON): sheet_config is ignored
For multi-sheet files with only one sheet: sheet_config is optional (defaults to first sheet)
For multi-sheet files with multiple sheets: sheet_config determines which sheet to use

selection_type

Defines the strategy for selecting which sheet to process:

INDEX: Select sheet by its position (0-indexed). Use this when you always want to process the sheet at a specific position (e.g., always the 2nd sheet)
NAME: Select sheet by its name. Use this when you always want to process a sheet with a specific name (e.g., always the "CRM" sheet)

sheet_index

The 0-based index of the sheet to process. Required when selection_type is INDEX.

Examples:

0 = First sheet
1 = Second sheet
2 = Third sheet

Important: Backend uses 0-indexing. The first sheet is at index 0, not 1.

sheet_name

The name of the sheet to process. Required when selection_type is NAME.

Examples:

"Sheet1" = Process the sheet named "Sheet1"
"CRM" = Process the sheet named "CRM"
"Data" = Process the sheet named "Data"

Important: Sheet names are case-sensitive. The execution will fail if a sheet with the specified name is not found in the file.

developer_mode

created_at

The date and time when the pipeline template was first created

created_by

Information about whom created the pipeline template

The ID of the user or sub-organization who created the pipeline template

name

The name of the user or sub-organization who created the pipeline template

identifier

The identifier of the user or sub-organization who created the pipeline template

type

Defines the type of user who created the pipeline template:

USER: A user of your organization
SUB_ORG: A sub-organization that is part of your organization

updated_at

The date and time when the pipeline templatewas last updated

updated_by

Information about whom last updated the pipeline template

The ID of the user or sub-organization who last updated the pipeline template

name

The name of the user or sub-organization who last updated the pipeline template

identifier

The identifier of the user or sub-organization who last updated the pipeline template

type

Defines the type of user who last updated the pipeline template:

USER: A user of your organization
SUB_ORG: A sub-organization that is part of your organization

pagination

An object containing metadata about the result

total

The number of entries in the data array

offset

The offset set in the request parameters

limit

The limit set in the request parameters

Response

{
  "data": [
    {
      "id": "string",
      "name": "string",
      "pipeline_name": "string",
      "developer_mode": false,
      "input_connectors": [
        "string"
      ],
      "output_connectors": [
        "string"
      ],
      "error_config": {
        "error_threshold": 0
      },
      "schedule_config": {
        "frequency": "HOURLY",
        "interval": 1,
        "starts_on": "2024-08-11T06:57:06.672Z",
        "ends_on": "2024-08-21T06:57:06.672Z"
      },
      "header_config": {
        "type": "SMART",
        "row_index": 0
      },
      "sheet_config": {
        "selection_type": "INDEX",
        "sheet_index": 0,
        "sheet_name": null
      },
      "created_at": "2022-03-07 12:48:28.653",
      "created_by": {
        "id": "string",
        "name": "string",
        "identifier": "string",
        "type": "USER"
      },
      "updated_at": "2022-03-07 12:48:28.653",
      "upadted_by": {
        "id": "string",
        "name": "string",
        "identifier": "string",
        "type": "USER"
      }
    }
  ],
  "pagination": {
    "total": 0,
    "offset": 0,
    "limit": 0
  }
}

Example

cURL

Javascript

curl -X 'GET' 'https://api-gateway.ingestro.com/dp/api/v1/pipeline-template/?sort=%7B%7D&filters=%7B%7D&options=%7B%7D&pagination=%7B%7D' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN'

Delete

Endpoint

DELETE /pipeline-template/{id}

Response

Attributes

message

Message confirming the deletion of the pipeline template or providing an error message

Response

{
  "data": {
    "message": "string"
  }
}

Example

cURL

Javascript

curl -X 'DELETE' 'https://api-gateway.ingestro.com/dp/api/v1/pipeline-template/${templateId}' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer ACCESS_TOKEN'

Create​

Response​

Example​

Update​

Response​

Example​

Example​

Sheet Selection Examples​

Example 1: Select Sheet by Position (INDEX)​

Example 2: Select Sheet by Name (NAME)​

Example 3: Pipeline Template Without Sheet Selection (Single-Sheet File)​

Read by ID​

Response​

Example​

Example​

Read all​

Response​

Example​

Delete​

Response​

Example​

Create

Response

Example

Update

Response

Example

Example

Sheet Selection Examples

Example 1: Select Sheet by Position (INDEX)

Example 2: Select Sheet by Name (NAME)

Example 3: Pipeline Template Without Sheet Selection (Single-Sheet File)

Read by ID

Response

Example

Example

Read all

Response

Example

Delete

Response

Example