Load external data

The Load External Data workflow action allows users to import data from various external sources into a Tabidoo application. This action supports multiple data formats and can be used to automate data synchronization, migrate records, or integrate with external databases and files. This action can be followed by the Map Fields and Save Records workflow action to process and store the imported data efficiently.

Data Source Options

When configuring this action, you can choose from the following data sources:

• Load data from an Excel file (.xlsx, .xls) stored in Tabidoo or provided via a URL or Import structured text data from a CSV file.
• Fetch data from any table within an accessible application, as long as the user has sufficient permissions to perform this action
• Load structured JSON data from an external API or file.
• MS SQL

Configuration Options

Data Source Internal Name

Defines the internal property in the workflow where the loaded data will be stored.

Format: doo.workflow.runningData["..."].items

This value can be referenced in subsequent workflow steps.

Data Location

Depending on the selected data source, you may need to specify how the data should be retrieved:

HTTP - Load data from a publicly accessible URL.

FTP - Retrieve data from an FTP server.

SMBv1 / SMBv2 - Fetch data from a shared network drive using the SMB protocol.

Tabidoo Table Configuration

When selecting Tabidoo Table as the data source, additional settings are required:

Choose Application - Select the Tabidoo application where the source table is located.

Choose Table - Select the specific table within the chosen application.

Filter - Optionally define filters to refine the data being loaded.

Data Source Internal Name - Define the internal property name where the retrieved data will be stored.

SQL Configuration

If SQL is selected as the data source, the following fields must be configured:

Data Source Internal Name - Specifies where the imported data will be stored in the workflow.

Server - The hostname or IP address of the SQL server.

Database - The name of the SQL database to connect to.

User Name - Credentials used for database authentication.

Password - Secure authentication for database access.

Procedure or Table - Choose between executing a stored procedure or retrieving data directly from a table.

Procedure Name or Table Name - If using a procedure, specify the stored procedure name; if using a table, provide the table name.

Configuration Steps

Select Load External Data as the workflow action.

Choose the appropriate data source from the list.

Define the Data Source Internal Name, which will store the imported data.

Select the method for retrieving the data (HTTP, FTP, SMBv1, SMBv2) if applicable.

If using Tabidoo Table, choose the application and table, and set any optional filters.

If using SQL, configure the server, database, credentials, and procedure/table details.

Configure additional settings such as file path, URL, API endpoint, or SQL query.

Execute or schedule the workflow to automate data loading.

Define Mapping Rules

When importing data into a Tabidoo application, mapping rules play a critical role in aligning the imported data structure with the target table's fields. This step ensures that the data flows into the correct fields and is properly formatted for use. Mapping rules are particularly useful in the following scenarios:

Field Name Differences:

The field names in the source data may not match the field names in the destination table. For example, a column labeled "First_Name" in the source file may need to map to a field labeled "Name" in the destination table.

Mapping allows you to explicitly specify the relationship between source columns and target fields.

Data Transformation Requirements:

Before data is saved to the destination table, it might need to be transformed. For example:

Converting date formats (e.g., from YYYY/MM/DD to DD-MM-YYYY).

Combining multiple source columns into one target field (e.g., concatenating "First Name" and "Last Name").

Splitting a single source column into multiple destination fields.

Validation and Alignment:

Mapping rules can ensure that the data complies with the structure and constraints of the destination table. For example:

Ensuring numeric data is correctly formatted before being saved into numeric fields.

Validating email formats before saving them into an "Email" field.

Usage of Formulas:

The mapping interface supports the use of formulas that act like "calculated fields." Users can:

Perform text operations such as concatenation, trimming, or upper/lower case transformations.

Apply mathematical operations like summing, dividing, or calculating percentages.

Work with date functions to calculate intervals, add/subtract days, or transform date formats.

Formulas can be validated directly in the interface, with a confirmation message displayed when the formula is correct (e.g., "The formula is valid").

Dynamic Field Behavior:

Mapping rules give users flexibility to include logic or computed values during the import. For example:

Automatically assign a category or status based on imported data values.

Populate default values in empty fields using conditional formulas.

By defining these mapping rules, users can ensure that imported data is seamlessly integrated into the target table while meeting all necessary structural and formatting requirements. This process not only automates data transformation but also enhances the accuracy and usability of the imported records.

Use Cases

Automated Data Import – Periodically update Tabidoo records from an external file or database.

Data Consolidation – Merge and unify data from multiple sources into one place.

ETL Processes – Extract, transform, and load data for reporting and analytics.

Third-party Integration – Sync data from APIs, external CRMs, or other business systems.

Database Connectivity – Fetch records from an SQL database for reporting or further processing.

Errors and Notes

Invalid SQL Server Connection:

If Tabidoo cannot connect to the SQL server due to incorrect server address, authentication details, or network issues, the operation will be interrupted.

Note: Ensure the server configuration, database name, username, and password are correct.

Incorrect Data Format:

Improperly formatted files, such as CSV or JSON, can lead to data loading errors. Common issues include invalid delimiters in CSV or malformed JSON structure.

Note: Verify that the file format matches the expected structure specified in the documentation.

SMB Connection Issues:

Errors can occur if the path to the shared network drive is incorrect or if access rights are insufficient.

Note: Check that the provided credentials and access permissions are valid and that the network drive is accessible.

URL Loading Failures:

If the specified URL is invalid or inaccessible, data retrieval will fail.

Note: Confirm that the URL points to publicly accessible and properly structured data.

File Size Limitations:

Large files may slow down the import process or exceed memory limits.

Note: Consider splitting large data files into smaller chunks or optimizing their structure.

User Permissions:

Data loading may be restricted based on the user's access level. For instance, users without proper permissions may not be able to access certain applications or tables.

Note: Ensure the user has the required permissions to perform this operation.

Mapping Errors:

If imported data isn't correctly mapped to the target Tabidoo structure, issues may arise during saving or synchronization.

Note: Spend time configuring mapping rules to align data properly.