Simple Agent Designer Documentation
Index

Data Splitting

Data splitting is the process of breaking a large dataset into individual workloads, each containing one record or row. Simple Agent Designer supports two types of splitting: Table Splitting for Business Central data and Excel Splitting for spreadsheet uploads.

Why Split Data?

AI models work best when focused on one item at a time. Instead of sending an entire customer table to an agent with "review all these customers," data splitting creates a focused workload per customer. This approach:

  • Produces better AI results (focused context per record)
  • Enables parallel processing of individual records
  • Provides per-record status tracking and result logging
  • Allows failed records to be reprocessed without re-running everything
  • Supports incremental processing (only new/changed records)

Table Splitting

Table splitting creates child workloads from Business Central table records.

How It Works

  1. You configure a DataBased workload type with a table number and optional filters
  2. When the workload is processed, the orchestration engine calls the Table Splitter
  3. The Table Splitter queries the specified table with the configured filters
  4. For each matching record, a child workload is created containing:
    • The original workload description
    • The record's data converted to JSON (all fields including calculated fields)
    • A DataContext reference (the record's SystemId)
    • An auto-generated title based on the record type

Configuration

On the Workload Type Card:

Setting Value
Operation DataBased
Table No. The table to query (e.g., 18 for Customer)
Filter String Optional BC filter expression
Add Since Last Filter Yes/No for incremental processing

Title Generation

The Table Splitter generates meaningful titles for child workloads based on the table type:

Table Title Format Example
Customer (18) "Name (No.)" "Contoso Ltd (C10000)"
Vendor (23) "Name (No.)" "Fabrikam (V20000)"
Item (27) "Description (No.)" "Widget A (ITEM-001)"
Other tables First text field value, or first key value Varies

Record JSON Format

Each child workload receives the record as a JSON object with all fields:

{
  "No.": "C10000",
  "Name": "Contoso Ltd",
  "Address": "123 Main Street",
  "City": "Seattle",
  "Balance (LCY)": 15000.00,
  "Credit Limit (LCY)": 50000.00,
  ...
}

The complete JSON is appended to the workload description, giving the agent full context about the record.

Incremental Processing

When Add Since Last Filter is enabled:

  1. First run: All records matching the filter are processed. The LastRun timestamp is set to the current time.
  2. Subsequent runs: Only records where SystemModifiedAt >= LastRun are included. This captures new records and modified records since the last processing.
  3. The LastRun field on the Workload Type card shows when the type was last processed.

This is ideal for scheduled or recurring batch processing where you only want to handle changes.

Data Context

Each child workload's DataContext field stores the SystemId of the source record. When the assigned agent initializes, it can use this DataContext to:

  • Open the original record for tool operations
  • Update the record directly through the Data Modifier worker type
  • Reference the record in subsequent tool calls

Excel Splitting

Excel splitting creates child workloads from rows in an uploaded Excel file.

How It Works

  1. You configure a PayloadBased workload type with an Excel Splitter setting
  2. The user uploads an Excel file to the workload
  3. When processed, the orchestration engine calls the Excel Splitter
  4. The Excel Splitter reads the file and creates a child workload per row containing:
    • The original workload description
    • The row data as JSON
    • An AI-generated title summarizing the row content

Configuration

On the Workload Type Card:

Setting Value
Operation PayloadBased
Use Excel Splitter Yes With Header or Yes No Header

Header Handling

Yes With Header

  • Row 1 is treated as the header row
  • Column names from the header become JSON field names
  • Processing starts from Row 2

Example:

Excel file:

Vendor Invoice Amount
Contoso INV-001 500.00
Fabrikam INV-002 1200.00

Child workload 1 JSON:

{
  "Vendor": "Contoso",
  "Invoice": "INV-001",
  "Amount": "500.00"
}

Yes No Header

  • All rows are data rows (no header)
  • Columns are named automatically: Column A, Column B, Column C, etc.
  • Processing starts from Row 1

Example:

Excel file:

Contoso INV-001 500.00
Fabrikam INV-002 1200.00

Child workload 1 JSON:

{
  "Column A": "Contoso",
  "Column B": "INV-001",
  "Column C": "500.00"
}

AI Title Generation

The Excel Splitter uses the AI to generate a brief title for each child workload by sending the row's JSON to a summarization prompt. This makes it easy to identify individual rows in the Workload List.

Duplicate Detection

The Excel Splitter checks for duplicates using HaveWeSeenThisWorkloadBefore(). If a row's content matches a previously processed workload:

  • The child workload is still created
  • But its status is automatically set to Fail
  • This prevents reprocessing of identical data when the same Excel file is uploaded again

Additional Setting

The workload's Additional Setting field can specify the Excel sheet name to process if the file contains multiple sheets.

Viewing Split Results

After splitting:

  1. The parent workload's status is set to Success (the splitting itself succeeded)
  2. Open the parent workload and click the Child Workloads count to see all child workloads
  3. Each child workload can be:
    • Viewed individually on its Workload Card
    • Processed individually or as part of "Process All"
    • Reviewed for status and results

Processing Split Workloads

Splitting and processing happen in a single Process action. As soon as the splitter finishes creating child workloads, the orchestration engine automatically iterates over every child with status Not Started and runs each one through the agent pipeline. You do not need to process the children manually — the parent's Process action carries through end to end. Each child is processed independently, so a failure in one does not affect the others.

If you later need to rerun specific children (for example, after fixing a failure), you can still process them manually:

  1. Navigate to the parent workload
  2. Drill down into child workloads
  3. Use Process All on the child workload list to reprocess all pending children
  4. Or process individual children one at a time

Tips and Best Practices

  • For table splitting, start with narrow filters to test before processing entire tables
  • Use incremental processing for regular batch jobs to avoid reprocessing unchanged records
  • For Excel splitting, ensure your data is clean — empty rows or inconsistent formatting can create empty child workloads
  • Use "Yes With Header" for Excel files whenever possible, as named fields are easier for agents to understand
  • Review a few child workloads before processing all to verify the split produced expected results
  • For very large datasets, consider splitting into smaller batches to manage processing time and costs