Data Processing and Multi-Step Workflows

Beyond Simple Automations

In the previous lesson, we explored email and communication automations. Now let’s build on that foundation. In Lesson 4, you built communication automations – mostly single-trigger, single-action workflows. Now it’s time to level up to automations that cross system boundaries, transform data, and chain multiple operations together.

This is where automation gets genuinely powerful. Instead of automating a single task, you’re automating entire processes.

What You’ll Learn

By the end of this lesson, you’ll design multi-step workflows that connect different tools, transform data between formats, and process information across systems. You’ll handle the common challenges that make multi-step automations harder than single-step ones.

From Communication to Data

Lesson 4 focused on sending messages. This lesson focuses on moving and processing data. The difference matters: communication automations produce content for humans. Data automations produce structured changes in systems. Both use the same building blocks (triggers, actions, conditions) but the complexity of data flow is significantly higher.

The Multi-System Challenge

Here’s a common scenario: A sales rep closes a deal in the CRM. Now you need to:

1. Create the customer account in the billing system
2. Generate the contract in the document management system
3. Create the project in the project management tool
4. Add the customer to the onboarding email sequence
5. Notify the customer success team in Slack
6. Update the sales dashboard
7. Create the customer folder in the file storage system

That’s 7 different systems that need to be updated when one event happens. Manually, this process takes 30-45 minutes and errors are common (someone always forgets to create the project folder). Automated, it takes seconds and nothing gets missed.

But there’s a catch: each system stores data differently.

The Data Format Problem

CRM stores the customer name as: “Acme Corporation” Billing system needs: company_name: “Acme Corporation”, billing_contact: “Jane Smith” Project tool needs: project_name: “Acme Corporation - Implementation” File storage needs: folder_name: “acme-corporation” (lowercase, hyphenated)

Same information. Four different formats. This is why data transformation is the core skill of multi-step automation.

Data Transformation Basics

Data transformation converts information from one format to another. Common transformations:

Text manipulation:

• Split: “Jane Smith” → first_name: “Jane”, last_name: “Smith”
• Join: “Jane” + “Smith” → “Jane Smith”
• Format: “ACME CORP” → “Acme Corp” (title case)
• Slug: “Acme Corporation” → “acme-corporation”

Date manipulation:

• Format conversion: “01/15/2026” → “2026-01-15”
• Timezone conversion: “2026-01-15T10:00 EST” → “2026-01-15T15:00 UTC”
• Calculation: “start_date + 30 days” → “end_date”

Number manipulation:

• Currency conversion: “$1,500.00” → 1500 (removing symbols/formatting)
• Rounding: 4.567 → 4.57
• Percentage: 0.15 → “15%”

Structure manipulation:

• Flatten: nested object → flat key-value pairs
• Nest: flat fields → structured object
• Array to string: [“tag1”, “tag2”, “tag3”] → “tag1, tag2, tag3”

Using AI to plan transformations:

I need to move customer data from [Source System] to
[Destination System].

Source data format:
{
  "customer_name": "Acme Corporation",
  "contact": "Jane Smith, CEO",
  "phone": "(555) 123-4567",
  "deal_value": "$15,000.00",
  "close_date": "01/15/2026"
}

Destination requires:
{
  "company": string,
  "first_name": string,
  "last_name": string,
  "title": string,
  "phone": string (digits only),
  "amount": number,
  "date": "YYYY-MM-DD" format
}

List every transformation needed, step by step.

Quick Check

Think about two tools you use regularly. If you had to move data from one to the other, what format differences would you encounter? Those differences are exactly what data transformation steps solve.

Lookup Steps

Lookups are steps where your automation queries a data source to find additional information. They’re the “go check this” steps.

Common lookup patterns:

Design a lookup step:

STEP 3: CRM Lookup
  Input: customer_email (from trigger)
  Query: Search CRM contacts where email = {customer_email}
  Output: customer_name, account_id, tier, owner

  If found: Continue to Step 4
  If not found: Create new contact record, then continue
  If multiple found: Use most recently updated record, log warning

Always handle the “not found” case. Lookups that fail silently are one of the most common causes of automation bugs.

Multi-Step Workflow Patterns

Pattern 1: Sequential Pipeline

Data flows through a series of transformations:

[Trigger] → [Extract] → [Transform] → [Load] → [Notify]

Example: Daily sales report

1. Trigger: 8 AM every day
2. Extract: Pull yesterday’s sales data from POS system
3. Transform: Calculate totals, top sellers, comparison to prior day
4. Load: Update dashboard spreadsheet with new data
5. Notify: Send summary to sales manager via Slack

Pattern 2: Fan-Out

One trigger spawns multiple parallel actions:

[Trigger] → [Action A]
           → [Action B]
           → [Action C]

Example: New deal closed

1. Trigger: Deal marked “Closed Won” in CRM
2. Create billing account (billing system)
3. Create project (project tool) – in parallel
4. Send welcome email (email tool) – in parallel
5. Notify team (Slack) – in parallel

Pattern 3: Gather and Process

Multiple sources feed into one processing step:

[Source A] →
[Source B] → [Combine] → [Process] → [Output]
[Source C] →

Example: Weekly executive report

1. Pull sales data (CRM)
2. Pull support tickets (help desk)
3. Pull website metrics (analytics)
4. Combine into unified dataset
5. Generate summary and trends
6. Format and send executive report

Pattern 4: Iterative Processing

Loop through a collection, processing each item:

[Get list] → [For each item] → [Process] → [Next item] → [Summary]

Example: Monthly invoice processing

1. Get all pending invoices
2. For each invoice: validate amounts, apply discounts, calculate tax
3. Generate PDF for each invoice
4. Send each invoice to the appropriate client
5. Log all sends in accounting system

Designing a Multi-Step Workflow

Let’s design a complete multi-step workflow together:

Scenario: When a customer submits a support ticket, automatically categorize it, look up their account details, route it to the right team, and set expectations.

Help me design a multi-step automation for customer
support ticket routing.

Trigger: New support ticket submitted via form

Available data from form:
- Customer email
- Subject line
- Description
- Priority (selected by customer)

Systems involved:
- Support ticket system
- CRM (customer details)
- Slack (team notifications)
- Email (customer communication)

Design the complete workflow with:
1. Every step in order
2. Data transformations needed
3. Lookup steps (what additional data to fetch)
4. Conditions for routing
5. Actions at each step
6. What data flows between steps

Example output:

STEP 1: Receive ticket (Trigger)
  Data: email, subject, description, priority

STEP 2: CRM Lookup
  Input: customer_email
  Get: customer_name, tier (free/pro/enterprise), owner
  Not found: Flag as "unknown customer"

STEP 3: Categorize ticket (AI/rules)
  Input: subject, description
  Output: category (billing/technical/general/urgent)

STEP 4: Route based on conditions
  IF tier = "enterprise" AND priority = "high":
    → Route to senior support, SLA = 1 hour
  IF category = "billing":
    → Route to billing team, SLA = 4 hours
  IF category = "technical":
    → Route to technical support, SLA = 2 hours
  ELSE:
    → Route to general queue, SLA = 8 hours

STEP 5: Create ticket in support system
  Data: all collected data + category + route + SLA

STEP 6: Send acknowledgment to customer
  Template based on tier and category
  Include: ticket number, expected response time

STEP 7: Notify assigned team via Slack
  Include: customer name, tier, category, link to ticket

Common Pitfalls

1. Race conditions. When parallel actions depend on the same data, they can conflict. Solution: Design clear dependencies and sequence operations that modify the same data.

2. Data type mismatches. One system returns a number as a string, but the next system expects an integer. Solution: Always explicitly transform data types between steps.

3. Rate limits. APIs often limit how many requests you can make per minute. Solution: Add delays between API calls in loops, or batch your requests.

4. Partial failures. Step 5 of 7 fails. Now steps 1-4 executed but 6-7 didn’t. Solution: Design for graceful recovery (more on this in Lesson 6).

Exercise: Design a Multi-Step Workflow

Pick one of these scenarios and design the complete workflow:

1. Employee onboarding: When HR creates a new employee record, set up accounts in 5 systems
2. Order fulfillment: When an order is placed, process payment, update inventory, generate shipping label, and notify the customer
3. Content publishing: When a blog post is approved, format it, publish to the website, share on social media, and send to the email list

For your chosen scenario, define every step with inputs, outputs, transformations, conditions, and failure handling.

Key Takeaways

• Multi-step workflows connect different systems that store data in different formats
• Data transformation is the core skill: converting between formats, splitting/joining fields, reformatting dates and numbers
• Lookup steps enrich your workflow with additional data from other systems
• Common patterns: sequential pipeline, fan-out, gather-and-process, and iterative loops
• Always handle the “not found” case in lookups and plan for partial failures
• Map data flow explicitly – know what each step receives and what it passes forward

Next lesson: things will go wrong. Let’s make sure your automations handle errors gracefully.