Automation Rules Audit

Last reviewed: 2026-05-10

This document explains what happens when a form submission is processed, how filter rules are resolved, how actions are executed, and what edge cases are currently covered or still need attention.

Executive Status

Automation Rules are production-usable for the normal public form submission path, especially after the latest backend hardening pass.

The most important guarantees now in place:

• Global settings and Automation Rules are independent. If both are configured, both can run.
• Automation Rules run after AI enrichment, so AI fields such as use_case_metrics.* and extracted_data.* are available for routing, integrations, alerts, tags, and case updates.
• Integration payloads are still built from the single canonical response.submitted payload builder.
• The filter layer now resolves raw answer fields, AI metrics, extracted data, contact fields, hidden fields, and custom fields.
• Alert notification plan gating no longer blocks non-notification automation actions.

Important remaining caveat:

Not every edge case is fully resolved. The normal form-submission path is solid, but there are known gaps around API Intake integration delivery, synchronous thank-you rule parity, duplicate delivery semantics, and a few operator-specific behaviors.

Main Form Submission Flow

The normal public form path is:

1. A respondent submits a ResponseSession.
2. create_inbox_case creates or reuses an InboxCase.
3. analyze_case_with_ai enriches the case:
   • ai_summary
   • sentiment
   • urgency_score
   • category
   • extracted_data
   • use_case_metrics
   • use_case_key_points
4. Always-on review destinations from link.settings.destination_assignments are routed.
5. WorkflowRuleEngine.evaluate_and_execute(case) evaluates Automation Rules.
6. WebhookTriggerService.trigger_response_completed(...) sends integration payloads for:
   • per-form integration assignments
   • global webhooks
   • global Zapier hooks
   • global Slack destinations
   • global Google Sheets destinations
   • workspace notification-settings raw webhook URL
7. Responder notification is queued after automation so rule-level responder overrides can be used.
8. Response embeddings and other async work continue.

The important ordering is:

AI enrichment
-> always-on destinations
-> Automation Rules
-> global/per-form integrations
-> responder notification

This means Automation Rules can use enriched AI fields and can prepare rule-level responder or thank-you overrides before downstream notification tasks run.

Where Rules Are Stored

Automation Rules are stored on the form/link:

{
  "alert_rules": [
    {
      "type": "workflow_rule",
      "name": "High ICP lead",
      "enabled": true,
      "conditions_logic": "AND",
      "conditions": [
        {
          "field": "use_case_metrics.icp_fit",
          "op": "equals",
          "value": "Strong Fit"
        }
      ],
      "actions": [
        {
          "type": "send_to_integration",
          "value": "integration-uuid"
        }
      ]
    }
  ]
}

Only rules with:

• type == "workflow_rule"
• enabled != false

are executed by the workflow rule engine.

Filter Resolution

All Automation Rule filters use ConditionEvaluator.

Supported condition shapes:

{
  "field": "company_size",
  "op": "equals",
  "value": "Enterprise"
}

Legacy shape is also supported:

{
  "source_question_key": "company_size",
  "operator": "equals",
  "value": "Enterprise"
}

Logic Resolution

The backend normalizes these logic values:

Input	Meaning
all	AND
AND	AND
any	OR
OR	OR

Unknown or empty logic defaults to all.

Field Resolution Order

For each condition, the backend resolves field in this order:

1. use_case_metrics.*
   • Reads from case.use_case_metrics.
   • Example: use_case_metrics.lead_score.
2. extracted_data.*
   • Reads from case.extracted_data.
   • Supports nested keys and unwraps { "value": ... }.
   • Example: extracted_data.budget.
3. responses.* or answers.*
   • Reads from submitted answers by Question.field_key.
   • Example: responses.company_size.
4. contact.*
   • Reads from session.contact, case.contact, hidden fields, raw API payload, or respondent metadata.
   • Example: contact.email.
5. hidden_fields.*
   • Reads from session.hidden_fields.
   • Example: hidden_fields.utm_source.
6. custom_fields.*
   • Reads from case.extracted_data.custom_fields, direct extracted fields, and answer-level extracted custom fields.
   • Example: custom_fields.procurement_owner.
7. Top-level InboxCase fields.
   • Examples: sentiment, urgency_score, priority, status, category, tags.
8. Bare answer field key fallback.
   • Example: company_size resolves the same answer as responses.company_size.

This is the key production fix: filter fields now align much more closely with the payload fields customers use in webhooks, Zapier, Slack, and Google Sheets.

Operators

Supported operators:

Operator	Behavior
equals, eq	Case-insensitive equality
not_equals, neq	Case-insensitive inequality
contains	String contains or list contains
not_contains	String/list does not contain
in_list, in	Actual value is in expected list or comma-separated string
not_in_list, not_in	Actual value is not in expected list
greater_than, gt	Numeric greater-than
less_than, lt	Numeric less-than
greater_than_or_equals, gte	Numeric greater-than-or-equal
less_than_or_equals, lte	Numeric less-than-or-equal
between	Numeric or date range
is_empty	Null, blank, empty list, or empty dict
is_not_empty, not_empty, exists	Present and not blank
is_true	Boolean-ish true
is_false	Boolean-ish false
before, after, days_ago_within	Date operators
domain_equals, domain_in_list	Email domain matching
sentiment_above, sentiment_below, sentiment_is	Specialized sentiment checks
urgency_is, intent_is, persona_is	Specialized exact checks
action_required	Boolean action-required check
topic_contains	Topic string contains
custom:*	Looks in case.extracted_data.custom_routing

Condition errors fail closed. A broken condition evaluates as false, but it does not stop the submission.

Automation Actions

When a rule matches, every action in the rule is attempted.

set_priority

Updates case.priority.

Example:

{ "type": "set_priority", "value": "high" }

set_status

Updates case.status.

Example:

{ "type": "set_status", "value": "qualified" }

add_tag

Adds a tag to case.tags if it is not already present.

Example:

{ "type": "add_tag", "value": "vip" }

assign

Assigns the case to a user by user UUID.

Example:

{ "type": "assign", "value": "user-uuid" }

If the user is missing, the action is skipped and logged.

set_category

Updates case.category.

Example:

{ "type": "set_category", "value": "enterprise_lead" }

route_to_destination

Routes the case to a review table (Destination) by destination UUID.

Example:

{ "type": "route_to_destination", "value": "destination-uuid" }

This is separate from integration destinations such as webhooks and Zapier.

show_thank_you

Stores a resolved thank-you config on case.extracted_data.resolved_thank_you.

Example:

{
  "type": "show_thank_you",
  "value": {
    "title": "Thanks, qualified lead",
    "message": "Book a call with our team.",
    "redirect_url": "https://example.com/calendar"
  }
}

First match wins. If another rule already set resolved_thank_you, later show_thank_you actions are skipped.

Important: the respondent-facing thank-you page is also resolved synchronously at submit time by WorkflowTYResolver, before the async InboxCase automation pass finishes.

send_auto_responder

Stores a resolved responder config on case.extracted_data.resolved_auto_responder.

Example:

{
  "type": "send_auto_responder",
  "value": {
    "enabled": true,
    "subject_line": "Thanks for your interest"
  }
}

First match wins. The responder task runs after automation, so it can use this rule-level override.

send_to_integration

Sends the canonical response.submitted payload to one specific integration destination, or multiple integration destinations.

Single destination:

{ "type": "send_to_integration", "value": "integration-uuid" }

Multiple destinations:

{ "type": "send_to_integration", "value": ["uuid-1", "uuid-2"] }

Supported destinations:

• webhook
• Zapier
• Slack
• Google Sheets

Plan gates and configuration checks are still applied. For example, Zapier will not send if the workspace plan does not include Zapier, and Slack will not send if no channel is configured.

Payload behavior:

• Webhook receives the nested canonical response.submitted payload.
• Zapier receives the flattened Zapier-compatible payload.
• Slack receives a Slack Block Kit payload built from the same canonical data.
• Google Sheets receives the canonical payload for row building.

notify

Creates a CaseAlert and dispatches notifications.

Example:

{
  "type": "notify",
  "value": {
    "channels": ["email", "slack"],
    "recipients": ["ops@example.com"]
  }
}

Notification config can now provide:

• channels
• email_targets
• recipients
• emails
• notify_email
• notify_webhook
• notify_slack

Alert notification delivery remains plan-gated by alert_notifications. This is intentional. Non-notification actions are no longer blocked by that alert feature gate.

Global Settings vs Automation Rules

Global settings and Automation Rules are independent.

If global/per-form integrations are configured, they fire through WebhookTriggerService.trigger_response_completed.

If an Automation Rule has send_to_integration, that rule also sends to its selected integration when conditions match.

This means the same integration can receive more than one delivery if it is configured in both places:

• once from a matching Automation Rule
• once from a global or per-form integration assignment

That is current behavior and matches "independent" execution. If the desired product behavior is "rules override global delivery for the same integration", then an explicit suppression/dedup layer needs to be added.

Integration Filter Rules

There are two filter systems:

1. Integration-level filters
   • Stored on IntegrationDestination.filter_rules.
   • Used by global integrations.
2. Per-form assignment filters
   • Stored on LinkIntegrationAssignment.filter_rules.
   • Override destination-level filters for that form assignment.

Filter behavior:

• No filters means deliver everything.
• If filters exist and do not match, delivery is skipped.
• Webhook and Zapier filtered skips create IntegrationDeliveryLog rows with status filtered.
• Case timeline receives integration_filtered events when a case exists.

Current limitation:

Slack and Google Sheets filtered skips log case events, but they do not always create IntegrationDeliveryLog rows because those delivery paths are task-based rather than webhook-log based.

Thank-You Resolution

There are two thank-you phases.

Synchronous respondent phase

WorkflowTYResolver runs during submission response generation. It:

1. Reads enabled workflow_rule rules with show_thank_you.
2. Builds raw answer values from the submitted answers.
3. If AI fields are referenced, runs a lightweight AI scoring call.
4. Evaluates rules in order.
5. Returns the first matching show_thank_you value.

This determines what the respondent sees immediately.

Async case phase

Later, WorkflowRuleEngine runs after the InboxCase has been AI-enriched. It can also execute show_thank_you and store resolved_thank_you on the case for audit/history.

Current limitation:

WorkflowTYResolver does not yet use the full ConditionEvaluator field resolution surface. It supports bare answer field keys and AI-style use_case_metrics.* or extracted_data.* quick scoring, but it does not fully support responses.*, hidden_fields.*, contact.*, or custom_fields.* in the same way the async workflow engine does.

API Intake Flow

API Intake uses a separate task: process_api_intake.

It currently:

1. Loads the raw intake payload.
2. Runs AI extraction.
3. Creates an InboxCase.
4. Fires legacy alert rules.
5. Routes always-on review destinations.
6. Runs WorkflowRuleEngine.
7. Creates suggested actions.
8. Embeds the case.
9. Sends a callback if requested.

Current limitation:

API Intake does not currently call WebhookTriggerService.trigger_response_completed. That means global/per-form integration delivery is not fully mirrored for API Intake unless a matching Automation Rule explicitly uses send_to_integration, or the caller provides a callback URL.

Deduplication Behavior

Within one WorkflowRuleEngine pass, duplicate non-notify actions are deduped by:

(action_type, action_value)

Examples:

• Two rules that both add tag vip will only execute that exact tag action once.
• Two rules that both send to the exact same integration UUID will only execute that exact action once.

Not deduped:

• notify actions are intentionally not deduped this way because separate rules may create separate alerts.
• Rule-triggered integration delivery is not deduped against later global/per-form integration delivery.
• A list-valued send_to_integration action is deduped as a whole list string, not per integration UUID across differently ordered lists.

Failure Behavior

Automation is designed to fail open for submissions and fail closed for bad conditions.

Failure	Behavior
Bad condition/operator	Condition evaluates false
Unknown action type	Logged and skipped
Missing destination	Logged and skipped
Missing integration config	Delivery skipped
Plan-gated integration	Delivery skipped
Alert notification feature unavailable	Notification skipped, non-notification actions still run
Delivery task fails	Delivery service/task records failure or logs error
Thank-you quick AI scoring fails	Falls back to default thank-you

Edge Case Audit

Resolved or covered

Edge case	Status	Notes
Rules using bare answer keys such as company_size	Resolved	Bare keys now resolve submitted answers by Question.field_key.
Rules using responses.company_size	Resolved	responses.* and answers.* now resolve submitted answers.
Rules using hidden fields	Resolved for async workflow/integration filters	hidden_fields.* is supported in ConditionEvaluator.
Rules using contact fields	Resolved for async workflow/integration filters	contact.* resolves contact, hidden fields, raw intake payload, and respondent metadata.
Rules using custom extracted fields	Resolved for async workflow/integration filters	custom_fields.* is supported.
AND/OR versus all/any mismatch	Resolved	Logic normalization is shared in async workflows and improved for thank-you resolver.
Alert plan gate blocking integrations/routing/tags	Resolved	Workflow engine no longer exits early on missing alert_notifications.
Zapier payload shape	Covered	Rule integration delivery uses flattened Zapier payload.
Canonical webhook payload	Covered	Webhooks use build_response_completed_payload.
Global integration filters	Covered	Destination-level filters evaluate before delivery.
Per-form integration filters	Covered	Assignment filters override destination filters.
Rule-level notification recipients	Improved	notify action config can now drive email recipient selection.
Rule-level notification channels	Improved	notify action config can now specify channels.

Remaining edge cases

Edge case	Risk	Recommendation
API Intake does not trigger global/per-form integrations	High if API campaigns rely on global destinations	Add WebhookTriggerService.trigger_response_completed(session, responses=[]) after WorkflowRuleEngine in process_api_intake, with tests.
Synchronous thank-you resolver does not support all ConditionEvaluator field sources	Medium	Extend WorkflowTYResolver to support responses.*, hidden_fields.*, contact.*, and custom_fields.*, or create a temporary case-like context.
Rule-triggered integration plus global/per-form integration can duplicate delivery	Medium, depending on product expectation	Keep if "independent" means both fire. Add explicit dedup/suppression if rule delivery should replace global delivery for the same destination.
Slack and Google Sheets filtered skips do not consistently create IntegrationDeliveryLog records	Medium for audit/reporting	Create filtered delivery-log records for all destination types, or introduce a generic automation execution log.
sentiment_is is numeric-score oriented	Low to medium	If UI sends field sentiment plus op sentiment_is, string sentiment values can behave unexpectedly. Prefer equals for sentiment, or update sentiment_is to handle strings directly.
send_to_integration list dedup is list-string based	Low	Dedup each integration UUID individually across all matched rules.
Invalid set_priority or set_status values are not model-choice validated before save	Low	Validate against model choices before saving to avoid bad workflow config causing save errors.
WorkflowTYResolver uses print debugging	Low operational noise	Replace with structured logger calls before production campaigns.
Notification action config is flexible but not schema-validated	Low to medium	Add serializer/save-time validation for notify.value.channels and recipient lists.
Integration filter API exposure exists, but form builder save contract should be verified end-to-end	Medium frontend risk	Confirm frontend sends and reloads filter_rules and filter_logic for both global destinations and per-form assignments.

Production Readiness Verdict

For normal public form submissions:

Ready with caveats.

The core backend path now supports conditional filters and actions using the same submission data that integrations receive. It is safe for campaigns where Automation Rules are used for:

• routing cases to review destinations
• sending conditional webhooks or Zapier payloads
• sending conditional Slack or Google Sheets deliveries
• setting priority/status/category/tags
• assigning cases
• overriding responder notification config
• creating conditional alerts and notifications

For API Intake campaigns:

Not fully equivalent yet.

API Intake runs Workflow Rules, but it does not yet fire the global/per-form integration pipeline. Use explicit send_to_integration Automation Rule actions or callback URLs until the API Intake integration delivery gap is closed.

For conditional thank-you pages:

Mostly ready for answer fields and AI fields, but not full parity.

Bare answer field keys and AI-scored fields work. More advanced field prefixes should be tested or avoided for respondent-facing synchronous thank-you rules until resolver parity is added.

Recommended Final Hardening Before Campaign

Highest-priority checks:

1. Submit a real test response that should match one Automation Rule and verify:
   • case timeline has rule_evaluated
   • case timeline has workflow_rule_fired
   • expected action event is logged
   • integration receives exactly the expected payload
2. Test one non-matching response and verify:
   • no rule action executes
   • integration delivery is filtered or absent as expected
3. Test one rule using a raw answer field key.
4. Test one rule using use_case_metrics.*.
5. Test one rule using a send_to_integration Zapier destination.
6. Test one global integration plus one Automation Rule integration and confirm whether duplicate delivery is desired.
7. If the campaign uses API Intake, add or manually verify API Intake integration behavior before launch.

Recommended backend fixes if time allows:

1. Add API Intake call to WebhookTriggerService.trigger_response_completed.
2. Bring WorkflowTYResolver onto the same field-resolution contract as ConditionEvaluator.
3. Add filtered delivery logs for Slack and Google Sheets.
4. Add schema validation for Automation Rule actions at form save time.
5. Replace WorkflowTYResolver print calls with logger calls.