Workflow
Pipeline Overview
The workflow is a 10-step deterministic pipeline with agent-in-the-loop judgment calls. Steps labeled [CODE] are pure deterministic functions. Steps labeled [AGENT] involve LLM judgment (schema-validated). [HUMAN] steps require explicit approval.
Step Details
Step 1–3: Ingestion & Normalization [CODE]
| Step | Description |
|---|
fetch_listings | Fetches 13 raw listings from 2 mock providers (Jaap, Funda) |
normalize_listings | Maps provider schemas to NormalizedListing. Runs prompt injection scanner. Flags suspicious content. |
deduplicate_listings | Cross-provider dedup by address + listing key. Removes exact and near-duplicates. |
Step 4–5: Enrichment
| Step | Actor | Description |
|---|
calculate_commutes | CODE | Computes commute time + distance in parallel with concurrency limit |
research_neighbourhoods | AGENT | Assesses each neighbourhood on 3 dimensions: quiet, transport, green space. Produces evidence-cited assessments. |
Step 6–7: Scoring
| Step | Actor | Description |
|---|
calculate_deterministic_scores | CODE | Affordability, commute, neighbourhood, space scores. Hard constraints applied. Recommendation tier. |
generate_qualitative_evaluations | AGENT | Agent evaluates fit against qualitative preferences. Cannot override hard constraint results. |
Step 8–10: Delivery & Approval
| Step | Actor | Description |
|---|
build_shortlist | CODE + AGENT | Ranks by combined score. Agent synthesizes summary with trade-offs. |
draft_realtor_message | AGENT | Drafts professional email. Agent cannot send — only draft. |
create_pending_action | CODE | Creates pending action with idempotency key + payload hash. Enters awaiting_approval. |
Approval & Execution [HUMAN + CODE]
| Step | Actor | Description |
|---|
await_human_approval | HUMAN | User reviews draft. Can edit, approve, or reject. |
execute_approved_action | CODE | Verifies payload hash. Checks idempotency. Sends email. Records atomically. |
State Machine
created → listings_fetched → listings_normalized → listings_deduplicated
→ enrichment_running → enrichment_complete → ranking_complete
→ shortlist_created → awaiting_approval → action_executed → completed
Any state can transition to: failed → retry
Transitions are validated by is_valid_transition() — invalid transitions rejected at the controller level.
Retry & Recovery
| Mechanism | How |
|---|
| Per-step retry | Tenacity-based with exponential backoff. Configurable per step. |
| Crash recovery | State persisted after each step. resume_workflow() reconstructs from DB. |
| Idempotency | Every action keyed: email:listing_id:recipient:payload_hash. Duplicates blocked. |
| Reconciliation | Post-crash: detects sent email, marks completed — no duplicate send. |
Concurrency
Steps processing multiple items run in parallel with a configurable semaphore (max_concurrent_enrichments, default 4). Applies to: calculate_commutes, research_neighbourhoods, generate_qualitative_evaluations.
Audit Trail
Every transition, agent call, security event, and external action produces an audit event:
| Event Type | Actor | Trigger |
|---|
workflow.started | SYSTEM | Workflow creation |
workflow.step_started | DET | Step begins |
workflow.step_completed | DET / AGENT | Step finishes |
security.suspicious_content | DET | Prompt injection detected |
action.created | SYSTEM | Pending action created |
action.approved | HUMAN | User approves |
action.executed | TOOL | External action complete |
action.duplicate_prevented | SYSTEM | Idempotency check blocks duplicate |
system.recovery | SYSTEM | Crash recovery reconciles |