Failure Modes v3.9

Last updated: September 30, 2026

Most vendors hide how their systems fail. We lead with it. Understanding failure behavior is more useful than understanding success behavior — because failures are where trust is actually tested.

---

Design Principle: Suppress Over Guess

AI4Love is designed to produce no output rather than incorrect output. This is the core failure behavior across every layer:

• If an agent cannot determine a pattern with confidence, it does not generate an insight.
• If the LLM returns text that doesn't match the expected pattern type, the insight is discarded.
• If a data sync encounters records that can't be matched to a supporter by email, those records are skipped — not force-matched.

Silence is safer than noise. A missing insight costs nothing. A wrong insight erodes trust.

---

Agent Failure Behavior

Scenario	System Response
Agent can't reach Airtable	Agent run fails. No insights written. Error logged. Next nightly run retries.
Agent can't reach Claude API	Pattern detection completes (it's math), but text generation is skipped. Insight is queued without a headline for the next run.
Supporter has insufficient data	Eligibility filters suppress the insight. Minimum activity thresholds prevent low-confidence recommendations.
LLM returns malformed text	Validation rejects the response. Insight is not saved. Error is logged.
LLM claims a pattern the data doesn't support	Server-side validation re-checks each insight against the supporter's actual timeline before anything is written: the supporter must be in the analyzed batch, the pattern must be one that agent detects, and eligibility thresholds and claimed metrics must match the data. Failing insights are discarded and logged.
Duplicate insight detected	Deduplication logic prevents the same pattern from being surfaced repeatedly for the same supporter within a configurable window.

No Hallucinated Insights

Three layers prevent fabrication:

1. Deterministic rules identify the pattern first. The LLM only writes text after the pattern is confirmed. If the pattern detection layer finds nothing, the LLM is never called.

2. Content Integrity Policy (v2026-09-09) is prepended to every agent prompt and every campaign generation call. It prohibits the LLM from inventing names, programs, outcomes, causal claims, quotes, or statistics not present in the provided context, and from inferring a condition, diagnosis, or personal reason behind a gift from its designation. If specifics are missing, the LLM must write categorically or return NEEDS_GROUNDING: instead of fabricating.

3. Post-generation verification via /api/verify-insight compares AI-claimed metrics (dollar amounts, participation counts, days silent) against actual Airtable data. Each new insight is badged as verified, mismatch, or unverifiable. Staff can see whether the numbers check out.

This means AI4Love cannot fabricate a donation trend, invent a beneficiary story, or misstate a supporter's giving history. The math supports the pattern, the policy constrains the language, and the verification catches discrepancies.

---

Integration Failure Behavior

Scenario	System Response
OAuth token expired	Nango automatically refreshes. If refresh fails, sync is paused and the integration shows "disconnected" in the dashboard. No partial writes.
Source API rate limited	Exponential backoff with max 3 retries. If all retries fail, sync stops and reports the error. No data is lost — the next sync picks up where it left off.
Email doesn't match existing supporter	For Blackbaud/Mailchimp syncs: a minimal People record is created (email only). No data is attributed to the wrong person.
Airtable API unavailable	All write operations fail atomically. No partial record creation. Dashboard shows cached data until the API recovers.
Environics returns no data for postal code	Enrichment is skipped for that supporter. No placeholder or guessed values are written.

---

MCP Failure Behavior

Scenario	System Response
Invalid access key or token	Rejected: the consent page refuses an invalid access key, and an invalid, expired or wrong-audience token receives HTTP 401 invalid_token. No data returned. Event logged.
Unauthenticated call to the MCP server	HTTP 401 with a WWW-Authenticate challenge pointing to the sign-in discovery document. No tools and no data. Only the public /health endpoint responds without credentials.
Airtable unavailable during MCP query	Error returned to the AI assistant. No cached or stale data served as a fallback.
LLM provider unavailable	MCP server continues to respond (it doesn't depend on the LLM). The AI assistant may not be able to process the response, but the MCP server's behavior is unchanged.

---

Dashboard Failure Behavior

Scenario	System Response
Backend API unavailable	Dashboard shows loading states, then error messages. No stale data presented as current.
User session expired	Automatic redirect to login. No cached session reuse.
Airtable returns partial data	Dashboard renders what it has. Missing sections show clear empty states, not zeros or placeholders.

---

Incident Response

In the event of a security incident affecting your organization's data:

Phase	Action	Timeline
Detection	Provider logs (Vercel, Axiom, Clerk, Nango, Doppler) and reported issues. Automated alerting is on the security roadmap.	—
Containment	Revoke affected credentials, disable compromised integrations, isolate affected org scope.	Within 4 hours of detection
Notification	Notify affected organization's designated security contact with incident summary, scope, and affected data categories.	Within 72 hours of confirmed breach involving customer data
Investigation	Root cause analysis, review of audit logs, assessment of data exposure scope.	Within 7 days
Remediation	Credential rotation, configuration hardening, process update. Post-incident report provided to affected organization.	Within 14 days

AI4Love will cooperate fully with your organization's own incident response procedures and any regulatory notification requirements.

---

Recovery Guarantees

Component	Recovery Path
Source data (Blackbaud, Mailchimp, Environics)	Never modified by AI4Love under any failure mode — these remain your system of record regardless of what happens on our side.
Working base	If lost, source-derived records regenerate from source systems, and insights regenerate through subsequent agent runs. Airtable's platform-level redundancy (Trash, Base Snapshots) provides recovery in the meantime.
Organization mirror (planned)	Not yet in production. Once provisioned for your organization, the mirror would hold an independent copy of people, activity, and insight records in your own workspace, recoverable through that platform's native tools. Until then, a full export of the working base is available on request at any time.
OAuth connections	Re-established by staff through the Integrations dashboard. No AI4Love intervention required.
Application state	Stateless. Vercel redeploys from git. No persistent state to corrupt or lose.