clienthub/.planning/research/PITFALLS_SUMMARY.md

Pitfalls Research Summary: Business Operations Suite (v2.0)

Project: ClientHub v2.0 — Adding Business Operations Suite to Production
Research Date: 2026-06-10
Confidence: HIGH (domain-specific research backed by schema review + security analysis)

---

Overview

Adding a complex Business Operations Suite (catalog consolidation + offer builder + public quotes + CRM automation) to a production app with real client data creates integration risks not present when building from scratch. This research documents the most dangerous pitfalls (rewrites/data loss) and operational friction points, with concrete prevention strategies for each.

Key insight: Most pitfalls cluster in four areas of high integration complexity:

1. Schema consolidation — breaking existing quote_items referential integrity
2. Template-to-instance copy semantics — offer phases mutating templates
3. Optimistic UI + concurrent edits — sort order conflicts
4. Multi-step workflow atomicity — CRM automation creating duplicates or partial state

---

Critical Pitfalls (Require Design Changes Before Code)

Pitfall 1: Catalog Consolidation Breaks Quotes (Data Loss Risk)

Severity: CRITICAL — Can orphan quote_items, break billing calculations

Problem: Two parallel service tables (service_catalog for costs, offer_services for marketing pricing) need consolidation. Naive merge breaks quote_items referential integrity or loses price history.

Prevention Strategy:

• Expand-Contract migration: Add new unified services table alongside old ones, backfill in phases, only drop old tables after validation
• Immutable snapshot: Store quote_snapshot: JSONB in project_offers (freeze prices at time of win)
• Referential integrity test: Query for orphaned quote_items before/after migration
• Backward compatibility: Old quotes read from old tables, new quotes use new table, transition over 2-3 weeks

Phase: 7 (Catalog & Offers) — Plan migration design BEFORE any code changes

Validation checklist:

• Migration mapping layer documented (catalog ↔ services ↔ offer_services)
• Backfill strategy written (small batches, checksum validation)
• Dual-write period defined (how long to run old+new in parallel)
• Referential integrity test query exists

---

Pitfall 2: Offer Template Mutation on Copy (Data Corruption Risk)

Severity: CRITICAL — Templates silently mutate when project phases edited

Problem: Copying offer phases to project via shallow copy = shared references. Admin edits project phase, template phase mutates too. Next deal uses corrupted template. Also: partial copies on retry create duplicate phases.

Prevention Strategy:

• Deep copy with atomic transaction: Use db.transaction() for entire copy operation (all-or-nothing); copy at database level, not in JS
• Immutable template flag: offer_micros.is_template = true, prevent UPDATE on templates
• Idempotency key: "Win" request includes idempotency key; retry returns same project (no duplicates)
• Explicit hierarchy mapping: Document what offer_micro → project_phase, what offer_service → project_task/deliverable

Phase: 7 (Offers) for design, 9 (CRM) for "Win" automation

Validation checklist:

• Hierarchy mapping documented (offer ↔ project structures)
• Atomic copy function implemented (single transaction, all-or-nothing)
• Idempotency key added to CRM leads
• Integration tests verify copy structure consistency across multiple instances

---

Pitfall 3: Public Quote Token Leakage (Security Risk)

Severity: CRITICAL — Token enumeration exposes all pricing

Problem: Public quote page with nanoid 21-char token can be brute-forced or enumerated. Attacker builds pricing database, breaches commercial confidentiality.

Prevention Strategy:

• Longer token: nanoid(32) instead of 21 (~190 bits vs ~122 bits entropy)
• Access control: Require both token + email (validate recipient match)
• Expiration: token_expires_at (default 7 days)
• Rate limiting: Max 3 views/token/minute (blocks enumeration)
• Never expose quote_items: Public API response excludes line items, shows TOTAL PRICE ONLY
• Activation state: Token valid only AFTER admin sends it (not auto-generated on create)

Phase: 8 (Public Quote Pages) — Implement security controls IN PARALLEL with feature

Validation checklist:

• Token length ≥ 32 chars, rate limiting ≥ 3/min
• Email validation on public page (token + email both required)
• Expiration enforced (test expired token returns 401)
• Code audit: quote_items never in public API response
• Brute-force test: verify rate limit activates

---

Moderate Pitfalls (Major Refactoring / Data Inconsistency)

Pitfall 4: Drag-and-Drop Sort Order Race Condition

Severity: MODERATE — Phases display in wrong order, user frustration

Problem: Concurrent drag-and-drop edits in multiple tabs cause sort_order conflicts. Last write wins, earlier update is lost. No version conflict detection.

Prevention Strategy:

• Optimistic locking: Add version field to offer_phases, update only if version matches (return 409 Conflict if mismatch)
• Server-side recomputation: Don't trust client's sort_order, recompute all orders atomically based on actual position
• Optimistic UI with reconciliation: Update frontend instantly, reconcile with server result (409 = refresh)

Phase: 7 (Offer Builder — Drag & Drop)

Validation checklist:

• version field added to offer_phases
• Update mutation includes version check
• Server recomputes all sort_orders atomically
• Concurrent edit test exists (open 2 tabs, drag in both)

---

Pitfall 5: CRM "Win" Double-Click Creates Duplicate Clients

Severity: MODERATE — Duplicate clients, broken billing, audit confusion

Problem: Multi-step "Win" automation (create client → project → phases → payments) has no idempotency. Double-click creates two clients, two projects. Network timeouts are invisible to user.

Prevention Strategy:

• Idempotency key: Every lead has unique idempotency_key; "Win" request checks if key already processed (if yes, return existing client_id)
• Atomic transaction: All steps (client, project, phases, payments) in single transaction — all succeed or all rollback
• UI feedback: Button disabled until success (prevents accidental double-click)
• Browser persistence: Store idempotency_key in localStorage, preserve across refresh

Phase: 9 (CRM Won Automation)

Validation checklist:

• CRM leads table has idempotency_key column (unique)
• "Win" mutation checks for existing key before creating
• All substeps in single db.transaction()
• Button disabled until response received
• Idempotency key stored in localStorage
• Test: double-click "Win", verify only 1 client created

---

Pitfall 6: Offer Phase Copy Misses Tasks/Deliverables

Severity: MODERATE — Client dashboard shows incomplete phases (no tasks to approve)

Problem: Copy offer → project phases but forget to copy tasks or deliverables. Client sees phase with no work items.

Prevention Strategy:

• Explicit mapping: Document offer_micro → project_phase, offer_service → project_task/deliverable mapping
• Full-tree copy function: Copy phases, then tasks for each phase, then deliverables for each task (all in one transaction)
• Validation test: Assert phase_count, task_count, deliverable_count match expected values after copy

Phase: 7 (Offers structure design), 9 (Copy implementation)

Validation checklist:

• Mapping documented (offer vs project hierarchies)
• Copy function handles entire tree (phases → tasks → deliverables)
• Integration test: copy offer, verify counts match expected
• Query test: check no deliverables are orphaned (task_id IS NULL)

---

Minor Pitfalls (Operational Friction)

Pitfall 7: Offer State Not Synced Between Tabs

Problem: Admin edits offer in tab 1, tab 2 doesn't know, last save wins (first admin's work lost)

Prevention: Add version field, return 409 on version mismatch, show "Offer changed, reload?"

---

Pitfall 8: CRM Schema Backward Incompatibility

Problem: Add new fields (budget, source) as NULL, code assumes populated, crashes

Prevention: Expand-Contract pattern — add as NULL, handle NULL defensively in code, backfill, then add NOT NULL constraint

---

Pitfall 9: CRM Scope Creep (Feature Bloat)

Problem: "Just add email templates" → 30 hours. "Add calls" → 40 hours. CRM never ships.

Prevention: Explicit Must/Should/Could/Won't scope document. Must-haves only for v2.0: lead pipeline, quote attachment, auto-onboarding. Defer rest.

---

Key Design Decisions for Phase 7–9

Decision	Why	Validation
Expand-Contract migration for catalog	Zero-downtime, safe rollback, no data loss	Dry-run on prod backup before go-live
Deep copy + atomic transaction for offer phases	Prevents template mutation, partial copies	Integration test, structure consistency query
Idempotency key on "Win" action	Safe retry, prevents duplicate clients	Double-click test, query for duplicates
Public quote token security layers (length, expiration, email, rate limit)	Blocks enumeration, limits leakage	Brute-force test, code audit for quote_items
Offer snapshot (immutable JSONB)	Preserves what was promised vs. what's executing	Store at win time, display in client dashboard
Version field on offer_phases + optimistic locking	Detects concurrent edit conflicts	Concurrent drag test, 409 Conflict handling

---

Roadmap Implications

Phase 7 (Catalog & Offers):

• Design catalog consolidation migration before any code
• Implement drag-drop with version fields from start
• Document offer/project hierarchy mapping explicitly
• Write copy function atomically (no partial copies)

Phase 8 (Public Quote Pages):

• Implement token security controls in parallel (not after)
• Rate limiting + email validation + expiration
• Code audit: quote_items never in response
• Test brute-force + enumeration

Phase 9 (CRM — Won Automation):

• Add idempotency_key to leads table
• Implement atomic "Win" transaction
• Store offer snapshot at win time
• Define Must/Should/Won't scope before design
• Integration tests for payment consistency

Phase 10+ (Future):

• Defer: email templates, call logging, team features, integrations
• Solo consultant doesn't need multi-user or GoHighLevel-scale features

---

Testing Strategy for Confidence

Phase 7 (Catalog & Offers)

1. Dry-run consolidation migration on production database backup
2. Verify all quote_items still resolve to services (no orphans)
3. Regenerate quotes from old projects, compare prices to originals
4. Concurrent drag-drop test: open offer in 2 tabs, drag in both, verify final state
5. Copy offer phases to test project, verify phase/task/deliverable counts

Phase 8 (Public Quote Pages)

1. Brute-force token space: 1000 guesses/sec, verify rate limit activates
2. Enumerate tokens: generate 100 quotes, try to access one meant for different email, verify 401
3. Expiration test: set token_expires_at to past, verify access fails
4. Code audit: search response JSON for price, quote_items, per_service_price (should be zero matches)
5. Authorized access test: correct email + token, verify success

Phase 9 (CRM — Won Automation)

1. Double-click "Win" button, verify only 1 client created
2. Network failure during "Win": simulate timeout after client creation, retry, verify same project returned
3. Partial failure: mock project creation failure, verify client not created (rollback)
4. Payment consistency: verify 2–4 payments created based on payment plan
5. Idempotency: call "Win" 10x with same idempotency_key, verify only 1 project, 1 set of payments

---

Confidence Assessment

Area	Level	Reason
Catalog consolidation risks	HIGH	Schema review shows existing quote_items dependencies; migration pattern verified via Drizzle docs
Offer copy semantics	HIGH	JavaScript shallow copy / deep copy distinction well-understood; transaction guarantees verified
Token security	HIGH	Brute-force math straightforward; nanoid 32 vs 21 entropy difference verified
CRM idempotency	HIGH	Idempotency pattern researched across multiple sources; double-click problem well-documented
Scope creep prevention	MEDIUM	CRM feature bloat is common, but solo consultant constraint makes Must/Should/Won't achievable
Concurrent edit handling	MEDIUM	Optimistic locking pattern standard, but requires careful implementation in Next.js + Drizzle

---

Next Steps for Phase Planning

1. Before Phase 7 design starts:

   • Finalize catalog consolidation migration plan (Expand-Contract timeline)
   • Document offer/project hierarchy mapping (1 page, code-level documentation)
   • Design offer copy function signature and transaction strategy

2. Before Phase 7 code starts:

   • Implement drag-drop versioning (version field + optimistic locking)
   • Write integration tests for offer phase copy

3. Before Phase 8 design starts:

   • Token security requirements: length (32), expiration (7d), email validation, rate limit (3/min)
   • API response schema: exclude quote_items, include total_price only

4. Before Phase 9 design starts:

   • Finalize Must/Should/Won't scope (document with user/consultant)
   • Design "Win" workflow: idempotency key + atomic transaction
   • Define payment plan algorithm (2–4 payments based on offer tier)

---

References for Implementation

• Drizzle migrations: https://dev.to/whoffagents/zero-downtime-postgres-migrations-with-drizzle-orm-22ga
• Idempotency pattern: https://codefarm0.medium.com/the-double-click-problem-how-idempotency-saved-our-checkout-system-a704be65d207
• Optimistic updates: https://www.nirtamir.com/articles/optimistic-updates-state-vs-render/
• Schema evolution: https://www.dataexpert.io/blog/backward-compatibility-schema-evolution-guide
• Token security: https://workos.com/blog/oauth-common-attacks-and-how-to-prevent-them/

---

Document location: /Users/simonecavalli/Vault/IAMCAVALLI/.planning/research/PITFALLS_V2.md (detailed pitfall reference)