clienthub/.planning/research/PITFALLS_SUMMARY.md

# Pitfalls Research Summary: Business Operations Suite (v2.0)

**Project:** ClientHub v2.0 — Adding Business Operations Suite to Production  
**Research Date:** 2026-06-10  
**Confidence:** HIGH (domain-specific research backed by schema review + security analysis)

---

## Overview

Adding a complex Business Operations Suite (catalog consolidation + offer builder + public quotes + CRM automation) to a production app with real client data creates integration risks not present when building from scratch. This research documents the most dangerous pitfalls (rewrites/data loss) and operational friction points, with concrete prevention strategies for each.

**Key insight:** Most pitfalls cluster in four areas of high integration complexity:
1. **Schema consolidation** — breaking existing quote_items referential integrity
2. **Template-to-instance copy semantics** — offer phases mutating templates
3. **Optimistic UI + concurrent edits** — sort order conflicts
4. **Multi-step workflow atomicity** — CRM automation creating duplicates or partial state

---

## Critical Pitfalls (Require Design Changes Before Code)

### Pitfall 1: Catalog Consolidation Breaks Quotes (Data Loss Risk)

**Severity:** CRITICAL — Can orphan quote_items, break billing calculations

**Problem:**
Two parallel service tables (`service_catalog` for costs, `offer_services` for marketing pricing) need consolidation. Naive merge breaks `quote_items` referential integrity or loses price history.

**Prevention Strategy:**
- **Expand-Contract migration:** Add new unified `services` table alongside old ones, backfill in phases, only drop old tables after validation
- **Immutable snapshot:** Store `quote_snapshot: JSONB` in project_offers (freeze prices at time of win)
- **Referential integrity test:** Query for orphaned quote_items before/after migration
- **Backward compatibility:** Old quotes read from old tables, new quotes use new table, transition over 2-3 weeks

**Phase:** 7 (Catalog & Offers) — Plan migration design BEFORE any code changes

**Validation checklist:**
- [ ] Migration mapping layer documented (catalog ↔ services ↔ offer_services)
- [ ] Backfill strategy written (small batches, checksum validation)
- [ ] Dual-write period defined (how long to run old+new in parallel)
- [ ] Referential integrity test query exists

---

### Pitfall 2: Offer Template Mutation on Copy (Data Corruption Risk)

**Severity:** CRITICAL — Templates silently mutate when project phases edited

**Problem:**
Copying offer phases to project via shallow copy = shared references. Admin edits project phase, template phase mutates too. Next deal uses corrupted template. Also: partial copies on retry create duplicate phases.

**Prevention Strategy:**
- **Deep copy with atomic transaction:** Use `db.transaction()` for entire copy operation (all-or-nothing); copy at database level, not in JS
- **Immutable template flag:** `offer_micros.is_template = true`, prevent UPDATE on templates
- **Idempotency key:** "Win" request includes idempotency key; retry returns same project (no duplicates)
- **Explicit hierarchy mapping:** Document what offer_micro → project_phase, what offer_service → project_task/deliverable

**Phase:** 7 (Offers) for design, 9 (CRM) for "Win" automation

**Validation checklist:**
- [ ] Hierarchy mapping documented (offer ↔ project structures)
- [ ] Atomic copy function implemented (single transaction, all-or-nothing)
- [ ] Idempotency key added to CRM leads
- [ ] Integration tests verify copy structure consistency across multiple instances

---

### Pitfall 3: Public Quote Token Leakage (Security Risk)

**Severity:** CRITICAL — Token enumeration exposes all pricing

**Problem:**
Public quote page with nanoid 21-char token can be brute-forced or enumerated. Attacker builds pricing database, breaches commercial confidentiality.

**Prevention Strategy:**
- **Longer token:** nanoid(32) instead of 21 (~190 bits vs ~122 bits entropy)
- **Access control:** Require both token + email (validate recipient match)
- **Expiration:** token_expires_at (default 7 days)
- **Rate limiting:** Max 3 views/token/minute (blocks enumeration)
- **Never expose quote_items:** Public API response excludes line items, shows TOTAL PRICE ONLY
- **Activation state:** Token valid only AFTER admin sends it (not auto-generated on create)

**Phase:** 8 (Public Quote Pages) — Implement security controls IN PARALLEL with feature

**Validation checklist:**
- [ ] Token length ≥ 32 chars, rate limiting ≥ 3/min
- [ ] Email validation on public page (token + email both required)
- [ ] Expiration enforced (test expired token returns 401)
- [ ] Code audit: quote_items never in public API response
- [ ] Brute-force test: verify rate limit activates

---

## Moderate Pitfalls (Major Refactoring / Data Inconsistency)

### Pitfall 4: Drag-and-Drop Sort Order Race Condition

**Severity:** MODERATE — Phases display in wrong order, user frustration

**Problem:**
Concurrent drag-and-drop edits in multiple tabs cause sort_order conflicts. Last write wins, earlier update is lost. No version conflict detection.

**Prevention Strategy:**
- **Optimistic locking:** Add `version` field to offer_phases, update only if version matches (return 409 Conflict if mismatch)
- **Server-side recomputation:** Don't trust client's sort_order, recompute all orders atomically based on actual position
- **Optimistic UI with reconciliation:** Update frontend instantly, reconcile with server result (409 = refresh)

**Phase:** 7 (Offer Builder — Drag & Drop)

**Validation checklist:**
- [ ] `version` field added to offer_phases
- [ ] Update mutation includes version check
- [ ] Server recomputes all sort_orders atomically
- [ ] Concurrent edit test exists (open 2 tabs, drag in both)

---

### Pitfall 5: CRM "Win" Double-Click Creates Duplicate Clients

**Severity:** MODERATE — Duplicate clients, broken billing, audit confusion

**Problem:**
Multi-step "Win" automation (create client → project → phases → payments) has no idempotency. Double-click creates two clients, two projects. Network timeouts are invisible to user.

**Prevention Strategy:**
- **Idempotency key:** Every lead has unique idempotency_key; "Win" request checks if key already processed (if yes, return existing client_id)
- **Atomic transaction:** All steps (client, project, phases, payments) in single transaction — all succeed or all rollback
- **UI feedback:** Button disabled until success (prevents accidental double-click)
- **Browser persistence:** Store idempotency_key in localStorage, preserve across refresh

**Phase:** 9 (CRM Won Automation)

**Validation checklist:**
- [ ] CRM leads table has idempotency_key column (unique)
- [ ] "Win" mutation checks for existing key before creating
- [ ] All substeps in single db.transaction()
- [ ] Button disabled until response received
- [ ] Idempotency key stored in localStorage
- [ ] Test: double-click "Win", verify only 1 client created

---

### Pitfall 6: Offer Phase Copy Misses Tasks/Deliverables

**Severity:** MODERATE — Client dashboard shows incomplete phases (no tasks to approve)

**Problem:**
Copy offer → project phases but forget to copy tasks or deliverables. Client sees phase with no work items.

**Prevention Strategy:**
- **Explicit mapping:** Document offer_micro → project_phase, offer_service → project_task/deliverable mapping
- **Full-tree copy function:** Copy phases, then tasks for each phase, then deliverables for each task (all in one transaction)
- **Validation test:** Assert phase_count, task_count, deliverable_count match expected values after copy

**Phase:** 7 (Offers structure design), 9 (Copy implementation)

**Validation checklist:**
- [ ] Mapping documented (offer vs project hierarchies)
- [ ] Copy function handles entire tree (phases → tasks → deliverables)
- [ ] Integration test: copy offer, verify counts match expected
- [ ] Query test: check no deliverables are orphaned (task_id IS NULL)

---

## Minor Pitfalls (Operational Friction)

### Pitfall 7: Offer State Not Synced Between Tabs

**Problem:** Admin edits offer in tab 1, tab 2 doesn't know, last save wins (first admin's work lost)

**Prevention:** Add `version` field, return 409 on version mismatch, show "Offer changed, reload?"

---

### Pitfall 8: CRM Schema Backward Incompatibility

**Problem:** Add new fields (budget, source) as NULL, code assumes populated, crashes

**Prevention:** Expand-Contract pattern — add as NULL, handle NULL defensively in code, backfill, then add NOT NULL constraint

---

### Pitfall 9: CRM Scope Creep (Feature Bloat)

**Problem:** "Just add email templates" → 30 hours. "Add calls" → 40 hours. CRM never ships.

**Prevention:** Explicit Must/Should/Could/Won't scope document. Must-haves only for v2.0: lead pipeline, quote attachment, auto-onboarding. Defer rest.

---

## Key Design Decisions for Phase 7–9

| Decision | Why | Validation |
|----------|-----|-----------|
| Expand-Contract migration for catalog | Zero-downtime, safe rollback, no data loss | Dry-run on prod backup before go-live |
| Deep copy + atomic transaction for offer phases | Prevents template mutation, partial copies | Integration test, structure consistency query |
| Idempotency key on "Win" action | Safe retry, prevents duplicate clients | Double-click test, query for duplicates |
| Public quote token security layers (length, expiration, email, rate limit) | Blocks enumeration, limits leakage | Brute-force test, code audit for quote_items |
| Offer snapshot (immutable JSONB) | Preserves what was promised vs. what's executing | Store at win time, display in client dashboard |
| Version field on offer_phases + optimistic locking | Detects concurrent edit conflicts | Concurrent drag test, 409 Conflict handling |

---

## Roadmap Implications

**Phase 7 (Catalog & Offers):**
- Design catalog consolidation migration before any code
- Implement drag-drop with version fields from start
- Document offer/project hierarchy mapping explicitly
- Write copy function atomically (no partial copies)

**Phase 8 (Public Quote Pages):**
- Implement token security controls in parallel (not after)
- Rate limiting + email validation + expiration
- Code audit: quote_items never in response
- Test brute-force + enumeration

**Phase 9 (CRM — Won Automation):**
- Add idempotency_key to leads table
- Implement atomic "Win" transaction
- Store offer snapshot at win time
- Define Must/Should/Won't scope before design
- Integration tests for payment consistency

**Phase 10+ (Future):**
- Defer: email templates, call logging, team features, integrations
- Solo consultant doesn't need multi-user or GoHighLevel-scale features

---

## Testing Strategy for Confidence

### Phase 7 (Catalog & Offers)
1. Dry-run consolidation migration on production database backup
2. Verify all quote_items still resolve to services (no orphans)
3. Regenerate quotes from old projects, compare prices to originals
4. Concurrent drag-drop test: open offer in 2 tabs, drag in both, verify final state
5. Copy offer phases to test project, verify phase/task/deliverable counts

### Phase 8 (Public Quote Pages)
1. Brute-force token space: 1000 guesses/sec, verify rate limit activates
2. Enumerate tokens: generate 100 quotes, try to access one meant for different email, verify 401
3. Expiration test: set token_expires_at to past, verify access fails
4. Code audit: search response JSON for price, quote_items, per_service_price (should be zero matches)
5. Authorized access test: correct email + token, verify success

### Phase 9 (CRM — Won Automation)
1. Double-click "Win" button, verify only 1 client created
2. Network failure during "Win": simulate timeout after client creation, retry, verify same project returned
3. Partial failure: mock project creation failure, verify client not created (rollback)
4. Payment consistency: verify 2–4 payments created based on payment plan
5. Idempotency: call "Win" 10x with same idempotency_key, verify only 1 project, 1 set of payments

---

## Confidence Assessment

| Area | Level | Reason |
|------|-------|--------|
| Catalog consolidation risks | HIGH | Schema review shows existing quote_items dependencies; migration pattern verified via Drizzle docs |
| Offer copy semantics | HIGH | JavaScript shallow copy / deep copy distinction well-understood; transaction guarantees verified |
| Token security | HIGH | Brute-force math straightforward; nanoid 32 vs 21 entropy difference verified |
| CRM idempotency | HIGH | Idempotency pattern researched across multiple sources; double-click problem well-documented |
| Scope creep prevention | MEDIUM | CRM feature bloat is common, but solo consultant constraint makes Must/Should/Won't achievable |
| Concurrent edit handling | MEDIUM | Optimistic locking pattern standard, but requires careful implementation in Next.js + Drizzle |

---

## Next Steps for Phase Planning

1. **Before Phase 7 design starts:**
   - Finalize catalog consolidation migration plan (Expand-Contract timeline)
   - Document offer/project hierarchy mapping (1 page, code-level documentation)
   - Design offer copy function signature and transaction strategy

2. **Before Phase 7 code starts:**
   - Implement drag-drop versioning (version field + optimistic locking)
   - Write integration tests for offer phase copy

3. **Before Phase 8 design starts:**
   - Token security requirements: length (32), expiration (7d), email validation, rate limit (3/min)
   - API response schema: exclude quote_items, include total_price only

4. **Before Phase 9 design starts:**
   - Finalize Must/Should/Won't scope (document with user/consultant)
   - Design "Win" workflow: idempotency key + atomic transaction
   - Define payment plan algorithm (2–4 payments based on offer tier)

---

## References for Implementation

- **Drizzle migrations:** https://dev.to/whoffagents/zero-downtime-postgres-migrations-with-drizzle-orm-22ga
- **Idempotency pattern:** https://codefarm0.medium.com/the-double-click-problem-how-idempotency-saved-our-checkout-system-a704be65d207
- **Optimistic updates:** https://www.nirtamir.com/articles/optimistic-updates-state-vs-render/
- **Schema evolution:** https://www.dataexpert.io/blog/backward-compatibility-schema-evolution-guide
- **Token security:** https://workos.com/blog/oauth-common-attacks-and-how-to-prevent-them/

---

**Document location:** `/Users/simonecavalli/Vault/IAMCAVALLI/.planning/research/PITFALLS_V2.md` (detailed pitfall reference)