docs(v2.0): finalize roadmap + requirements — phases 7-11 approved for planning

- Upgrade PROJECT.md: v2.0 Business Operations Suite (unified catalog, offers with phases, public quotes, CRM pipeline, auto-provisioning)
- Create REQUIREMENTS.md sections: CAT-U, OFFER-B, QUOTE, CRM, WIN requirements for phases 7-11
- Extend ROADMAP.md: 5-phase structure (7-11), 14-19 days total, risk/flag registry
- Archive v1.0 milestone (phases 1-6, complete)
- Switch STATE.md to v2.0, status=planning
- Create research documents: STACK.md, FEATURES_v2.0.md, ARCHITECTURE.md, PITFALLS.md, SUMMARY.md

Ready for: /gsd-plan-phase 7

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-06-11 06:01:59 +02:00
parent 6239eed6e6
commit 3b1f49d059
8 changed files with 2085 additions and 84 deletions
+210
View File
@@ -0,0 +1,210 @@
# Pitfalls Checklist: Business Operations Suite v2.0
Quick reference for preventing the 9 major pitfalls during Phase 79 implementation.
---
## Critical Pitfalls (Before Code)
### 1. Catalog Consolidation Breaks Quotes
- [ ] Migration strategy documented (Expand-Contract, not drop)
- [ ] Mapping layer between old/new tables designed
- [ ] Backfill script tested on production backup
- [ ] Referential integrity test query written
- [ ] Dual-write period defined (2-3 weeks)
- [ ] Rollback plan documented (drop new table, revert code)
### 2. Offer Template Mutation
- [ ] Offer/project hierarchy mapping documented (1 page)
- [ ] Copy function uses `db.transaction()` (all-or-nothing)
- [ ] Uses `structuredClone()` for deep copy, not `{...spread}`
- [ ] Idempotency key added to CRM leads table
- [ ] Integration test: copy offer, verify structure consistent across instances
- [ ] Test: edit project phase, verify offer template unchanged
### 3. Public Quote Token Leakage
- [ ] Token length ≥ 32 chars (nanoid(32) or nanoid(64))
- [ ] Expiration field added (default 7 days)
- [ ] Email validation on public page (token + email required)
- [ ] Rate limiting implemented (max 3 views/token/minute)
- [ ] Code audit: quote_items NOT in public API response
- [ ] Test: brute-force 1000 guesses, verify rate limit activates
- [ ] Test: enumerate tokens with different emails, verify 401
---
## Moderate Pitfalls (During Implementation)
### 4. Drag-and-Drop Race Condition
- [ ] `version` field added to offer_phases
- [ ] Update mutation checks version (returns 409 if mismatch)
- [ ] Server recomputes all sort_orders atomically
- [ ] Frontend handles 409 Conflict (refresh UI)
- [ ] Test: concurrent drag in 2 tabs, verify final state correct
### 5. CRM "Win" Double-Click
- [ ] Idempotency key added to crm_leads (unique)
- [ ] "Win" mutation checks if key already processed
- [ ] All sub-steps in single `db.transaction()`
- [ ] Button disabled until success
- [ ] Idempotency key stored in localStorage
- [ ] Test: double-click "Win", verify only 1 client created
### 6. Offer Copy Missing Tasks/Deliverables
- [ ] Mapping documented: offer_micro → project_phase → project_task → deliverable
- [ ] Copy function recursive (phases → tasks → deliverables)
- [ ] Validation test: assert counts match expected
- [ ] Integration test exists (copy, verify structure)
- [ ] Query test: no orphaned deliverables (task_id IS NULL)
---
## Minor Pitfalls (Polish)
### 7. Offer State Not Synced Between Tabs
- [ ] `version` field on offer_micros
- [ ] Update returns 409 on version mismatch
- [ ] UI shows "Offer changed, reload?" dialog
### 8. CRM Schema Backward Incompatibility
- [ ] New columns added as NULLABLE
- [ ] Code handles NULL defensively (e.g., `lead.budget ?? 0`)
- [ ] Backfill script written (sets defaults for existing rows)
- [ ] NOT NULL constraint added only after backfill + validation
- [ ] Test: production data copied to test db, code runs without errors
### 9. CRM Scope Creep
- [ ] Must-have features defined for v2.0 (lead pipeline, quote, auto-onboard)
- [ ] Should-have features listed for v2.1+ (email, calls, source)
- [ ] Won't-have features documented (team, multi-user, integrations)
- [ ] Scope document signed off by product
- [ ] Success metric defined: "Can go from lead to won project in <30 min"
---
## Phase 7 Checklist (Catalog & Offers)
**Before code:**
- [ ] Catalog consolidation migration plan reviewed
- [ ] Offer/project hierarchy documented
- [ ] Drag-drop version/locking strategy designed
**During implementation:**
- [ ] Migration backfill tested on production backup
- [ ] Drag-drop version field added
- [ ] Offer phase copy function atomic (transaction)
- [ ] Idempotency key on CRM leads
- [ ] Integration tests for copy consistency
**Before go-live:**
- [ ] Dry-run migration on production DB
- [ ] Verify referential integrity (no orphaned quote_items)
- [ ] Concurrent drag-drop test passes
- [ ] Offer copy count validation passes
---
## Phase 8 Checklist (Public Quote Pages)
**Before code:**
- [ ] Token security requirements finalized
- [ ] API response schema designed (no quote_items)
- [ ] Rate limiting strategy defined
**During implementation:**
- [ ] Token generation (nanoid(32))
- [ ] Expiration + email validation
- [ ] Rate limiting middleware
- [ ] Code audit for quote_items exposure
- [ ] Brute-force test (>1000 guesses/sec)
**Before go-live:**
- [ ] Brute-force test activates rate limit
- [ ] Enumeration test returns 401 for wrong email
- [ ] Expiration test returns 401 after expiration
- [ ] Code review: quote_items not in response
---
## Phase 9 Checklist (CRM Won Automation)
**Before code:**
- [ ] Scope document (Must/Should/Could/Won't) reviewed
- [ ] Idempotency strategy designed
- [ ] Payment plan algorithm defined
- [ ] Offer snapshot structure (JSONB) designed
**During implementation:**
- [ ] Idempotency key on crm_leads
- [ ] "Win" mutation atomic (client + project + phases + payments)
- [ ] Offer snapshot stored on win
- [ ] Button disabled until success
- [ ] Payment consistency tests
**Before go-live:**
- [ ] Double-click test: only 1 client created
- [ ] Network timeout test: retry returns same project
- [ ] Partial failure test: rollback on error
- [ ] Payment consistency test: correct number created
- [ ] Idempotency test: 10x "Win" with same key = 1 client
---
## Key Questions Before Each Phase
**Phase 7:**
- ✓ Is catalog consolidation migration safe (Expand-Contract, dual-write)?
- ✓ Does offer/project hierarchy mapping make sense (1 micro = 1 phase)?
- ✓ Is copy function atomic (all-or-nothing)?
- ✓ Do we have integration tests for offer phase structure consistency?
**Phase 8:**
- ✓ Is token length ≥ 32 chars and properly random?
- ✓ Do we validate both token AND email on public page?
- ✓ Is rate limiting active (max 3 views/token/min)?
- ✓ Can we prove quote_items never appears in public response?
**Phase 9:**
- ✓ Is "Win" action atomic (transaction)?
- ✓ Does idempotency key prevent duplicates on retry?
- ✓ Is button disabled until success?
- ✓ Have we defined Must/Should/Won't scope with user?
---
## Testing Validation Matrix
| Pitfall | Test Case | Expected Outcome | Status |
|---------|-----------|------------------|--------|
| 1. Catalog consolidation | Backfill migration on prod backup | Zero orphaned quote_items | [ ] |
| 2. Offer template mutation | Edit project phase, check offer unchanged | Offer template immutable | [ ] |
| 3. Token leakage | Brute-force 1000 guesses/sec | Rate limit activates (429) | [ ] |
| 3. Token leakage | Enumerate with wrong email | Returns 401 Unauthorized | [ ] |
| 4. Drag-drop race | Concurrent drag in 2 tabs | Final sort_order is coherent | [ ] |
| 5. Double-click clients | Click "Win" twice | Only 1 client created | [ ] |
| 5. Double-click clients | Network timeout + retry | Same project returned | [ ] |
| 6. Missing deliverables | Copy offer to project | All tasks and deliverables copied | [ ] |
| 8. Schema backward compat | Copy production data to test DB | No crashes, NULL handled | [ ] |
| 9. Payment consistency | "Win" with partial failure | Rollback, no orphaned payments | [ ] |
---
## Red Flags During Development
🚨 **Stop and review if:**
- Offer/project hierarchy mapping is unclear (should be 1 page, explicit)
- Drag-drop sort_order updates without version check
- "Win" mutation has any non-transactional steps (client created outside txn)
- Copy function uses shallow spread (`{...obj}`) instead of `structuredClone()`
- Token length < 32 chars
- Public API response includes quote_items or per_service_price
- CRM leads table has no idempotency_key column
- Schema changes add NOT NULL without backfill test
- "Win" button not disabled until response received
- No rate limiting on public quote page
---
**Owner:** Development team
**Review:** Before each phase kickoff
**Update:** As testing results come in
+302
View File
@@ -0,0 +1,302 @@
# Pitfalls Research Summary: Business Operations Suite (v2.0)
**Project:** ClientHub v2.0 — Adding Business Operations Suite to Production
**Research Date:** 2026-06-10
**Confidence:** HIGH (domain-specific research backed by schema review + security analysis)
---
## Overview
Adding a complex Business Operations Suite (catalog consolidation + offer builder + public quotes + CRM automation) to a production app with real client data creates integration risks not present when building from scratch. This research documents the most dangerous pitfalls (rewrites/data loss) and operational friction points, with concrete prevention strategies for each.
**Key insight:** Most pitfalls cluster in four areas of high integration complexity:
1. **Schema consolidation** — breaking existing quote_items referential integrity
2. **Template-to-instance copy semantics** — offer phases mutating templates
3. **Optimistic UI + concurrent edits** — sort order conflicts
4. **Multi-step workflow atomicity** — CRM automation creating duplicates or partial state
---
## Critical Pitfalls (Require Design Changes Before Code)
### Pitfall 1: Catalog Consolidation Breaks Quotes (Data Loss Risk)
**Severity:** CRITICAL — Can orphan quote_items, break billing calculations
**Problem:**
Two parallel service tables (`service_catalog` for costs, `offer_services` for marketing pricing) need consolidation. Naive merge breaks `quote_items` referential integrity or loses price history.
**Prevention Strategy:**
- **Expand-Contract migration:** Add new unified `services` table alongside old ones, backfill in phases, only drop old tables after validation
- **Immutable snapshot:** Store `quote_snapshot: JSONB` in project_offers (freeze prices at time of win)
- **Referential integrity test:** Query for orphaned quote_items before/after migration
- **Backward compatibility:** Old quotes read from old tables, new quotes use new table, transition over 2-3 weeks
**Phase:** 7 (Catalog & Offers) — Plan migration design BEFORE any code changes
**Validation checklist:**
- [ ] Migration mapping layer documented (catalog ↔ services ↔ offer_services)
- [ ] Backfill strategy written (small batches, checksum validation)
- [ ] Dual-write period defined (how long to run old+new in parallel)
- [ ] Referential integrity test query exists
---
### Pitfall 2: Offer Template Mutation on Copy (Data Corruption Risk)
**Severity:** CRITICAL — Templates silently mutate when project phases edited
**Problem:**
Copying offer phases to project via shallow copy = shared references. Admin edits project phase, template phase mutates too. Next deal uses corrupted template. Also: partial copies on retry create duplicate phases.
**Prevention Strategy:**
- **Deep copy with atomic transaction:** Use `db.transaction()` for entire copy operation (all-or-nothing); copy at database level, not in JS
- **Immutable template flag:** `offer_micros.is_template = true`, prevent UPDATE on templates
- **Idempotency key:** "Win" request includes idempotency key; retry returns same project (no duplicates)
- **Explicit hierarchy mapping:** Document what offer_micro → project_phase, what offer_service → project_task/deliverable
**Phase:** 7 (Offers) for design, 9 (CRM) for "Win" automation
**Validation checklist:**
- [ ] Hierarchy mapping documented (offer ↔ project structures)
- [ ] Atomic copy function implemented (single transaction, all-or-nothing)
- [ ] Idempotency key added to CRM leads
- [ ] Integration tests verify copy structure consistency across multiple instances
---
### Pitfall 3: Public Quote Token Leakage (Security Risk)
**Severity:** CRITICAL — Token enumeration exposes all pricing
**Problem:**
Public quote page with nanoid 21-char token can be brute-forced or enumerated. Attacker builds pricing database, breaches commercial confidentiality.
**Prevention Strategy:**
- **Longer token:** nanoid(32) instead of 21 (~190 bits vs ~122 bits entropy)
- **Access control:** Require both token + email (validate recipient match)
- **Expiration:** token_expires_at (default 7 days)
- **Rate limiting:** Max 3 views/token/minute (blocks enumeration)
- **Never expose quote_items:** Public API response excludes line items, shows TOTAL PRICE ONLY
- **Activation state:** Token valid only AFTER admin sends it (not auto-generated on create)
**Phase:** 8 (Public Quote Pages) — Implement security controls IN PARALLEL with feature
**Validation checklist:**
- [ ] Token length ≥ 32 chars, rate limiting ≥ 3/min
- [ ] Email validation on public page (token + email both required)
- [ ] Expiration enforced (test expired token returns 401)
- [ ] Code audit: quote_items never in public API response
- [ ] Brute-force test: verify rate limit activates
---
## Moderate Pitfalls (Major Refactoring / Data Inconsistency)
### Pitfall 4: Drag-and-Drop Sort Order Race Condition
**Severity:** MODERATE — Phases display in wrong order, user frustration
**Problem:**
Concurrent drag-and-drop edits in multiple tabs cause sort_order conflicts. Last write wins, earlier update is lost. No version conflict detection.
**Prevention Strategy:**
- **Optimistic locking:** Add `version` field to offer_phases, update only if version matches (return 409 Conflict if mismatch)
- **Server-side recomputation:** Don't trust client's sort_order, recompute all orders atomically based on actual position
- **Optimistic UI with reconciliation:** Update frontend instantly, reconcile with server result (409 = refresh)
**Phase:** 7 (Offer Builder — Drag & Drop)
**Validation checklist:**
- [ ] `version` field added to offer_phases
- [ ] Update mutation includes version check
- [ ] Server recomputes all sort_orders atomically
- [ ] Concurrent edit test exists (open 2 tabs, drag in both)
---
### Pitfall 5: CRM "Win" Double-Click Creates Duplicate Clients
**Severity:** MODERATE — Duplicate clients, broken billing, audit confusion
**Problem:**
Multi-step "Win" automation (create client → project → phases → payments) has no idempotency. Double-click creates two clients, two projects. Network timeouts are invisible to user.
**Prevention Strategy:**
- **Idempotency key:** Every lead has unique idempotency_key; "Win" request checks if key already processed (if yes, return existing client_id)
- **Atomic transaction:** All steps (client, project, phases, payments) in single transaction — all succeed or all rollback
- **UI feedback:** Button disabled until success (prevents accidental double-click)
- **Browser persistence:** Store idempotency_key in localStorage, preserve across refresh
**Phase:** 9 (CRM Won Automation)
**Validation checklist:**
- [ ] CRM leads table has idempotency_key column (unique)
- [ ] "Win" mutation checks for existing key before creating
- [ ] All substeps in single db.transaction()
- [ ] Button disabled until response received
- [ ] Idempotency key stored in localStorage
- [ ] Test: double-click "Win", verify only 1 client created
---
### Pitfall 6: Offer Phase Copy Misses Tasks/Deliverables
**Severity:** MODERATE — Client dashboard shows incomplete phases (no tasks to approve)
**Problem:**
Copy offer → project phases but forget to copy tasks or deliverables. Client sees phase with no work items.
**Prevention Strategy:**
- **Explicit mapping:** Document offer_micro → project_phase, offer_service → project_task/deliverable mapping
- **Full-tree copy function:** Copy phases, then tasks for each phase, then deliverables for each task (all in one transaction)
- **Validation test:** Assert phase_count, task_count, deliverable_count match expected values after copy
**Phase:** 7 (Offers structure design), 9 (Copy implementation)
**Validation checklist:**
- [ ] Mapping documented (offer vs project hierarchies)
- [ ] Copy function handles entire tree (phases → tasks → deliverables)
- [ ] Integration test: copy offer, verify counts match expected
- [ ] Query test: check no deliverables are orphaned (task_id IS NULL)
---
## Minor Pitfalls (Operational Friction)
### Pitfall 7: Offer State Not Synced Between Tabs
**Problem:** Admin edits offer in tab 1, tab 2 doesn't know, last save wins (first admin's work lost)
**Prevention:** Add `version` field, return 409 on version mismatch, show "Offer changed, reload?"
---
### Pitfall 8: CRM Schema Backward Incompatibility
**Problem:** Add new fields (budget, source) as NULL, code assumes populated, crashes
**Prevention:** Expand-Contract pattern — add as NULL, handle NULL defensively in code, backfill, then add NOT NULL constraint
---
### Pitfall 9: CRM Scope Creep (Feature Bloat)
**Problem:** "Just add email templates" → 30 hours. "Add calls" → 40 hours. CRM never ships.
**Prevention:** Explicit Must/Should/Could/Won't scope document. Must-haves only for v2.0: lead pipeline, quote attachment, auto-onboarding. Defer rest.
---
## Key Design Decisions for Phase 79
| Decision | Why | Validation |
|----------|-----|-----------|
| Expand-Contract migration for catalog | Zero-downtime, safe rollback, no data loss | Dry-run on prod backup before go-live |
| Deep copy + atomic transaction for offer phases | Prevents template mutation, partial copies | Integration test, structure consistency query |
| Idempotency key on "Win" action | Safe retry, prevents duplicate clients | Double-click test, query for duplicates |
| Public quote token security layers (length, expiration, email, rate limit) | Blocks enumeration, limits leakage | Brute-force test, code audit for quote_items |
| Offer snapshot (immutable JSONB) | Preserves what was promised vs. what's executing | Store at win time, display in client dashboard |
| Version field on offer_phases + optimistic locking | Detects concurrent edit conflicts | Concurrent drag test, 409 Conflict handling |
---
## Roadmap Implications
**Phase 7 (Catalog & Offers):**
- Design catalog consolidation migration before any code
- Implement drag-drop with version fields from start
- Document offer/project hierarchy mapping explicitly
- Write copy function atomically (no partial copies)
**Phase 8 (Public Quote Pages):**
- Implement token security controls in parallel (not after)
- Rate limiting + email validation + expiration
- Code audit: quote_items never in response
- Test brute-force + enumeration
**Phase 9 (CRM — Won Automation):**
- Add idempotency_key to leads table
- Implement atomic "Win" transaction
- Store offer snapshot at win time
- Define Must/Should/Won't scope before design
- Integration tests for payment consistency
**Phase 10+ (Future):**
- Defer: email templates, call logging, team features, integrations
- Solo consultant doesn't need multi-user or GoHighLevel-scale features
---
## Testing Strategy for Confidence
### Phase 7 (Catalog & Offers)
1. Dry-run consolidation migration on production database backup
2. Verify all quote_items still resolve to services (no orphans)
3. Regenerate quotes from old projects, compare prices to originals
4. Concurrent drag-drop test: open offer in 2 tabs, drag in both, verify final state
5. Copy offer phases to test project, verify phase/task/deliverable counts
### Phase 8 (Public Quote Pages)
1. Brute-force token space: 1000 guesses/sec, verify rate limit activates
2. Enumerate tokens: generate 100 quotes, try to access one meant for different email, verify 401
3. Expiration test: set token_expires_at to past, verify access fails
4. Code audit: search response JSON for price, quote_items, per_service_price (should be zero matches)
5. Authorized access test: correct email + token, verify success
### Phase 9 (CRM — Won Automation)
1. Double-click "Win" button, verify only 1 client created
2. Network failure during "Win": simulate timeout after client creation, retry, verify same project returned
3. Partial failure: mock project creation failure, verify client not created (rollback)
4. Payment consistency: verify 24 payments created based on payment plan
5. Idempotency: call "Win" 10x with same idempotency_key, verify only 1 project, 1 set of payments
---
## Confidence Assessment
| Area | Level | Reason |
|------|-------|--------|
| Catalog consolidation risks | HIGH | Schema review shows existing quote_items dependencies; migration pattern verified via Drizzle docs |
| Offer copy semantics | HIGH | JavaScript shallow copy / deep copy distinction well-understood; transaction guarantees verified |
| Token security | HIGH | Brute-force math straightforward; nanoid 32 vs 21 entropy difference verified |
| CRM idempotency | HIGH | Idempotency pattern researched across multiple sources; double-click problem well-documented |
| Scope creep prevention | MEDIUM | CRM feature bloat is common, but solo consultant constraint makes Must/Should/Won't achievable |
| Concurrent edit handling | MEDIUM | Optimistic locking pattern standard, but requires careful implementation in Next.js + Drizzle |
---
## Next Steps for Phase Planning
1. **Before Phase 7 design starts:**
- Finalize catalog consolidation migration plan (Expand-Contract timeline)
- Document offer/project hierarchy mapping (1 page, code-level documentation)
- Design offer copy function signature and transaction strategy
2. **Before Phase 7 code starts:**
- Implement drag-drop versioning (version field + optimistic locking)
- Write integration tests for offer phase copy
3. **Before Phase 8 design starts:**
- Token security requirements: length (32), expiration (7d), email validation, rate limit (3/min)
- API response schema: exclude quote_items, include total_price only
4. **Before Phase 9 design starts:**
- Finalize Must/Should/Won't scope (document with user/consultant)
- Design "Win" workflow: idempotency key + atomic transaction
- Define payment plan algorithm (24 payments based on offer tier)
---
## References for Implementation
- **Drizzle migrations:** https://dev.to/whoffagents/zero-downtime-postgres-migrations-with-drizzle-orm-22ga
- **Idempotency pattern:** https://codefarm0.medium.com/the-double-click-problem-how-idempotency-saved-our-checkout-system-a704be65d207
- **Optimistic updates:** https://www.nirtamir.com/articles/optimistic-updates-state-vs-render/
- **Schema evolution:** https://www.dataexpert.io/blog/backward-compatibility-schema-evolution-guide
- **Token security:** https://workos.com/blog/oauth-common-attacks-and-how-to-prevent-them/
---
**Document location:** `/Users/simonecavalli/Vault/IAMCAVALLI/.planning/research/PITFALLS_V2.md` (detailed pitfall reference)
File diff suppressed because it is too large Load Diff