docs(v2.0): finalize roadmap + requirements — phases 7-11 approved for planning
- Upgrade PROJECT.md: v2.0 Business Operations Suite (unified catalog, offers with phases, public quotes, CRM pipeline, auto-provisioning) - Create REQUIREMENTS.md sections: CAT-U, OFFER-B, QUOTE, CRM, WIN requirements for phases 7-11 - Extend ROADMAP.md: 5-phase structure (7-11), 14-19 days total, risk/flag registry - Archive v1.0 milestone (phases 1-6, complete) - Switch STATE.md to v2.0, status=planning - Create research documents: STACK.md, FEATURES_v2.0.md, ARCHITECTURE.md, PITFALLS.md, SUMMARY.md Ready for: /gsd-plan-phase 7 Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,302 @@
|
||||
# Pitfalls Research Summary: Business Operations Suite (v2.0)
|
||||
|
||||
**Project:** ClientHub v2.0 — Adding Business Operations Suite to Production
|
||||
**Research Date:** 2026-06-10
|
||||
**Confidence:** HIGH (domain-specific research backed by schema review + security analysis)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Adding a complex Business Operations Suite (catalog consolidation + offer builder + public quotes + CRM automation) to a production app with real client data creates integration risks not present when building from scratch. This research documents the most dangerous pitfalls (rewrites/data loss) and operational friction points, with concrete prevention strategies for each.
|
||||
|
||||
**Key insight:** Most pitfalls cluster in four areas of high integration complexity:
|
||||
1. **Schema consolidation** — breaking existing quote_items referential integrity
|
||||
2. **Template-to-instance copy semantics** — offer phases mutating templates
|
||||
3. **Optimistic UI + concurrent edits** — sort order conflicts
|
||||
4. **Multi-step workflow atomicity** — CRM automation creating duplicates or partial state
|
||||
|
||||
---
|
||||
|
||||
## Critical Pitfalls (Require Design Changes Before Code)
|
||||
|
||||
### Pitfall 1: Catalog Consolidation Breaks Quotes (Data Loss Risk)
|
||||
|
||||
**Severity:** CRITICAL — Can orphan quote_items, break billing calculations
|
||||
|
||||
**Problem:**
|
||||
Two parallel service tables (`service_catalog` for costs, `offer_services` for marketing pricing) need consolidation. Naive merge breaks `quote_items` referential integrity or loses price history.
|
||||
|
||||
**Prevention Strategy:**
|
||||
- **Expand-Contract migration:** Add new unified `services` table alongside old ones, backfill in phases, only drop old tables after validation
|
||||
- **Immutable snapshot:** Store `quote_snapshot: JSONB` in project_offers (freeze prices at time of win)
|
||||
- **Referential integrity test:** Query for orphaned quote_items before/after migration
|
||||
- **Backward compatibility:** Old quotes read from old tables, new quotes use new table, transition over 2-3 weeks
|
||||
|
||||
**Phase:** 7 (Catalog & Offers) — Plan migration design BEFORE any code changes
|
||||
|
||||
**Validation checklist:**
|
||||
- [ ] Migration mapping layer documented (catalog ↔ services ↔ offer_services)
|
||||
- [ ] Backfill strategy written (small batches, checksum validation)
|
||||
- [ ] Dual-write period defined (how long to run old+new in parallel)
|
||||
- [ ] Referential integrity test query exists
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 2: Offer Template Mutation on Copy (Data Corruption Risk)
|
||||
|
||||
**Severity:** CRITICAL — Templates silently mutate when project phases edited
|
||||
|
||||
**Problem:**
|
||||
Copying offer phases to project via shallow copy = shared references. Admin edits project phase, template phase mutates too. Next deal uses corrupted template. Also: partial copies on retry create duplicate phases.
|
||||
|
||||
**Prevention Strategy:**
|
||||
- **Deep copy with atomic transaction:** Use `db.transaction()` for entire copy operation (all-or-nothing); copy at database level, not in JS
|
||||
- **Immutable template flag:** `offer_micros.is_template = true`, prevent UPDATE on templates
|
||||
- **Idempotency key:** "Win" request includes idempotency key; retry returns same project (no duplicates)
|
||||
- **Explicit hierarchy mapping:** Document what offer_micro → project_phase, what offer_service → project_task/deliverable
|
||||
|
||||
**Phase:** 7 (Offers) for design, 9 (CRM) for "Win" automation
|
||||
|
||||
**Validation checklist:**
|
||||
- [ ] Hierarchy mapping documented (offer ↔ project structures)
|
||||
- [ ] Atomic copy function implemented (single transaction, all-or-nothing)
|
||||
- [ ] Idempotency key added to CRM leads
|
||||
- [ ] Integration tests verify copy structure consistency across multiple instances
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 3: Public Quote Token Leakage (Security Risk)
|
||||
|
||||
**Severity:** CRITICAL — Token enumeration exposes all pricing
|
||||
|
||||
**Problem:**
|
||||
Public quote page with nanoid 21-char token can be brute-forced or enumerated. Attacker builds pricing database, breaches commercial confidentiality.
|
||||
|
||||
**Prevention Strategy:**
|
||||
- **Longer token:** nanoid(32) instead of 21 (~190 bits vs ~122 bits entropy)
|
||||
- **Access control:** Require both token + email (validate recipient match)
|
||||
- **Expiration:** token_expires_at (default 7 days)
|
||||
- **Rate limiting:** Max 3 views/token/minute (blocks enumeration)
|
||||
- **Never expose quote_items:** Public API response excludes line items, shows TOTAL PRICE ONLY
|
||||
- **Activation state:** Token valid only AFTER admin sends it (not auto-generated on create)
|
||||
|
||||
**Phase:** 8 (Public Quote Pages) — Implement security controls IN PARALLEL with feature
|
||||
|
||||
**Validation checklist:**
|
||||
- [ ] Token length ≥ 32 chars, rate limiting ≥ 3/min
|
||||
- [ ] Email validation on public page (token + email both required)
|
||||
- [ ] Expiration enforced (test expired token returns 401)
|
||||
- [ ] Code audit: quote_items never in public API response
|
||||
- [ ] Brute-force test: verify rate limit activates
|
||||
|
||||
---
|
||||
|
||||
## Moderate Pitfalls (Major Refactoring / Data Inconsistency)
|
||||
|
||||
### Pitfall 4: Drag-and-Drop Sort Order Race Condition
|
||||
|
||||
**Severity:** MODERATE — Phases display in wrong order, user frustration
|
||||
|
||||
**Problem:**
|
||||
Concurrent drag-and-drop edits in multiple tabs cause sort_order conflicts. Last write wins, earlier update is lost. No version conflict detection.
|
||||
|
||||
**Prevention Strategy:**
|
||||
- **Optimistic locking:** Add `version` field to offer_phases, update only if version matches (return 409 Conflict if mismatch)
|
||||
- **Server-side recomputation:** Don't trust client's sort_order, recompute all orders atomically based on actual position
|
||||
- **Optimistic UI with reconciliation:** Update frontend instantly, reconcile with server result (409 = refresh)
|
||||
|
||||
**Phase:** 7 (Offer Builder — Drag & Drop)
|
||||
|
||||
**Validation checklist:**
|
||||
- [ ] `version` field added to offer_phases
|
||||
- [ ] Update mutation includes version check
|
||||
- [ ] Server recomputes all sort_orders atomically
|
||||
- [ ] Concurrent edit test exists (open 2 tabs, drag in both)
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 5: CRM "Win" Double-Click Creates Duplicate Clients
|
||||
|
||||
**Severity:** MODERATE — Duplicate clients, broken billing, audit confusion
|
||||
|
||||
**Problem:**
|
||||
Multi-step "Win" automation (create client → project → phases → payments) has no idempotency. Double-click creates two clients, two projects. Network timeouts are invisible to user.
|
||||
|
||||
**Prevention Strategy:**
|
||||
- **Idempotency key:** Every lead has unique idempotency_key; "Win" request checks if key already processed (if yes, return existing client_id)
|
||||
- **Atomic transaction:** All steps (client, project, phases, payments) in single transaction — all succeed or all rollback
|
||||
- **UI feedback:** Button disabled until success (prevents accidental double-click)
|
||||
- **Browser persistence:** Store idempotency_key in localStorage, preserve across refresh
|
||||
|
||||
**Phase:** 9 (CRM Won Automation)
|
||||
|
||||
**Validation checklist:**
|
||||
- [ ] CRM leads table has idempotency_key column (unique)
|
||||
- [ ] "Win" mutation checks for existing key before creating
|
||||
- [ ] All substeps in single db.transaction()
|
||||
- [ ] Button disabled until response received
|
||||
- [ ] Idempotency key stored in localStorage
|
||||
- [ ] Test: double-click "Win", verify only 1 client created
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 6: Offer Phase Copy Misses Tasks/Deliverables
|
||||
|
||||
**Severity:** MODERATE — Client dashboard shows incomplete phases (no tasks to approve)
|
||||
|
||||
**Problem:**
|
||||
Copy offer → project phases but forget to copy tasks or deliverables. Client sees phase with no work items.
|
||||
|
||||
**Prevention Strategy:**
|
||||
- **Explicit mapping:** Document offer_micro → project_phase, offer_service → project_task/deliverable mapping
|
||||
- **Full-tree copy function:** Copy phases, then tasks for each phase, then deliverables for each task (all in one transaction)
|
||||
- **Validation test:** Assert phase_count, task_count, deliverable_count match expected values after copy
|
||||
|
||||
**Phase:** 7 (Offers structure design), 9 (Copy implementation)
|
||||
|
||||
**Validation checklist:**
|
||||
- [ ] Mapping documented (offer vs project hierarchies)
|
||||
- [ ] Copy function handles entire tree (phases → tasks → deliverables)
|
||||
- [ ] Integration test: copy offer, verify counts match expected
|
||||
- [ ] Query test: check no deliverables are orphaned (task_id IS NULL)
|
||||
|
||||
---
|
||||
|
||||
## Minor Pitfalls (Operational Friction)
|
||||
|
||||
### Pitfall 7: Offer State Not Synced Between Tabs
|
||||
|
||||
**Problem:** Admin edits offer in tab 1, tab 2 doesn't know, last save wins (first admin's work lost)
|
||||
|
||||
**Prevention:** Add `version` field, return 409 on version mismatch, show "Offer changed, reload?"
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 8: CRM Schema Backward Incompatibility
|
||||
|
||||
**Problem:** Add new fields (budget, source) as NULL, code assumes populated, crashes
|
||||
|
||||
**Prevention:** Expand-Contract pattern — add as NULL, handle NULL defensively in code, backfill, then add NOT NULL constraint
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 9: CRM Scope Creep (Feature Bloat)
|
||||
|
||||
**Problem:** "Just add email templates" → 30 hours. "Add calls" → 40 hours. CRM never ships.
|
||||
|
||||
**Prevention:** Explicit Must/Should/Could/Won't scope document. Must-haves only for v2.0: lead pipeline, quote attachment, auto-onboarding. Defer rest.
|
||||
|
||||
---
|
||||
|
||||
## Key Design Decisions for Phase 7–9
|
||||
|
||||
| Decision | Why | Validation |
|
||||
|----------|-----|-----------|
|
||||
| Expand-Contract migration for catalog | Zero-downtime, safe rollback, no data loss | Dry-run on prod backup before go-live |
|
||||
| Deep copy + atomic transaction for offer phases | Prevents template mutation, partial copies | Integration test, structure consistency query |
|
||||
| Idempotency key on "Win" action | Safe retry, prevents duplicate clients | Double-click test, query for duplicates |
|
||||
| Public quote token security layers (length, expiration, email, rate limit) | Blocks enumeration, limits leakage | Brute-force test, code audit for quote_items |
|
||||
| Offer snapshot (immutable JSONB) | Preserves what was promised vs. what's executing | Store at win time, display in client dashboard |
|
||||
| Version field on offer_phases + optimistic locking | Detects concurrent edit conflicts | Concurrent drag test, 409 Conflict handling |
|
||||
|
||||
---
|
||||
|
||||
## Roadmap Implications
|
||||
|
||||
**Phase 7 (Catalog & Offers):**
|
||||
- Design catalog consolidation migration before any code
|
||||
- Implement drag-drop with version fields from start
|
||||
- Document offer/project hierarchy mapping explicitly
|
||||
- Write copy function atomically (no partial copies)
|
||||
|
||||
**Phase 8 (Public Quote Pages):**
|
||||
- Implement token security controls in parallel (not after)
|
||||
- Rate limiting + email validation + expiration
|
||||
- Code audit: quote_items never in response
|
||||
- Test brute-force + enumeration
|
||||
|
||||
**Phase 9 (CRM — Won Automation):**
|
||||
- Add idempotency_key to leads table
|
||||
- Implement atomic "Win" transaction
|
||||
- Store offer snapshot at win time
|
||||
- Define Must/Should/Won't scope before design
|
||||
- Integration tests for payment consistency
|
||||
|
||||
**Phase 10+ (Future):**
|
||||
- Defer: email templates, call logging, team features, integrations
|
||||
- Solo consultant doesn't need multi-user or GoHighLevel-scale features
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy for Confidence
|
||||
|
||||
### Phase 7 (Catalog & Offers)
|
||||
1. Dry-run consolidation migration on production database backup
|
||||
2. Verify all quote_items still resolve to services (no orphans)
|
||||
3. Regenerate quotes from old projects, compare prices to originals
|
||||
4. Concurrent drag-drop test: open offer in 2 tabs, drag in both, verify final state
|
||||
5. Copy offer phases to test project, verify phase/task/deliverable counts
|
||||
|
||||
### Phase 8 (Public Quote Pages)
|
||||
1. Brute-force token space: 1000 guesses/sec, verify rate limit activates
|
||||
2. Enumerate tokens: generate 100 quotes, try to access one meant for different email, verify 401
|
||||
3. Expiration test: set token_expires_at to past, verify access fails
|
||||
4. Code audit: search response JSON for price, quote_items, per_service_price (should be zero matches)
|
||||
5. Authorized access test: correct email + token, verify success
|
||||
|
||||
### Phase 9 (CRM — Won Automation)
|
||||
1. Double-click "Win" button, verify only 1 client created
|
||||
2. Network failure during "Win": simulate timeout after client creation, retry, verify same project returned
|
||||
3. Partial failure: mock project creation failure, verify client not created (rollback)
|
||||
4. Payment consistency: verify 2–4 payments created based on payment plan
|
||||
5. Idempotency: call "Win" 10x with same idempotency_key, verify only 1 project, 1 set of payments
|
||||
|
||||
---
|
||||
|
||||
## Confidence Assessment
|
||||
|
||||
| Area | Level | Reason |
|
||||
|------|-------|--------|
|
||||
| Catalog consolidation risks | HIGH | Schema review shows existing quote_items dependencies; migration pattern verified via Drizzle docs |
|
||||
| Offer copy semantics | HIGH | JavaScript shallow copy / deep copy distinction well-understood; transaction guarantees verified |
|
||||
| Token security | HIGH | Brute-force math straightforward; nanoid 32 vs 21 entropy difference verified |
|
||||
| CRM idempotency | HIGH | Idempotency pattern researched across multiple sources; double-click problem well-documented |
|
||||
| Scope creep prevention | MEDIUM | CRM feature bloat is common, but solo consultant constraint makes Must/Should/Won't achievable |
|
||||
| Concurrent edit handling | MEDIUM | Optimistic locking pattern standard, but requires careful implementation in Next.js + Drizzle |
|
||||
|
||||
---
|
||||
|
||||
## Next Steps for Phase Planning
|
||||
|
||||
1. **Before Phase 7 design starts:**
|
||||
- Finalize catalog consolidation migration plan (Expand-Contract timeline)
|
||||
- Document offer/project hierarchy mapping (1 page, code-level documentation)
|
||||
- Design offer copy function signature and transaction strategy
|
||||
|
||||
2. **Before Phase 7 code starts:**
|
||||
- Implement drag-drop versioning (version field + optimistic locking)
|
||||
- Write integration tests for offer phase copy
|
||||
|
||||
3. **Before Phase 8 design starts:**
|
||||
- Token security requirements: length (32), expiration (7d), email validation, rate limit (3/min)
|
||||
- API response schema: exclude quote_items, include total_price only
|
||||
|
||||
4. **Before Phase 9 design starts:**
|
||||
- Finalize Must/Should/Won't scope (document with user/consultant)
|
||||
- Design "Win" workflow: idempotency key + atomic transaction
|
||||
- Define payment plan algorithm (2–4 payments based on offer tier)
|
||||
|
||||
---
|
||||
|
||||
## References for Implementation
|
||||
|
||||
- **Drizzle migrations:** https://dev.to/whoffagents/zero-downtime-postgres-migrations-with-drizzle-orm-22ga
|
||||
- **Idempotency pattern:** https://codefarm0.medium.com/the-double-click-problem-how-idempotency-saved-our-checkout-system-a704be65d207
|
||||
- **Optimistic updates:** https://www.nirtamir.com/articles/optimistic-updates-state-vs-render/
|
||||
- **Schema evolution:** https://www.dataexpert.io/blog/backward-compatibility-schema-evolution-guide
|
||||
- **Token security:** https://workos.com/blog/oauth-common-attacks-and-how-to-prevent-them/
|
||||
|
||||
---
|
||||
|
||||
**Document location:** `/Users/simonecavalli/Vault/IAMCAVALLI/.planning/research/PITFALLS_V2.md` (detailed pitfall reference)
|
||||
Reference in New Issue
Block a user