simone
bdcdd26f17
Two-plan, two-wave migration: schema + backfill (07-01) then admin catalog CRUD rewire (07-02), per CAT-U-01/CAT-U-02 and the expand-contract pattern from PITFALLS_V2 Pitfall 1.
24 KiB
24 KiB
phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves
| phase | plan | type | wave | depends_on | files_modified | autonomous | requirements | must_haves | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 07-unified-service-catalog | 01 | execute | 1 |
|
true |
|
|
Purpose: Establish the single source of truth for service pricing without touching service_catalog, offer_services, quote_items, or offer_micro_services — those keep working unchanged. This is the "Expand" half of expand-contract; Phase 8 will build offer_phase_services against the new services table and progressively retire the old junction tables. Per Data Safety (LOCKED): no migration in this plan drops or truncates clients, projects, payments, phases, service_catalog, or offer_services.
Output: services table live in production Postgres, fully backfilled with audit trail (migrated_from, migrated_id), validated against zero data loss with a checksum/row-count script.
<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>
@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/research/ARCHITECTURE.md @.planning/research/PITFALLS_V2.mdFrom src/db/schema.ts (service_catalog — 21 rows, used by quote_items.service_id):
export const service_catalog = pgTable("service_catalog", {
id: text("id").primaryKey().$defaultFn(() => nanoid()),
name: text("name").notNull(),
description: text("description"),
unit_price: numeric("unit_price", { precision: 10, scale: 2 }).notNull(),
active: boolean("active").notNull().default(true),
});
From src/db/schema.ts (offer_services — 35 rows, used by offer_micro_services junction):
export const offer_services = pgTable("offer_services", {
id: text("id").primaryKey().$defaultFn(() => nanoid()),
name: text("name").notNull(),
price: numeric("price", { precision: 10, scale: 2 }).notNull(),
transformation_description: text("transformation_description"),
active: boolean("active").notNull().default(true),
});
From src/db/index.ts (db client — used identically in scripts/seed.ts):
import { drizzle } from "drizzle-orm/postgres-js";
import postgres from "postgres";
const client = postgres(process.env.DATABASE_URL ?? "postgres://build-placeholder");
export const db = drizzle(client);
Target shape for the new table (from ARCHITECTURE.md "NEW: services Table"):
export const services = pgTable("services", {
id: text("id").primaryKey().$defaultFn(() => nanoid()),
name: text("name").notNull(),
description: text("description"),
unit_price: numeric("unit_price", { precision: 10, scale: 2 }).notNull(),
category: text("category"),
active: boolean("active").notNull().default(true),
migrated_from: text("migrated_from"), // "service_catalog" | "offer_services" | null
migrated_id: text("migrated_id"), // original row id from source table
created_at: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
});
```typescript
// ============ SERVICES (UNIFIED CATALOG — replaces service_catalog + offer_services) ============
// migrated_from/migrated_id provide audit trail for safe rollback during the
// expand-contract migration (Phase 7 expand; Phase 8 begins contract on offer side).
export const services = pgTable("services", {
id: text("id")
.primaryKey()
.$defaultFn(() => nanoid()),
name: text("name").notNull(),
description: text("description"),
unit_price: numeric("unit_price", { precision: 10, scale: 2 }).notNull(),
category: text("category"),
active: boolean("active").notNull().default(true),
migrated_from: text("migrated_from"), // "service_catalog" | "offer_services" | null (new rows after Phase 7)
migrated_id: text("migrated_id"), // original id from source table, null for new rows
created_at: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
});
export const servicesRelations = relations(services, (_) => ({
// No FK relations yet in Phase 7 — quote_items and offer_micro_services
// continue to reference service_catalog/offer_services until Phase 8.
}));
```
Add TypeScript types in the "TYPESCRIPT TYPES" section at the bottom of the file, near `ServiceCatalog`:
```typescript
export type Service = typeof services.$inferSelect;
export type NewService = typeof services.$inferInsert;
```
Do NOT modify `service_catalog`, `offer_services`, `quote_items`, `offer_micro_services`, or any other existing table. Do NOT add foreign keys from `services` to anything yet — this is a standalone additive table.
Push the schema change to production Postgres on Coolify using the same SSH-based `drizzle-kit push` pattern used in Phase 5 (commit d670d49 / 1b0b2ea). Run:
```bash
npx drizzle-kit push
```
Confirm the prompt shows ONLY a new table creation (`services`) with no ALTER/DROP statements on existing tables. If drizzle-kit proposes anything beyond `CREATE TABLE services` (e.g., proposes dropping or renaming an unrelated column), STOP and report — do not proceed with an unexpected diff.
Implementation outline:
```typescript
import { db } from "@/db";
import { service_catalog, offer_services, services } from "@/db/schema";
import { eq, and } from "drizzle-orm";
async function migrate() {
console.log("Starting services unification migration...\n");
// 1. Backfill from service_catalog (operational pricing, used by quote_items)
const catalogRows = await db.select().from(service_catalog);
let catalogInserted = 0;
let catalogSkipped = 0;
for (const row of catalogRows) {
const existing = await db
.select()
.from(services)
.where(and(eq(services.migrated_from, "service_catalog"), eq(services.migrated_id, row.id)))
.limit(1);
if (existing.length > 0) {
catalogSkipped++;
continue;
}
await db.insert(services).values({
name: row.name,
description: row.description,
unit_price: row.unit_price,
category: "catalog",
active: row.active,
migrated_from: "service_catalog",
migrated_id: row.id,
});
catalogInserted++;
}
console.log(`service_catalog: ${catalogInserted} inserted, ${catalogSkipped} skipped (already migrated)`);
// 2. Backfill from offer_services (marketing pricing, used by offer_micro_services)
const offerRows = await db.select().from(offer_services);
let offerInserted = 0;
let offerSkipped = 0;
let renamed = 0;
for (const row of offerRows) {
const existing = await db
.select()
.from(services)
.where(and(eq(services.migrated_from, "offer_services"), eq(services.migrated_id, row.id)))
.limit(1);
if (existing.length > 0) {
offerSkipped++;
continue;
}
// Name collision check against ALL services rows inserted so far (both sources)
const collision = await db
.select()
.from(services)
.where(eq(services.name, row.name))
.limit(1);
const finalName = collision.length > 0 ? `${row.name} (Offer)` : row.name;
if (collision.length > 0) renamed++;
await db.insert(services).values({
name: finalName,
description: row.transformation_description,
unit_price: row.price,
category: "offer",
active: row.active,
migrated_from: "offer_services",
migrated_id: row.id,
});
offerInserted++;
}
console.log(`offer_services: ${offerInserted} inserted, ${offerSkipped} skipped (already migrated), ${renamed} renamed for collision`);
console.log("\nMigration complete. Run scripts/validate-services-migration.ts next.");
process.exit(0);
}
migrate().catch((err) => {
console.error("Migration failed:", err);
process.exit(1);
});
```
Run the script against production:
```bash
npx tsx scripts/migrate-services.ts
```
Report the printed counts (inserted/skipped/renamed for each source table).
```typescript
import { db } from "@/db";
import { service_catalog, offer_services, services, quote_items, offer_micro_services } from "@/db/schema";
import { sql, eq } from "drizzle-orm";
async function validate() {
let failures = 0;
// Check 1: row counts match
const [catalogCount] = await db.select({ n: sql<number>`count(*)::int` }).from(service_catalog);
const [offerCount] = await db.select({ n: sql<number>`count(*)::int` }).from(offer_services);
const [migratedCatalog] = await db
.select({ n: sql<number>`count(*)::int` })
.from(services)
.where(eq(services.migrated_from, "service_catalog"));
const [migratedOffer] = await db
.select({ n: sql<number>`count(*)::int` })
.from(services)
.where(eq(services.migrated_from, "offer_services"));
console.log(`service_catalog rows: ${catalogCount.n} | migrated: ${migratedCatalog.n}`);
console.log(`offer_services rows: ${offerCount.n} | migrated: ${migratedOffer.n}`);
if (catalogCount.n !== migratedCatalog.n) {
console.log("FAIL: service_catalog row count does not match migrated count");
failures++;
} else {
console.log("PASS: service_catalog fully migrated");
}
if (offerCount.n !== migratedOffer.n) {
console.log("FAIL: offer_services row count does not match migrated count");
failures++;
} else {
console.log("PASS: offer_services fully migrated");
}
// Check 2: no orphaned migrated_id (every migrated_id from service_catalog still exists in service_catalog)
const orphanedCatalog = await db.execute(sql`
SELECT COUNT(*)::int AS n FROM services s
WHERE s.migrated_from = 'service_catalog'
AND NOT EXISTS (SELECT 1 FROM service_catalog sc WHERE sc.id = s.migrated_id)
`);
const orphanedCatalogCount = (orphanedCatalog as unknown as Array<{ n: number }>)[0]?.n ?? 0;
if (orphanedCatalogCount > 0) {
console.log(`FAIL: ${orphanedCatalogCount} services rows reference missing service_catalog ids`);
failures++;
} else {
console.log("PASS: no orphaned service_catalog references");
}
const orphanedOffer = await db.execute(sql`
SELECT COUNT(*)::int AS n FROM services s
WHERE s.migrated_from = 'offer_services'
AND NOT EXISTS (SELECT 1 FROM offer_services os WHERE os.id = s.migrated_id)
`);
const orphanedOfferCount = (orphanedOffer as unknown as Array<{ n: number }>)[0]?.n ?? 0;
if (orphanedOfferCount > 0) {
console.log(`FAIL: ${orphanedOfferCount} services rows reference missing offer_services ids`);
failures++;
} else {
console.log("PASS: no orphaned offer_services references");
}
// Check 3: existing quote_items.service_id still resolve in service_catalog (untouched FK — must remain valid)
const orphanedQuoteItems = await db.execute(sql`
SELECT COUNT(*)::int AS n FROM quote_items qi
WHERE qi.service_id IS NOT NULL
AND NOT EXISTS (SELECT 1 FROM service_catalog sc WHERE sc.id = qi.service_id)
`);
const orphanedQuoteItemsCount = (orphanedQuoteItems as unknown as Array<{ n: number }>)[0]?.n ?? 0;
if (orphanedQuoteItemsCount > 0) {
console.log(`FAIL: ${orphanedQuoteItemsCount} quote_items reference missing service_catalog ids (pre-existing FK broken!)`);
failures++;
} else {
console.log("PASS: quote_items.service_id -> service_catalog FK intact (unchanged by this migration)");
}
// Check 4: existing offer_micro_services.service_id still resolve in offer_services (untouched FK — must remain valid)
const orphanedMicroServices = await db.execute(sql`
SELECT COUNT(*)::int AS n FROM offer_micro_services oms
WHERE NOT EXISTS (SELECT 1 FROM offer_services os WHERE os.id = oms.service_id)
`);
const orphanedMicroServicesCount = (orphanedMicroServices as unknown as Array<{ n: number }>)[0]?.n ?? 0;
if (orphanedMicroServicesCount > 0) {
console.log(`FAIL: ${orphanedMicroServicesCount} offer_micro_services reference missing offer_services ids (pre-existing FK broken!)`);
failures++;
} else {
console.log("PASS: offer_micro_services.service_id -> offer_services FK intact (unchanged by this migration)");
}
// Check 5: name collision report (informational)
const collisions = await db.execute(sql`
SELECT name, COUNT(*)::int AS n FROM services GROUP BY name HAVING COUNT(*) > 1
`);
const collisionRows = collisions as unknown as Array<{ name: string; n: number }>;
if (collisionRows.length > 0) {
console.log(`WARNING: ${collisionRows.length} duplicate names remain in services (review):`, collisionRows);
} else {
console.log("PASS: no duplicate names in services");
}
console.log(`\n${failures === 0 ? "ALL CHECKS PASSED" : `${failures} CHECK(S) FAILED`}`);
process.exit(failures === 0 ? 0 : 1);
}
validate().catch((err) => {
console.error("Validation failed:", err);
process.exit(1);
});
```
Run:
```bash
npx tsx scripts/validate-services-migration.ts
```
All checks must print PASS. If any check prints FAIL, STOP — do not proceed to Plan 07-02 until resolved (do not drop/modify any table to "fix" a failure; investigate the migrate-services.ts logic instead since source tables must remain untouched).
<threat_model>
Trust Boundaries
| Boundary | Description |
|---|---|
| Migration script -> Production Postgres | Script runs with full DB credentials (DATABASE_URL); writes new rows only, must never DELETE/UPDATE/DROP on service_catalog or offer_services |
| drizzle-kit push -> Production schema | Schema diff applied directly to production (no staging DB available); diff must be additive-only |
STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|---|---|---|---|---|
| T-07-01 | Tampering | drizzle-kit push proposes unintended ALTER/DROP on existing tables | mitigate | Task 1 explicitly requires reviewing the push diff before confirming; abort if anything beyond CREATE TABLE services appears |
| T-07-02 | Repudiation | Migration cannot be traced back to source rows after running | mitigate | migrated_from + migrated_id columns on every backfilled row provide full audit trail for rollback |
| T-07-03 | Tampering | Re-running migrate-services.ts creates duplicate rows on retry | mitigate | Idempotency check via migrated_from + migrated_id lookup before each insert (Task 2 behavior spec) |
| T-07-04 | Information Disclosure | DATABASE_URL exposed in script output/logs | accept | Scripts use process.env.DATABASE_URL via existing @/db client; never logged; same pattern as scripts/seed.ts already in repo |
| T-07-05 | Denial of Service | Backfill script run against production during business hours locks tables | accept | 56 rows total — INSERT-only operations on a new empty table; no lock contention with existing read paths (service_catalog/offer_services untouched) |
| </threat_model> |
<success_criteria>
servicestable exists in production Postgres with the full column set from ARCHITECTURE.md- 100% of service_catalog and offer_services rows have a corresponding services row with migrated_from/migrated_id set
- Zero orphaned references; existing quote_items and offer_micro_services FKs remain valid (untouched)
- Name collisions resolved via deterministic suffixing, no silent overwrites
- service_catalog and offer_services tables remain in place, unmodified — rollback is possible by simply not using the new table
- npm run build passes </success_criteria>