tmp
Twenty Manor Projects · Front Porch
Project Blueprint
Issued2026-05-17
Updated2026-06-16
StatusIn build
Scale1:1
A.00
Set of 5
The Machine.
The TMP Front Porch / exploded view
callout numbers route to detail sheets
▸ Data Inputs · today ▸ Pipeline ▸ Output Tools WinRed hourly CSV → S3 Revv hourly CSV → S3 Email Octopus engagement events (planned) ▸ Scribe V2 inputs · for data monetization signal Salesforce data-monetization $$ raised Wondercave creative + clicks (P2P) V2 · direct to Scribe A S3 Buckets front door twentymanorbucket/ B ParquetPipeline the translator Lambda · Python · pyarrow C Canonical Parquet safety net D Tinybird the fast brain 11 live endpoints · effectively complete 1 Drop capture creative 2 Tasks workflow + approval 3 Courier partner distribution · 6 open 4 Performance Layer Homepage Pulse · live Rundown · in validation Client Reports · building 5 Scribe the AI writer · Phase 0.5 LIVE P0 P1 P2 2.5 P3 P4 P5 P6 ▲ un-park decision pending ▸ Supporting · legacy + replacement plumbing Apps Script + Sheets ⚠ legacy · hazard Netlify Blobs replacement Zapier $500/mo · retiring
01
Sheet
A.01 · The Parts

Components – what the machine produces.

5 components
callouts ① – ⑤
1
Drop
Capture creative + metadata
Live
FORM 14-COL SCHEMA slug package_id utm_content brief, ... SHEETS
Captures every piece of creative + metadata so it can be tied to performance later.
  • Form capture
  • 14-col schema · slug, package_id, utm_content, brief
  • Slug + UTM attribution
  • Adam-only review transitions
Next Feature-complete. Backend migrated to Netlify Blobs 2026-06-16 (Apps Script retired from the path). Browsable Drops Ledger added.
2
Tasks
Workflow + approval
Live
NEW ADAM APPROVED SENT queue review writer ships
Workflow + approval state for every Drop moving through the pipeline.
  • 11-col Tasks + 6-col Comments schemas
  • Approval routing + Mark Sent lifecycle
  • Permission model + comment recipient rule
Open issue
None. The inbox-accumulation issue resolved via the approver clause + Mark-Sent-gated-on-Approved (2026-06-16).
Next Migrated to Netlify Blobs 2026-06-16 — Apps Script retired, reads sub-500ms. Recovery surface + Drops Ledger added.
3
Courier
Partner distribution
6 Open
COMPOSER email · P2P AGENCIES 34-col schema SENDERS 3-col schema PARTNER → EO / WC → agency
Shopping creative to outside agencies + importing partner creative back in.
  • Agency + sender + template data on Netlify Blobs · 2026-06-30 (was the Master Agency List Sheet/Apps Script; now Blobs-first, sheet = fallback)
  • Agency Editor (Adam-only, Back Porch) · form add/edit, paired Slack-mention rows, replaces editing the sheet
  • 34-col Agencies + 3-col Senders schemas
  • Agency pitch composer (email + P2P)
  • Tier 1 dynamic agency-top.js
  • Pitch Templates tab schema · undocumented
6 open issues from 2026-05-15
TMA-on-H4H 82% slider seed · Tier 1 precedence ambiguous · Pitch Templates undocumented · deprecated courier-message-raise.gs · proxy debug instrumentation · stress-test plan not back-filled
Next message-raise Tinybird cutover COMPLETE 2026-05-17 (legacy retired, perf panel ~5x faster, silently fixed 2x overcount bug). Dead "+Add new sender" button RESOLVED 2026-06-30 via the Agency Editor. Remaining: optional test-send to TMP Test; resolve the older open issues.
4
Performance Layer
See the numbers
In validation
TINYBIRD same data HOMEPAGE pulse cards RUNDOWN CLIENT REPORTS 2 of N · hand-built
See the numbers. Internal tracking + homepage at-a-glance + client-facing reports, all on the same data.
  • Homepage Client Pulse Cards · live & validated. WinRed recurring tile hidden as workaround.
  • Internal Rundown · UI wired on Tinybird for all 6. Only MTD gross validated. Order: STPC → H4H → Moody.
  • Client-Facing Reports · 2 hand-authored (H4H, STPC). Generator not started.
Next Per-client per-section Rundown validation rollout. Build report generator once numbers are trusted.
5
Scribe
The AI writer · crown jewel
Phase 0.5 LIVE
P0 v3 eval 0.5 LIVE P1 corpus 1.5 Courier-C 1.6 EO+WC-C P2 feedback P3 Twilio 7 PHASES · plan of record locked 2026-05-17
AI writer for political fundraising. 4-phase Scribe Build plan locked 2026-05-17 evening after multi-round pressure-test with chat-Claude + Grok.
  • Phase 0 · Prompt validation gates passed 2026-05-19 evening (cross-brief + mixed-verdict model comparison). One structural fix remaining: pre-flight A-log stub for Lambda 60s timeout coverage.
  • Phase 0.5 · Claude/Grok toggle LIVE + A-logging to S3 with stop_reason instrumentation now writing through (commit 3a9cd6b 2026-05-19 evening)
  • Phase 1 · Corpus pipeline LIVE on 2 clients (H4H + Moody) as of 2026-05-21 evening. Bundled-Doc defensive parser shipped (V1 -short/-long detection, duplicate fallback). Cameron backfill next.
  • Phase 1.5 · Courier C-capture intent-text, sequenced first (cheapest + largest volume)
  • Phase 1.6 · EO+Wondercave C · Phase 2 · kept_ratio feedback · Phase 3 · Twilio pitch
Phase 2 v1 quality issues RESOLVED via Phase 0 track
LP regurgitation · SMS over-limit · citation marker leak → addressed via reasoning prompt rework + model toggle + LP/email single-variant fixes (2026-05-19 evening)
Next Phase 1 corpus pipeline LIVE on H4H + Moody end-to-end as of 2026-05-21 evening. Moody Vindman SMS brief surfaced real corpus records (cold-hard-revenge, five-alarm-inferno, Don Jr. Vindman bundle, democrats-target-florida as failure-mode counter-example) — second client validated. Defensive bundled-Doc parser shipped in backfill_cli.py (V1: -short/-long header detection, duplicate fallback). All 15 Moody bundles fell to duplicated under V1; shipping as-is, Phase 1.5 expands heuristics with real data. Four robustness follow-ups banked from live-validation debug: office-wifi heartbeat (Bluecoat proxy), Anthropic blocked-domain error wrap (Politico failure mode), source-diet.mjs known-blocked-domains filter, draft-truncation A-log diagnostic. Cameron backfill (~95 records, 21 bundled) is next planned client — skipped in ongoing cron (officially offboarded + archived from FP dashboards today). STPC / SWP / TVE / Mercury One queued after.
02
Sheet
A.02 · The Plumbing

Infrastructure – the wiring under the hood.

callouts A – G
keyed to hero
▸ Data Flow A S3 front door B ParquetPipeline the translator C Canonical Parquet safety net D Tinybird the fast brain · effectively complete ▾ consumers above (Lane 01) ▾ E Salesforce donor identity F Apps Script + Sheets ⚠ LEGACY · hazard G Netlify Blobs replacement ▸ Delivery Pipes Email Octopus · Wondercave · Twilio (future)
A · S3
The front door
Every client donation report from WinRed + Revv lands here first. Future: EO engagement events.
B · ParquetPipeline
The translator
Lambda that makes Revv and WinRed look the same downstream. Python + pyarrow + SAM. Schema canonicalization in schema_mapper.py.
C · Canonical Parquet
The safety net
Our own vendor-neutral copy of every donation, hive-partitioned. If we ever swap tools, the data is still ours.
D · Tinybird
The fast brain
One always-warm query layer. 11 live endpoints. ~$49.74/mo on vCPU-seconds billing (QPS billing retired by Tinybird 5/18; `rundown-warm` cron paused 6/10). Storage ~0.06%, compute ~27%. Bronze → silver → gold.
Hard rule: FROM donations_v2 FINAL on any pipe reading bronze directly.
SF · Salesforce
Data-monetization data source
Where ALL data-monetization donations live (the bulk routes through WinRed → SF; some come in via direct APIs and skip WinRed entirely). Plus Group A/B/G/W bucketing + Wonder Cave dynamic segments via API.
Scribe V2 input. $2k/yr. Doug's domain.
WC · Wondercave
Data-monetization messages + clicks
P2P deployment platform. The actual creative copy + click data for data-monetization sends lives here.
Scribe V2 input. S3 access pending.
F · Apps Script + Sheets
Legacy storage
Drops / Tasks / Comments / Agency-list. 10-15s cache miss. Being retired by Blobs migration.
⚠ tasks.gs editor has Grok stubs · DO NOT click Deploy → New version
G · Netlify Blobs
Replacement storage
Already in the stack as the 30s cache layer. Target for Tasks/Drops/Comments migration. No new platform.
Z · Zapier
Legacy outbound · being retired
Today pushes WinRed/Revv donors → Email Octopus (deployment) + Salesforce (Doug's hooks). Replacement path: S3 custom reports drive both EO and SF directly.
~$500/mo · saved when retired
03
Sheet
A.03 · Work In Flight

Initiatives – cross-cutting work with finish lines.

10 initiatives
in flight + queued
100% complete
Tinybird Migration
Old per-Lambda DuckDB approach didn't scale past 6 clients. Replaced with one always-warm query layer everyone hits the same way. Closed out 2026-05-23: home-data 100% + Rundown 100% + Courier message-raise 100% + agency-top 100% (new `top_agencies` pipe). Bridge retired entirely (not migrated) — confirmed write-only artifact, nothing downstream read its Live Summary Docs; code deleted, 9 AM ET cron killed, Drive Docs frozen as historical. DuckDB-in-Lambda fully gone: `duckdb` dropped from package.json (npm removed 112 transitive packages), three consumers cleaned up. Net diff across the close-out: -3169 lines.
Complete
Step 3 CLOSED 6/11 – validated; next: context cleanup → Step 4 radar
Scribe Learning Layer Build
Prompt rebuild progressed Step 1 → Step 2 → Step 3 over the 5/31–6/2 sessions. Step 3 LIVE 2026-06-02 via single master toggle STEP3_ACTIVE in _scribe-lib/phase1-assembly.mjs: whole-email perf-ranked corpus block in the prompt (top 3 client-scoped records ranked by performance.dollars_raised desc, deduped by source_doc_id, no Layer B cross-client). Step 2 (full profile excerpt minus radar sections, client-agnostic name-pattern exclusion) shipped earlier same session. Performance data wired into the corpus for the first time: 222 / 373 records (60%) carry refund-netted dollars + donor counts via message_raise_v2 Tinybird join + a hardcoded MVE-FC archive table from TVE profile §7 (retired committee never in donations_v2). 6 prompt blocks now (preamble + URL rule + principles + profile excerpt + corpus + directives) totaling ~8.3K projected tokens for STPC (vs ~4.9K Step 2, ~2.4K Step 1, ~30K old-prod-mode bloat). Three production hard-directive bugs fixed first: voice (3-class structured AUTHORITATIVE override of past-hits sender patterns), channel (explicit email branch with ONLY-email exclusivity), tone (renamed Length hint → Tone AUTHORITATIVE). Brief + output cards now clear on client switch. New infrastructure: ScribeCorpus under git for the first time at github.com/amwaldeck/scribecorpus (private, follows ParquetPipeline precedent). Phase 1 corpus pipeline + retrieval module + bundled-Doc parser all shipped earlier; corpus consumed from local deploy bundle (_scribe-corpus/*.jsonl via netlify.toml included_files). Update 2026-06-11 — Step 3 CLOSED: relevance-primary corpus selection live (cosine 0.65 / dollar-tiebreak band 0.03, validated for variety); "The arc" structural principles + meta-narration opener fix shipped 6/4; the 6/5-staged trio deployed 6/11 (ce6e14d) — anti-fabrication + rubric pinned on both selection paths, the 8-angle angle menu as a retrievable principles section (embeddings regenerated, 21→22 sections), 388M stat scrub (upstream Google Docs also fixed; "owed source commits" closed as moot). Validated in production via URL-anchored STPC draft. First recurring-surrogate profile entry (Boykin) added to STPC profile + mirror (8d062c7).
In flight
$$500/mo
Zapier Retirement
Currently $500/mo. Pushes WinRed/Revv donors → Email Octopus (deployment) and lets Doug pull data into Salesforce. Replace with S3-custom-reports path driving both EO and SF directly. Saves the cost and removes a vendor.
Retiring
Live 2026-06-16
Tasks-on-Netlify-Blobs Migration
Killed Apps Script. Drop / Tasks / Comments + Recently Launched + the Scribe corpus cron now run on Netlify Blobs; reads sub-500ms and the recurring break class is gone. Built → tested (22/22) → imported (275 drops / 55 tasks / 24 comment threads) → hard cutover → live, all in one session. New: `callBackend`/`blobs-backend` seam, `drops-by-client` cron endpoint, recovery surface (`admin.html`), browsable Drops Ledger (`ledger.html`). Decisions: Mark Sent gated on Drop=Approved; `listMyTasks` approver clause; `getBySlug` spine fallback.
Live
Live 2026-06-30
Courier Agency List → Netlify Blobs Migration
Finished the job Tasks started — moved Courier's agency roster + sender lists off the last live Apps Script dependency (the separate Master Agency List Sheet, whose intermittent slowness caused blank Agencies steps + missing sender dropdowns). `agency-list.js` now reads Blobs-first with the sheet as fallback (safe parallel build, zero-risk flip). New: `_agency-lib/agency-store.js`, Adam-gated `agency-admin-api.js` (CRUD + one-click seed/rollback), form-based `agency-editor.html` in Back Porch. UX: paired Slack-mention rows; "Allyz URL" → "Submission portal URL". Round-trip parity verified; seeded live (29 agencies / 25 senders); confirmed `source:blobs`.
Live
In flight
Cancellation-Delta Subs Pilots
Solve the WinRed recurring subscriptions data problem with a lighter approach than manual lifetime exports.
In flight
Deferred
WinRed Subs Build
Same problem as cancellation-delta pilots above, heavier solution. May be obsolete if pilots work.
Deferred
15%Started
Client-Facing Report Generator
Today every client report is hand-built HTML with numbers pasted in manually. Make it automatic so every TMP client gets a fresh report subdomain with zero manual work.
Not started
Resolved
Bridge Form-Factor Reconsideration
Bridge writes data into Google Docs for human briefing. Doesn't work for Scribe, which needs queryable structured data, not narrative prose. Form needs to change or get supplemented.
Resolved
Parallel
Doug's SF Custom-Reports Pipeline
When live + EO Donor Flow Lambda lives, Zapier middle-layer on donation flow becomes retirable.
Parallel
Pending
EO Donor Flow Lambda
Pipe Email Octopus engagement data (opens, clicks) into the data layer for join with donations. Foundation for the Donor 360 layer.
Scaffolded
04
Sheet
A.04 · Punch List

Decisions pending – the calls that unlock progress.

4 active
+ 2 resolved 2026-05-17
D1 ✓
Scribe un-park · RESOLVED 2026-05-17 evening
The 3-option trichotomy (strip prompt back · swap base model · continue park) collapsed into Phase 0 prompt rework + Phase 0.5 model toggle running in parallel. Phase 0 IS the un-park.
D2 ✓
Bridge form factor for Scribe brain feed (Bucket 3) · RESOLVED 2026-05-17 evening
Bridge stays as Google Doc for human briefing only; Scribe brain feed (Bucket 3) goes on a separate queryable layer via the Phase 1 corpus pipeline. No reframe of Bridge needed.
D3
TMA-on-H4H 82% override codification
Add AGENCY_OVERRIDES map / new Agencies-tab column, or leave as Adam-fixes-slider-manually.
Blocks Courier pitch draft accuracy (slider seeds at 80% even for TMA packages).
D4
Tier 1 manual-vs-computed precedence
Retire manual Agencies-tab flags, retire agency-top.js computation, or define explicit precedence.
Blocks Courier Tier 1 correctness.
D5
WinRed subs architecture commitment
Cancellation-delta pilot path, or manual lifetime export path.
Awaiting pilot data (H4H + Cameron pilots configured but not yet fired).
D6 ✓
Tasks-on-Blobs sprint kickoff timing · RESOLVED 2026-06-16
Built + cut over live in one session, triggered by the recurring Apps Script break. Apps Script dormant pending post-watch deletion.
A.x
Appendix
Reference

Why each piece exists.

for when you need
to look it up
Every donation TMP raises lands as a CSV in S3. A Lambda turns those CSVs into clean canonical Parquet files. Tinybird ingests Parquet into one fast query layer. Every TMP tool that needs to see numbers asks Tinybird the same way and gets the answer in under 100ms. Salesforce holds donor identity and CRM workload separately because that's a different job than analytics.
S3
The front door. Every donation report from every platform lands here first.
ParquetPipeline Lambda
The translator. Makes Revv and WinRed look the same downstream.
Canonical Parquet
The safety net. Our own vendor-neutral copy of every donation.
Tinybird
The fast brain. One always-warm query layer everyone hits the same way.
Salesforce
Donor identity. Knows people, not events.
Apps Script + Sheets
Legacy storage for Drops/Tasks. Slow. Being retired.
Netlify Blobs
Replacement storage. Already in the stack.
EO / Wondercave / Twilio
Delivery pipes. Email, P2P SMS, future AM alerts.