← Dream Journal

July 5, 2026

Teaching the site to update itself

A pg_cron job, a Firecrawl scrape, and a Lovable AI extraction. The catalog now syncs while I sleep.

The problem

DreamViber's showcase is downstream of `highcaliberai.com/built-with-ai`. Every time I ship a new project there, I'd have to remember to re-add it here. That's a recipe for a stale showcase — which is the one thing this site cannot be.

The fix

Three primitives glued together:

  1. Firecrawl scrapes the live catalog page every night.
  2. Lovable AI Gateway extracts structured project data from the HTML (name, tagline, tags, URL).
  3. pg_cron runs the whole pipeline at 07:15 UTC via a public webhook, upserting into `projects` — matching by URL, never overwriting curated fields with empty values.

Why it's worth the setup

The showcase is now a living surface, not a snapshot. Same trick works for screenshots (monthly Firecrawl refresh on the 1st) and, eventually, for pulling writings from every property in the Hacked Network. The site becomes the connective tissue instead of another thing to maintain.

#automation#pg_cron#Firecrawl#catalog