The problem
I spent eight years on the web team at Abbey Carpet, maintaining more than 600 production dealer websites, including sites on both the Abbey Carpet and Floors To Go platforms. In 2026 the company decided to close its web department and outsource the function, so the sites I worked on are expected to change or disappear.
I wanted a record of that work that would outlast the live sites. Taking screenshots by hand was not realistic: the sites I wanted to keep came to 141 subdomains with about ten custom pages each, and many shared the same stock corporate pages that I had nothing to do with.
The starting point: an audit
Before writing any code, I went through every dealer site in a spreadsheet and answered two questions: is my work obvious on this site, and should it be part of a showcase? That gave me 110 Abbey Carpet sites and 31 Floors To Go sites to capture, plus a shortlist of featured sites. The spreadsheet is the input to everything below, so the judgment calls stay with a person and the repetitive work goes to the scripts.
How it works
The pipeline is five Python scripts, each with one job:
- capture.py opens each homepage in headless Chromium through Playwright at a 1440px viewport, collects the internal links one level deep, drops anything on the exclusion list, and takes a full-page screenshot of every page that survives.
- report.py works out which subdomains are really the same website and writes a JSON manifest for each site.
- meta.py collects each dealer’s name and location for the page headers.
- derive.py turns the raw PNGs into web-ready WebP files in three sizes: a 600px card, a 1200px above-the-fold banner, and an 800px-wide full page that opens on click.
- build-portfolio.py writes a markdown entry per site into this Astro project and copies the images over. It never overwrites an existing entry unless told to, so notes I write by hand are safe when I regenerate.
Every run supports a dry run first. It loads each homepage for real and writes the list of pages it would capture and the pages it rejected, with the rule that caught each one, without saving any images. That made it possible to check the filtering on five sites before committing to a long run.
Problems worth writing down
Stock pages arriving through the back door. Several dealer sites linked to a clean-looking /carpet.aspx that redirected into the stock product catalog. Checking the link alone let those through. The fix was to check every exclusion twice, once on the link and again on the final URL after redirects. In the five-site test, the second check caught 12 catalog pages the first one missed.
Matching on path segments, not text. A rule that excludes /carpet should not throw away /carpet-cleaning-tips.aspx, which is a custom page worth keeping. Exclusions match whole path segments for that reason.
Finding duplicate sites. Some dealers run several subdomains that all show the same website. The script compares homepages using normalized page text with the digits stripped out, so two copies of a template with different phone numbers still match, plus the final host after redirects. Grouping is transitive, and I set it up to prefer missing a merge over merging two different sites. A missed duplicate is a little redundancy; a false merge hides real work. On the Abbey side, seven subdomains turned out to be aliases of other sites, which left 102 canonical sites.
Moved versus gone. A dealer that moved to its own domain is a rebrand, and the work is still live. A subdomain that redirects back to the corporate homepage means the site is gone. The pipeline records those as two different things instead of calling both “redirected.”
Two runs at once. I once started a second image conversion while the first was still going. Both wrote to the same files and left nine truncated WebP images that opened as broken pictures. Now every bulk conversion is checked with Pillow’s Image.verify() afterward, and the rule is one run at a time.
WebP has a size ceiling. WebP cannot store a dimension above 16,383px. The tallest capture in the set is a testimonials page at 29,206px, which comes to 16,226px once scaled to 800px wide, so it just fits. If a future page goes over, the script scales it down until it fits rather than cropping it.
The size trade-off
The raw captures came to 2.71 GB. The WebP output is 274 MB, about a tenth of that, and the PNGs never go into the website repository.
The 800px width for full-page images was a deliberate choice. Full pages get scrolled through, not studied, and width drives file size hard: the 1,150 interior pages come to 112 MB at 800px and roughly 200 MB at 1200px. Cards link to the large file, so it only downloads when someone asks for it.
Reusing it
Floors To Go runs on the same dealer platform, so the second brand was mostly a copy. I changed the domain in one place, added three page names that Floors To Go labels differently, and gave it its own folder on the site, since a few city names exist on both brands. The whole capture now runs as one command.
The results are on this site: Abbey Carpet dealer sites and Floors To Go dealer sites.