Anti-VRBO Playbook — Beating Vacation Rental Site Bot Protection

Test Results (Aug 1, 2026)

Site	Protection	Camofox Result	Notes
Airbnb	Cloudflare + custom	BEATEN ✓	Full page load, 5.7KB body, all content extracted. Camofox + DataImpulse proxy cleared Cloudflare with zero issues.
VRBO	Imperva/Incapsula	BLOCKED ✗	"Bot or Not?" challenge wall. Imperva blocks at the application layer even after Cloudflare passthrough. Challenge doesn't auto-resolve even after 45s wait.
Booking.com	Akamai	Untested	—
Vacasa	Unknown	Untested	—
Evolve	Unknown	Untested	—

VRBO / Imperva — What We Tried

Failed Approaches

Direct camofox navigation — Landed on "Bot or Not?" challenge page with Imperva session ID
Extended wait (45s) — Challenge regenerates but never auto-resolves
Snapshot inspection — Imperva challenge page has almost zero interactive elements; just a logo link

Why Imperva Is Harder Than Cloudflare

Cloudflare challenges happen at the edge (before the page loads) and Camoufox's C++-level fingerprint spoofing clears them
Imperva challenges happen at the application layer AFTER the page loads — they inject their own JS that runs fingerprinting on top of whatever the browser reports
Imperva's SDK (window._Incapsula_Resource) does behavioral analysis: mouse movements, scroll patterns, timing
The challenge ID changes on every request (different UUID each time), suggesting it's stateless at the edge but stateful in their backend

Strategies to Beat VRBO

Strategy 1: Headful Browser with Human Emulation (Most Promising)

Instead of headless camofox, use a real Chrome/Chromium with CDP and emulate realistic human behavior:

1. Launch Chrome with puppeteer-real-browser or nodriver
2. Navigate to VRBO
3. Inject human-like mouse movements (bezier curves, not straight lines)
4. Random scrolls and hover events
5. Wait for Imperva challenge to auto-resolve (usually 5-15s in a real browser)
6. Extract data once page loads normally

Strategy 2: VRBO API Reverse Engineering

VRBO's frontend calls internal APIs. If we can find the endpoints:

Search: vrbo.com/api/v1/search?...
Property details: vrbo.com/api/v1/properties/1313017
Calendar: vrbo.com/api/v1/properties/1313017/calendar

Many of these may have weaker bot protection than the HTML pages. Imperva primarily protects the SSR HTML; API calls may use simpler auth.

Strategy 3: Browserbase / Anchor (Cloudflare Web Bot Auth)

If VRBO opts into Cloudflare's Bot Verifier program, Browserbase/Anchor can get a signed "verified bot" token. But this only works if VRBO explicitly allows it — unlikely for a booking site.

Strategy 4: Alternative Data Sources

Since Airbnb works perfectly with camofox, and most vacation rentals cross-list:

Airbnb (working): Property details, pricing, reviews, calendar, photos
Google Travel / Hotel search: Aggregates VRBO listings
VRBO via Google cache: webcache.googleusercontent.com/search?q=cache:vrbo.com/1313017

Strategy 5: Proxy Rotation + Fingerprint Diversity

Imperva may be flagging specific DataImpulse IP ranges. Try:

Different proxy providers (SOAX, Bright Data, IPRoyal)
Different geolocations
Different Camoufox fingerprint profiles
Session-level IP rotation

Recommended Approach for Moore Sunny Daze

Short-term (now): Use Airbnb data since camofox clears it consistently. All the same content is there — Tim's listing is synced across platforms.

Medium-term (this week): Implement Strategy 1 (headful browser with human emulation) specifically for VRBO. This is the most reliable path.

Long-term (product): Build a unified scraper that tries Airbnb first, falls back to headful browser for VRBO, and caches results. This becomes the "Competitive Intelligence" feature of Beach Direct — show owners how their property compares to nearby listings on ALL platforms.

Airbnb Data We Successfully Extracted

From the Airbnb listing (camofox, Aug 1 2026):

Full property description
47 amenities list
5.0 rating across all categories (cleanliness, accuracy, check-in, communication, location, value)
4 reviews with text
Calendar availability for Aug-Sep 2026
Host: Ashley, 1 year hosting
Property photos from Airbnb CDN
Pricing: shown after date selection (requires interaction)

Next Steps

Confirm camofox beats Airbnb
Document VRBO Imperva failure
Test headful browser approach for VRBO
Test Booking.com, Vacasa, Evolve
Build unified scraper module for Beach Direct

4.6 KiB Raw Blame History