Initial skills documentation — 25 categories, all SKILL.md + references + scripts
This commit is contained in:
root
@@ -0,0 +1,26 @@
|
||||
# Avis Email Verification Code Workflow
|
||||
|
||||
When a service (Avis, bank, etc.) emails a one-time security code, the agent can read the inbox faster than the user can find the email.
|
||||
|
||||
## Pattern: Monitor inbox for verification code
|
||||
|
||||
1. User triggers login at avis.com → email goes out
|
||||
2. Agent checks inbox for "security code" from Avis
|
||||
3. Extracts the code and tells the user
|
||||
4. User enters it — logged in
|
||||
|
||||
The email content is HTML-only (no text/plain alternative). The code itself may be embedded in invisible HTML constructs — extract from body text after stripping tags.
|
||||
|
||||
## Confirmations: Avis reservation chain
|
||||
|
||||
After login, the user will click through to book. Avis sends immediate confirmations:
|
||||
1. Avis sends "Reservation Confirmation" email — extract reservation #, vehicle, pick-up/drop-off, total
|
||||
2. Agent reports the key details back to user
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Avis uses email-based login — the security code goes to the user's inbox, making the agent's IMAP access essential
|
||||
- Avis For Business sends separate password reset emails — the password reset email from "Avis For Business" is a different flow from the personal account login. Distinguish by subject line
|
||||
- Verification code emails have near-identical subjects — "Avis: Your security code" multiple times. The last one (highest UID) is the current login attempt
|
||||
- Ford Explorer is the default mid-size SUV — Avis frequently assigns Ford Explorer or similar for mid-size SUV bookings
|
||||
- Confirmation numbers — Avis reservations starting with #1958XXXXUSX pattern. Store both reservation #s when booking a round-trip-two-reservation itinerary
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
# Global Entry — Customs Return Procedure
|
||||
|
||||
When a user returns to the US from an international trip and has Global Entry, use this reference for the customs clearance section of the itinerary.
|
||||
|
||||
## What Global Entry does
|
||||
|
||||
- Bypasses the regular CBP line at US airports
|
||||
- Self-service kiosk: scan passport, fingerprint scan, answer customs declaration → printed receipt → hand to officer
|
||||
- Time: ~2 min per person, ~10-15 min total for family
|
||||
|
||||
## Key rules
|
||||
|
||||
- **First port of entry clears customs.** Even if the final destination is elsewhere (e.g. fly MCO, drive to Savannah), customs is at MCO. The itinerary timeline must account for this.
|
||||
- **Children under 12** go through Global Entry with a parent. No separate enrollment needed.
|
||||
- **Global Entry cards** are optional — passport scanning at the kiosk works.
|
||||
- **No pre-departure form required.** Unlike Mobile Passport Control (MPC), GE has no app to fill out beforehand.
|
||||
|
||||
## Timeline placement
|
||||
|
||||
In a return-day itinerary timeline:
|
||||
|
||||
```
|
||||
1:44 PM Arrive MCO
|
||||
1:44 PM Deplane, follow signs to US Customs
|
||||
1:50 PM At Global Entry kiosk area
|
||||
2:00 PM Through customs (~10 min total for 4)
|
||||
2:15 PM Shuttle/bus to rental car lot
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **No app needed.** The user asked "is there anything we need to fill out before landing" — answer is no for Global Entry. MPC app is optional backup only.
|
||||
- **Duty-free limit:** $800/person. With carry-ons only, unlikely to be exceeded.
|
||||
- **No restricted agricultural products** from most standard tourist destinations. Advise user to check CBP.gov for destination-specific restrictions.
|
||||
- **GE + PreCheck = two separate programs.** PreCheck is airport security (TSA); GE is customs (CBP). Having both covers both sides of the trip.
|
||||
- **Known Traveler Number (KTN)** is needed on the flight reservation for PreCheck to print on the boarding pass. Global Entry members automatically get a KTN — it's the PASSID number on the back of the GE card.
|
||||
@@ -0,0 +1,66 @@
|
||||
# Inbox Solicitation Triaging — Design & Pitfalls
|
||||
|
||||
Pattern for cleaning an inbox by classifying and moving solicitations.
|
||||
|
||||
## Core Technique
|
||||
|
||||
1. **Connect via IMAP (imaplib)**, select INBOX, fetch ALL messages
|
||||
2. **Iterate each message** with `BODY.PEEK[]` (non-destructive read — doesn't mark as seen)
|
||||
3. **Run multi-heuristic solicitor detection** on sender domain, subject line, and body text
|
||||
4. **Copy flagged to target folder**, then `+FLAGS \\Deleted` + `EXPUNGE` from INBOX
|
||||
5. **Save detailed JSON report** and print a grouped summary
|
||||
|
||||
## Multi-Heuristic Detection Strategy
|
||||
|
||||
Four layered checks:
|
||||
|
||||
### 1. Allowlist Bypass (highest priority)
|
||||
- **Internal senders** (first-name patterns, company domain): skip entirely
|
||||
- **Legit domains** (`boxpilotlogistics.com`, etc.): skip entirely
|
||||
|
||||
### 2. Subject-Line Keywords (high signal)
|
||||
Immediate giveaways: "insurance quote", "free quote", "factoring", "compliance", "load board", "mc number", "advertisement"
|
||||
|
||||
### 3. Body Content Analysis (by sender domain class)
|
||||
|
||||
**Free/personal domains** (gmail, yahoo, outlook, etc. — defined in FREE_DOMAINS set):
|
||||
- If sender has free domain AND references an external business domain in the body → strong solicitation signal
|
||||
- Also scan for solicitation keywords + phrases in body text
|
||||
|
||||
**Business-domain senders** (not free, not internal):
|
||||
- Count multiple trucking-solicitation signals (keywords + phrases)
|
||||
- Threshold: 2+ signals = solicitation
|
||||
|
||||
**Transactional invoice guard:** If subject/body contains ≥2 invoice keywords ("invoice", "receipt", "payment confirmation") and no other signals triggered, keep in inbox.
|
||||
|
||||
### 4. Combined Signal Accumulation
|
||||
- Body keywords: +1 each
|
||||
- Solicitation phrases: +2 each (trucking-specific phrases are stronger)
|
||||
- Subject keyword match: +1
|
||||
- Threshold: 2+ total = flagged
|
||||
|
||||
## Known False-Positive Categories
|
||||
|
||||
| Category | Why flagged | Typical senders | Action |
|
||||
|---|---|---|---|
|
||||
| Government registration notices | Body has "fmcsa", "usdot", "authority" | ucr.gov, login.gov, dot.gov | Move back — official notices |
|
||||
| Bank product promos | Boilerplate contains "eld", "limited time" | americanexpress@blueprint.americanexpress.com | Review individually |
|
||||
| Slack onboarding | Template has "eld", "limited time" in footer | no-reply@email.slackhq.com | Keep transactional, move promos |
|
||||
| FMCSA process agent confirmations | Body has "boc-3", "process agent", "fmcsa" | fmcsaprocessagents.com | Keep if service was ordered |
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **HTML boilerplate noise**: Some email templates (Amex, Slack) contain trucking keywords like "eld" in their footer HTML. The heuristic catches them as false positives. Mitigation: whitelist known transactional templates or strip common boilerplate before matching.
|
||||
- **UCR.gov is legitimate** but its subject/body triggers "compliance", "authority", "usdot". These are official registration notices, not solicitations.
|
||||
- **BODY.PEEK[] vs BODY[]**: Always use PEEK to avoid marking messages as read during review.
|
||||
- **IMAP COPY + STORE \\Deleted** is the portable move pattern. Some servers support `MOVE` but not all.
|
||||
- **conn.expunge()** must be called after all deletions or the messages remain hidden but not freed.
|
||||
- **Character encodings**: Fall back through `utf-8` → `iso-8859-1` → `windows-1252` when `get_content_charset()` returns None.
|
||||
- **HTML-only emails**: Need regex tag stripping (`re.sub(r'<[^>]+>', ' ', body)`) before keyword matching. CSS class names (`style-Xia0GT4t_-text`) are noise — filter them out.
|
||||
|
||||
## Script
|
||||
|
||||
See `scripts/inbox-solicitation-cleaner.py` — a standalone runnable script with the full heuristic set. Adjust the Config block (host, credentials, folder name) and run:
|
||||
```bash
|
||||
python3 scripts/inbox-solicitation-cleaner.py
|
||||
```
|
||||
@@ -0,0 +1,54 @@
|
||||
# PDF Attachment Extraction from Multipart/Mixed Emails
|
||||
|
||||
Airline ticket receipts (Copa, American, United, Delta) often embed PDF attachments inside `multipart/mixed` MIME messages alongside inline images (logos, QR codes).
|
||||
|
||||
## Key pattern
|
||||
|
||||
The PDF is typically the **last meaningful attachment** in the MIME tree, surrounded by 6-8 `image/jpeg` inline images. Don't assume the first `.pdf` match is the right one — iterate all parts and pick the one with the correct filename.
|
||||
|
||||
```python
|
||||
for part in raw.walk():
|
||||
fn = part.get_filename()
|
||||
if fn and fn.endswith('.pdf'):
|
||||
payload = part.get_payload(decode=True)
|
||||
# Save to disk
|
||||
with open("/tmp/ticket.pdf", 'wb') as f:
|
||||
f.write(payload)
|
||||
# Or parse directly from memory
|
||||
from pdfminer.high_level import extract_text
|
||||
from io import BytesIO
|
||||
text = extract_text(BytesIO(payload))
|
||||
```
|
||||
|
||||
## Parsing e-ticket PDFs with pdfminer
|
||||
|
||||
Install: `pip3 install pdfminer.six`
|
||||
|
||||
```python
|
||||
from pdfminer.high_level import extract_text
|
||||
from io import BytesIO
|
||||
|
||||
text = extract_text(BytesIO(pdf_bytes))
|
||||
|
||||
# Extract flight numbers (Copa uses CM###)
|
||||
for line in text.split('\n'):
|
||||
if 'Flight Number' in line or 'CM ' in line:
|
||||
flights.append(line.strip())
|
||||
```
|
||||
|
||||
## Common airline PDF fields
|
||||
|
||||
| Airline | Flight prefix | Confirmation length | PDF attachment name |
|
||||
|---|---|---|---|
|
||||
| Copa | CM | 6 alphanumeric (A4GSFD) | ETKT_*.pdf |
|
||||
| American | AA | 6 alphanumeric | e-ticket-*.pdf |
|
||||
| United | UA | 6 alphanumeric | itinerary-*.pdf |
|
||||
| Delta | DL | 6 alphanumeric | eTicket*.pdf |
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Copa tickets have 6-8 inline image parts** (logo, QR, barcode, social icons) before the PDF. The PDF is typically the second-to-last or last attachment.
|
||||
- **`get_content_charset()` may return None** on PDFs — use `utf-8` or `iso-8859-1` fallback for the PDF parsing step (the PDF itself is binary, the charset applies to the text portion of the email, not the attachment).
|
||||
- **Copa confirmation codes look like A4GSFD** — 6 chars, alphanumeric. Found in the email subject line and the PDF body. Also stored in the email's `X-Confirmation` header (if present).
|
||||
- **Header-only fetch** (`BODY.PEEK[HEADER.FIELDS]`) won't show attachment filenames. You need to fetch the full message body to find PDFs.
|
||||
- **Some airlines use `Content-Type: application/octet-stream`** instead of `application/pdf` for PDF attachments. Check `get_filename()` regardless of content type.
|
||||
@@ -0,0 +1,150 @@
|
||||
# Travel Itinerary Building from Emails
|
||||
|
||||
This pattern covers extracting flight/hotel/car rental data from IMAP emails and assembling a structured itinerary document.
|
||||
|
||||
## Typical sources
|
||||
|
||||
| Type | Senders | Data format |
|
||||
|---|---|---|
|
||||
| Flights | Copa Airlines, United, American, Delta, Allegiant | PDF attachment in multipart/mixed email |
|
||||
| Hotel/Airbnb | Airbnb, Booking.com, VRBO | HTML email with dates, address, confirmation |
|
||||
| Car rental | Avis, Hertz, Enterprise, Budget | HTML email with confirmation #, vehicle class |
|
||||
| Account numbers | Avis account statements | HTML email with Wizard/Preferred number |
|
||||
|
||||
## Extraction flow
|
||||
|
||||
1. **Search inbox** — broad OR search across senders and subject keywords
|
||||
2. **Filter** — skip survey/feedback/survey emails (Allegiant sends many post-flight surveys)
|
||||
3. **Extract PDFs** — for airline tickets, extract the PDF attachment from multipart/mixed (see `references/pdf-attachment-extraction.md`)
|
||||
4. **Parse with pdfminer** — extract flight numbers, times, dates, passenger names
|
||||
5. **Build structured data** — leg-by-leg table with date, time, airport codes, flight numbers
|
||||
6. **Search for ancillary bookings** — rental car, hotel, insurance in the same inbox
|
||||
|
||||
## Email search strategies for trip data
|
||||
|
||||
**Multiple sender search (one call per sender):**
|
||||
```python
|
||||
for sender in ["Copa", "Avis", "Allegiant", "Airbnb"]:
|
||||
status, data = conn.search(None, f'FROM "{sender}" SUBJECT "itinerary"')
|
||||
```
|
||||
|
||||
**Keyword OR across subject lines:**
|
||||
```python
|
||||
status, data = conn.search(None, 'OR OR SUBJECT "Cartagena" SUBJECT "flight" SUBJECT "itinerary"')
|
||||
```
|
||||
|
||||
**Chain multiple filters:**
|
||||
```python
|
||||
status, data = conn.search(None, 'FROM "Copa" OR SUBJECT "Cartagena" BODY "Cartagena"')
|
||||
```
|
||||
|
||||
## Account number extraction (Avis Wizard #, etc.)
|
||||
|
||||
Account numbers are often embedded in HTML email statements rather than PDFs. The Wizard number may be partially redacted in the HTML (`P72***`). If the user has past rentals, the confirmation email will show the full account number in the header or body.
|
||||
|
||||
**Method:** Search the raw HTML for the account section:
|
||||
```python
|
||||
idx = html.find('Wizard')
|
||||
if idx >= 0:
|
||||
# Extract text between Wizard #: and next HTML tag
|
||||
m = re.search(r'Wizard\s*#[:\\s]*<span[^>]*>(.*?)</span>', html[idx:idx+300], re.DOTALL)
|
||||
```
|
||||
|
||||
For Avis Wizard numbers specifically: the HTML statement email shows the number partially redacted (e.g. `P72***`). The full number appears on the Avis website after login. If the user needs to book, they'll need to get it from their Avis account or a past confirmation email.
|
||||
|
||||
## Itinerary document structure
|
||||
|
||||
Use a consistent markdown template:
|
||||
|
||||
```markdown
|
||||
# [Destination] Trip — [Date Range]
|
||||
|
||||
**Travelers:** [Names with ages if relevant]
|
||||
|
||||
---
|
||||
|
||||
## Flight Schedule
|
||||
|
||||
### [Traveler Names] — [Airline] (Confirmation: [CODE])
|
||||
|
||||
| Date | Leg | Flight | Departs | Arrives | Duration |
|
||||
|---|---|---|---|---|---|
|
||||
|
||||
---
|
||||
|
||||
## Departure Day — [Date]
|
||||
|
||||
### Time-based Agenda
|
||||
|
||||
| Time | Activity | Notes |
|
||||
|---|---|---|
|
||||
|
||||
---
|
||||
|
||||
## Activities
|
||||
|
||||
- Attractions categorized as Must-See, Kid-Friendly, Adult, Rainy Day
|
||||
- Estimated time per attraction
|
||||
- Location relative to accommodation
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Items specific to this trip
|
||||
```
|
||||
|
||||
## Travel plan adjustments
|
||||
|
||||
When the user provides corrections to the itinerary (all on same flight, TSA PreCheck, no checked bags, gate arrival 90 min before boarding), apply them in order:
|
||||
|
||||
1. **Same flight for all** — update the flight schedule table to list all travelers under one confirmation
|
||||
2. **TSA PreCheck** — adjust MCO arrival: PreCheck average wait ~10-15 min vs 30-45 min standard. Can shift airport arrival 30 min later.
|
||||
3. **No checked bags** — remove bag claim step from timeline. Carry-ons go straight to security after shuttle.
|
||||
4. **Gate 90min before departure** — work backward from boarding time (~30 min before departure). Add 30 min buffer for food/restroom. This sets the "at security" time, then subtract PreCheck estimate to set "arrive airport" time.
|
||||
5. **No rental car in destination** — note as "taxis/ride-share from airport" in the itinerary. Research Avis options as reference data only, document clearly that no booking is needed.
|
||||
6. **Car rental: monitor inbox for confirmations** — the user may start the Avis rental process themselves (email verification code login). Search inbox for "Avis" since last 24 hours for confirmation emails. Avis sends multiple emails per booking (security code, then reset password, then confirmation). The confirmation email contains the reservation number, vehicle class, pick-up/drop-off locations and times, and estimated total. Extract and add to the itinerary.
|
||||
|
||||
## Real-time inbox monitoring during trip planning
|
||||
|
||||
When the user says "check email now for [rental/flight/hotel]" during active trip planning:
|
||||
|
||||
1. Search inbox for the relevant sender since last 24h: `conn.search(None, f'(FROM "Avis" SINCE {since})')`
|
||||
2. Multiple emails from the same sender in the same session likely form a chain: security code → password reset → confirmation
|
||||
3. The confirmation email is the most important — extract reservation number, dates, vehicle class, and any cost
|
||||
4. Report findings concisely in a table
|
||||
5. Update the itinerary checklist to mark the booking as completed
|
||||
6. Remove the associated todo item from the trip planning checklist
|
||||
|
||||
## Drive time estimation
|
||||
|
||||
For Savannah → MCO (July 11, Saturday, ~285 miles):
|
||||
|
||||
| Condition | Time | Notes |
|
||||
|---|---|---|
|
||||
| No traffic | 4h 15m | Typical late-night / early morning |
|
||||
| Light traffic | 4h 30m | Leave before 9 AM Saturday |
|
||||
| Moderate | 5h | Jacksonvlille I-95 afternoon backup |
|
||||
| Heavy | 5h 30m | Holiday weekend, accident, or rain |
|
||||
|
||||
- Saturday I-95 southbound through Jacksonville typically backs up around lunchtime (11 AM - 1 PM)
|
||||
- Best departure: 8:30 AM from Savannah → arrives MCO ~1:30 PM with buffer
|
||||
- Worst case: leave later than 9 AM, hit Jacksonville lunch rush, arrive MCO 2:30+ PM
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Allegiant sends survey/feedback emails** for 48+ hours after each flight. Exclude these by filtering out subject lines containing "feedback" or "survey".
|
||||
- **Allegiant flight confirmation details are in HTML tables** — the FROM/TO airport codes appear inline (e.g. "SAV to CAK") rather than in structured fields. Parse these with regex.
|
||||
- **Copa confirmation emails are in two parts** — 1) a "Your reservation is processing" email, then 2) the "Electronic ticket receipt and confirmation" with the actual PDF. The processing email has the confirmation code but no flight details.
|
||||
- **Copa flights depart from MCO, not SAV** — Copa flies out of Orlando (MCO) for Cartagena. The user drives from Savannah to MCO. If searching for the user's flight and finding Copa, check the departure airport — it may not be their home city.
|
||||
- **Carrier gateways are unreliable for SMS delivery** — AT&T's `@txt.att.net` gateway was shut down. Don't use carrier gateways for automated messages unless confirmed working.
|
||||
- **Anita and Germaine were booked on Copa together** — same confirmation code (A4GSFD) produces two separate PDFs, one per passenger. Download both: the text extraction gives the same data repeated with different names.
|
||||
- **Avis Wizard number is partially redacted** in account statement emails. The last 3 characters show as `***`. For booking purposes, the user must retrieve the full number from their Avis account profile page.
|
||||
- **One-way rental from Savannah to MCO**: Avis allows one-way rentals. Pick up at Savannah location, drop at MCO airport. Need full Wizard # for Preferred Plus benefits.
|
||||
- **Copa flight PDFs contain 6-7 inline image attachments** alongside the ticket PDF. The PDF is the last or second-to-last MIME part in multipart/mixed.
|
||||
- **Global Entry on return: MCO is first port of entry.** Though final destination is Savannah, US customs is cleared at Orlando. Account for ~10-15 min GE kiosk time in the return timeline.
|
||||
- **Avis sends multiple emails per booking in a chain**: security code → (optional) password reset → confirmation. The confirmation email arrives last and contains the reservation number, vehicle class, and pricing. Search `FROM "Avis"` since the last 24h to find all of them.
|
||||
- **Avis confirmation has a "Ford Explorer or Similar"** as default for mid-size SUV. The reservation number format is `#XXXXXXXXUSX` (9 digits + US + 1 digit).
|
||||
- **Avis one-way rental allowed**: pick up at Avis Savannah (SAV), drop at MCO. No extra fee.
|
||||
- **Avis verification code emails** have subject "Avis: Your security code." The code appears in the email body. The agent can read it from the inbox before the user finds it.
|
||||
- **Avis confirmation email arrives last** in a chain: security code → (optional) password reset → confirmation. Search FROM "Avis" since last 24h to get all of them.
|
||||
@@ -0,0 +1,34 @@
|
||||
# Verification Code Reader
|
||||
|
||||
When a service sends a verification code by email (Avis, banks, accounts), the agent can read it from the inbox before the user can find it manually.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. User triggers a login that sends a code email (e.g. entering email on avis.com)
|
||||
2. User says "check email for code" or the workflow expects the code
|
||||
3. Search IMAP for the security-code email from that sender
|
||||
4. Extract the code and relay it
|
||||
|
||||
## Finding the code in the email
|
||||
|
||||
Security-code emails are typically text-only or minimal HTML. The code itself can be:
|
||||
- In the subject line (rare)
|
||||
- In the body as a bold or large number after "security code" or "verification code"
|
||||
- In a table row labeled "Your Security Code"
|
||||
|
||||
## Searching strategy
|
||||
|
||||
```python
|
||||
# Search recent emails from the sender
|
||||
status, data = conn.search(None, f'(FROM "avis@e.avis.com" SINCE "{since}")')
|
||||
```
|
||||
|
||||
The sender is typically:
|
||||
- Avis: `avis@e.avis.com` — subject "Avis: Your security code"
|
||||
- Other services follow similar patterns
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Security codes expire fast** — most are valid for 5-15 minutes. Don't over-optimize search patterns, just grab the latest email from that sender.
|
||||
- **HTML emails may obfuscate the code** — if you can't find a numeric pattern, strip all tags and look for "code" context: `re.search(r'code[:\s]*(\d{4,8})', clean_body, re.IGNORECASE)`
|
||||
- **One-time passwords (OTP) vs link-based auth** — some services send a magic link instead of a numeric code. If no numeric pattern matches, check for a URL containing `token=` or `code=` in the body.
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
# WordPress SMTP Credential Discovery
|
||||
|
||||
When a site uses WP Mail SMTP (or similar) plugin, credentials are stored in the `wp_options` table as a serialized PHP array under `wp_mail_smtp`.
|
||||
|
||||
## Query Pattern
|
||||
|
||||
```sql
|
||||
SELECT option_value FROM wp_options WHERE option_name = 'wp_mail_smtp';
|
||||
```
|
||||
|
||||
The value is a PHP serialized array. Extract fields with:
|
||||
```bash
|
||||
mysql -h 127.0.0.1 -P <PORT> -u <USER> -p'<PASS>' <DB> \
|
||||
-e "SELECT option_value FROM wp_options WHERE option_name = 'wp_mail_smtp'" \
|
||||
--skip-column-names 2>/dev/null | python3 -c "
|
||||
import sys, re
|
||||
data = sys.stdin.read()
|
||||
# PHP serialized array — extract key fields via regex
|
||||
for key in ['mailer', 'host', 'port', 'user', 'pass']:
|
||||
match = re.search(r's:4:\"$key\";s:(\d+):\"(.*?)\"', data)
|
||||
if match: print(f'{key}: {match.group(2)}')
|
||||
"
|
||||
```
|
||||
|
||||
Or parse the PHP serialized format more robustly with Python (the `php-serialize` library, or use the regex approach above for the specific fields needed).
|
||||
|
||||
## Common Locations
|
||||
|
||||
| Site | DB Name | Host | Port | User | How to access |
|
||||
|---|---|---|---|---|---|
|
||||
| Apex Track Experience | `apextrackexperience_1781549652` | c1113726.sgvps.net:2525 | 2525 | `contact@apextrackexperience.com` | MySQL tunnel on Core (`127.0.0.1:33060`) |
|
||||
| Any WordPress site on wphost02 | varies | varies | varies | varies | SSH to wphost02, query local MySQL |
|
||||
|
||||
## MySQL Tunnel Access
|
||||
|
||||
On Core, the Apex Track Experience database is accessible via SSH tunnel through wphost02:
|
||||
```bash
|
||||
mysql -h 127.0.0.1 -P 33060 -u <USER> -p'<PASS>' <DB> -e "SELECT 1"
|
||||
```
|
||||
|
||||
The tunnel is maintained by `mysql-tunnel` systemd service (Core → wphost02:3306).
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- The `wp_mail_smtp` option stores the password as plaintext (not hashed) in the PHP serialized array
|
||||
- The `wp_mail_smtp_initial_version` and `wp_mail_smtp_version` options confirm the plugin version
|
||||
- WPForms uses the same WP Mail SMTP plugin for email — form notification emails are sent via these same credentials
|
||||
- The MySQL password in the connection string is also in plaintext in systemd unit files — keep unit files `chmod 600`
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
# WPForms Email Delivery Verification
|
||||
|
||||
Pattern for checking whether WPForms confirmation emails were actually delivered to form submitters, by cross-referencing IMAP inbox, bounce messages, WordPress DB entries, and DNS records.
|
||||
|
||||
## Overview
|
||||
|
||||
WPForms sends two categories of email on form submission:
|
||||
|
||||
1. **Admin notification** → sent to site admin/owner (e.g. `contact@domain.com`). These appear in the admin IMAP inbox.
|
||||
2. **Confirmation to registrant** → sent to the person who filled the form. These go to external email addresses and may bounce.
|
||||
|
||||
## Verification Flow
|
||||
|
||||
### Step 1: Identify form IDs
|
||||
|
||||
Query the WordPress DB to discover which forms are active and their IDs:
|
||||
|
||||
```sql
|
||||
SELECT ID, post_title FROM wp_posts
|
||||
WHERE post_type='wpforms' AND post_status='publish'
|
||||
ORDER BY ID;
|
||||
```
|
||||
|
||||
Common conventions: form ID 268 = waiver, 270 = paid registration (varies per site).
|
||||
|
||||
### Step 2: Get form entries from the DB
|
||||
|
||||
```sql
|
||||
SELECT entry_id, date, status, LEFT(fields,800)
|
||||
FROM wp_wpforms_entries WHERE form_id=270
|
||||
ORDER BY entry_id;
|
||||
```
|
||||
|
||||
Extract registrant names and emails from the `fields` JSON column. The JSON structure uses numbered field IDs as keys; each has a `value` and an `email` field.
|
||||
|
||||
### Step 3: Check admin notifications in IMAP inbox
|
||||
|
||||
Connect to the site's IMAP inbox and search for WPForms submission notifications:
|
||||
|
||||
```python
|
||||
conn.search(None, f'(SINCE "{yesterday}")')
|
||||
# Look for messages FROM the site's email with subjects containing
|
||||
# registration, waiver, or the form title
|
||||
```
|
||||
|
||||
Admin notifications confirm the form was submitted and the pipeline triggered.
|
||||
|
||||
### Step 4: Check for bounce messages
|
||||
|
||||
Search for mailer-daemon / postmaster bounces:
|
||||
|
||||
```python
|
||||
conn.search(None, 'OR FROM "mailer-daemon" FROM "postmaster"')
|
||||
```
|
||||
|
||||
For each bounce, extract the original recipient address from the body:
|
||||
|
||||
```
|
||||
Delivery to the following recipient failed permanently: user@gmail.com
|
||||
Technical details: 550-5.7.26 Sender unauthenticated (SPF/DKIM)
|
||||
```
|
||||
|
||||
### Step 5: Cross-reference bounces with registrants
|
||||
|
||||
Match bounce recipient addresses against registrant emails from Step 2. These are the registrants whose confirmations didn't arrive.
|
||||
|
||||
### Step 6: Trace SPF/DKIM root cause
|
||||
|
||||
From the bounce message body, extract the sending IP reported by the receiving MTA:
|
||||
|
||||
```
|
||||
SPF [domain.com] with ip: [X.X.X.X] = did not pass
|
||||
```
|
||||
|
||||
Then check:
|
||||
|
||||
```bash
|
||||
# Current SPF record
|
||||
dig +short TXT domain.com | grep spf
|
||||
|
||||
# Current DKIM
|
||||
dig +short TXT default._domainkey.domain.com
|
||||
|
||||
# Current DMARC
|
||||
dig +short TXT _dmarc.domain.com
|
||||
```
|
||||
|
||||
**Common failure:** WP Mail SMTP sends through a relay (e.g. SiteGround `c*.sgvps.net`) which resolves to an IP not in the SPF record. Check the SMTP relay IP:
|
||||
|
||||
```bash
|
||||
dig +short c1113726.sgvps.net
|
||||
```
|
||||
|
||||
Then verify that IP is in the SPF include chain. If not, add it.
|
||||
|
||||
### Step 7: Assess remaining registrants
|
||||
|
||||
If no bounce was received for a registrant, the confirmation likely reached them. Limitations:
|
||||
- Some MTAs silently drop (no bounce) if the mailbox is full
|
||||
- Some spam filters never bounce
|
||||
- Only Gmail/Outlook typically send proper bounce notifications
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Bounces from MXroute IPs** — If the WordPress site was migrated between hosting providers, old SPF records may still reference the previous provider's sending IPs. The bounce IP is the actual sending IP, not the SMTP relay hostname.
|
||||
- **WP Mail SMTP logs disabled** — The `wp_mail_smtp` option has `logs.enabled => false` by default in Lite mode. There are no sent-mail logs to check directly.
|
||||
- **Form entries include test/submitted-but-unpaid entries** — Early entries may be test submissions from the site owner with $1.00 amounts. Filter these out by checking the payment field (`amount_raw`).
|
||||
- **Admin notification vs registrant confirmation** — Just because the admin got a notification doesn't mean the registrant did. WPForms sends them separately through the same SMTP pipeline.
|
||||
- **Form field IDs vary** — The `fields` JSON uses numeric field IDs that differ per form configuration. Parse the JSON and check field names, not IDs.
|
||||
Reference in New Issue
Block a user