Initial skills documentation — 25 categories, all SKILL.md + references + scripts
This commit is contained in:
@@ -0,0 +1,158 @@
|
||||
---
|
||||
name: security-audits
|
||||
description: "Implement and maintain automated and periodic security audits for IT Pro Partner infrastructure — SSH attempt monitoring, package vulnerability scanning, config drift detection, port scanning, and failed login reviews."
|
||||
version: 1.1.0
|
||||
author: Sho'Nuff
|
||||
platforms: [linux]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [security, audit, monitoring, compliance, logging]
|
||||
---
|
||||
|
||||
# Security Audits
|
||||
|
||||
Automated and periodic security audits for all ITPP servers. Results should eventually be surfaced through the IT Pro Partner operations portal.
|
||||
|
||||
## Lynis nightly scan (deployed Jul 6, 2026)
|
||||
|
||||
A `lynis` audit runs nightly at 3 AM on the netcup box via `~/.hermes/scripts/lynis-scan.sh`. It compares warnings count against the previous run and only alerts on NEW findings — silent if nothing changed.
|
||||
|
||||
**Installation:**
|
||||
```bash
|
||||
apt-get install lynis
|
||||
lynis audit system --quick # initial baseline
|
||||
```
|
||||
|
||||
**Cron setup:**
|
||||
- Schedule: `0 3 * * *`
|
||||
- Script: `lynis-scan.sh` (no_agent mode)
|
||||
- Alert: only on warning count change, via Telegram
|
||||
- Next step: install on wphost02 and Tony's box
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
lynis audit system --quick | grep -E "Warnings|Hardening|Score"
|
||||
grep "^warning\[\]" /var/log/lynis-report.dat
|
||||
```
|
||||
|
||||
**Current baseline (Jul 6, 2026):**
|
||||
- Netcup box: Hardening index 62/100, 3 warnings (1 vulnerable package, 2 DNS)
|
||||
- wphost02: Hardening index 63/100, 3 warnings
|
||||
|
||||
| Type | Frequency | What it checks |
|
||||
|---|---|---|
|
||||
| **Automated** | Every tick | Open ports, recent failed SSH logins, disk usage alerts, service status |
|
||||
| **Periodic** | Daily / Weekly | Package vulnerability scan, config drift vs known-good baseline, SSL cert expiry, user account audit |
|
||||
|
||||
## Wazuh vs simple cron approach
|
||||
|
||||
The user asked about Wazuh for security monitoring. Honest assessment:
|
||||
|
||||
**Wazuh pros:** Covers vulnerability scanning (CVEs), file integrity monitoring, intrusion detection (HIDS), log correlation, compliance checks (CIS), centralized dashboard — all in one.
|
||||
|
||||
**Wazuh cons:** Needs ~4GB RAM + 2 vCPU for the manager (CPX21 class). Takes a full day to deploy properly. Replaces the need for Graylog entirely since it handles log management.
|
||||
|
||||
**Simple cron approach (what we'd start with):** `lynis` weekly audits, SSH auth.log monitoring, automated port scans. Takes 30 minutes. Covers immediate gaps until Wazuh is ready.
|
||||
|
||||
**Recommendation:** Start with the cron approach for quick wins, plan Wazuh for when the portal goes customer-facing and compliance matters.
|
||||
|
||||
## Audit scope (per server)
|
||||
|
||||
### SSH audit
|
||||
- Check `/var/log/auth.log` or `journalctl -u ssh` for failed login attempts in last 24h
|
||||
- Count unique IPs attempting to brute force
|
||||
- Check for non-root user accounts with shell access
|
||||
- Verify `PermitRootLogin` and `PasswordAuthentication` settings
|
||||
|
||||
### Package audit
|
||||
- `apt list --upgradable` for known security patches
|
||||
- Check for unattended-upgrades status
|
||||
- Verify kernel version matches expected image
|
||||
|
||||
### Service audit
|
||||
- Check that critical services (Hermes gateway, cron, email) are running
|
||||
- Verify Tailscale status
|
||||
- Check Docker containers are running (if applicable)
|
||||
|
||||
### Network audit
|
||||
- Scan for unexpected open ports with `ss -tlnp`
|
||||
- Check firewall rules (ufw/iptables)
|
||||
- Verify no services are listening on 0.0.0.0 that shouldn't be
|
||||
|
||||
## Alerting
|
||||
|
||||
- Security issues go to the user's Telegram immediately
|
||||
- Low-urgency items (package updates, cert expiry > 30 days) go into a daily digest
|
||||
- False positives should be tracked to prevent re-alerting on the same finding
|
||||
- No alert is the expected state — silent means healthy
|
||||
|
||||
## Portal integration
|
||||
|
||||
When the portal is built, these audit results should appear on a dedicated Security Dashboard page with:
|
||||
- Per-server health indicators
|
||||
- Failed login trends over time
|
||||
- Update status
|
||||
- Last audit timestamp
|
||||
|
||||
**Pitfalls:**
|
||||
|
||||
- **auth.log rotation:** On Debian 13, auth logs may be under `/var/log/auth.log` or journalctl. Check both.
|
||||
- **netcup SMTP block:** Automated email alerts may not send if the SMTP script uses port 587. Always use 2525.
|
||||
- **Don't alert on the same thing twice:** Track last-reported state in a sentinel file under `/root/.hermes/scripts/.audit-*`.
|
||||
- **For standalone scripts (no_agent), output must be self-explanatory:** The script output is delivered verbatim. No LLM is running to interpret it. Format alerts as plain text that the user can understand immediately.
|
||||
- **Wazuh is on the roadmap** for full SIEM/XDR coverage across all servers (added Jul 6, 2026 to portal todo). Until deployed, Lynis + SSH monitoring are the active security audit tools.
|
||||
- **Bitdefender GravityZone → Wazuh integration (Jul 15, 2026):** Germaine uses Bitdefender through SyncroMSP for Windows client endpoints. Official Wazuh integration path: Bitdefender GravityZone Event Push Service → `gz-evpsc` connector (Ubuntu VM) → Rsyslog → Wazuh manager. This enables unified Bitdefender + Wazuh events per MSP client, correlated timelines, and per-client compliance reporting (CIS/PCI). Bitdefender detects malware/EDR at the endpoint; Wazuh correlates with HIDS/FIM/SCA/network data for the full attack chain. Requires: Wazuh manager (CPX21), Bitdefender connector VM (~2GB), GravityZone API key for endpoint inventory.
|
||||
|
||||
- **Wazuh MCP plan (Jul 14, 2026):** Build `wazuh-mcp` as separate CPX21. Expose agent deployment, SCA/CIS checks, syscollector inventory, alert retrieval, and aggregated per-server security assessment tool. Note: Wazuh 4.8 removed the vulnerability API endpoint — CVE data accessible through Wazuh indexer (OpenSearch) side-channel.
|
||||
- **VirusTotal integration (Jul 14, 2026):** Add to long-range plan as supplementary threat intel. Use cases: hash/file lookup, URL scan, IP/domain reputation. Integrate into security assessment reports alongside Wazuh + Lynis findings. API key pending.
|
||||
- **Lynis is deployable on any Linux server** via a single `apt-get install lynis` + cron script. No subagent, tracking, or licensing needed. Done on netcup and wphost02. Tony's box is next.
|
||||
|
||||
## Operational security (approved user policy)
|
||||
|
||||
These rules govern how the agent operates on infrastructure, approved by Germaine on July 5, 2026.
|
||||
|
||||
### Read-only default
|
||||
- Explore/check/read operations are automatic and need no approval
|
||||
- Any state change (write, delete, modify, restart, reboot, config change) requires explicit user approval
|
||||
|
||||
### Destructive action approval
|
||||
Before executing a destructive action, state the exact impact:
|
||||
> "[action] on [server]. Impact: [what will happen]. Proceed?"
|
||||
|
||||
Then wait for "yes" or another affirmative. Do not proceed on implied consent.
|
||||
|
||||
### API key least-privilege
|
||||
- Every API key used by the agent must be scoped to minimum permissions needed
|
||||
- When a user provides a new key, verify its scope if possible and flag if it seems over-privileged
|
||||
- Examples: Cloudflare = DNS edit only (not zone delete), Wasabi = write to backup buckets (not delete), Stripe = read-only (no refunds/charges)
|
||||
|
||||
### Session summaries
|
||||
After any session that involved changes (config edits, server modifications, cron job changes, API key setup), provide a concise summary of every operation performed, its result, and any side effects.
|
||||
|
||||
### No assumed consent between actions
|
||||
Each action stands alone. Completing task A does not imply consent for task B, even if B is closely related. Ask again.
|
||||
|
||||
## ITGC / Compliance Context
|
||||
|
||||
During a Jul 7, 2026 session, the user asked where we stand on ITGC controls (IT General Controls). Honest assessment for internal reference:
|
||||
|
||||
**What would pass a SOC 2 / ITGC review:**
|
||||
- Access management: MFA on every admin access, quarterly access reviews, formal on/offboarding
|
||||
- Change management: Documented tickets with approval, rollback plans
|
||||
- Computer operations: Formal incident response plan, runbooks, documented backup restore testing
|
||||
- Program development: Code review, source control
|
||||
|
||||
**What we actually have:**
|
||||
- SSH key-only (no passwords), least-privilege API keys, Tailscale isolation
|
||||
- Daily backups to S3 + warm standby + nightly Lynis scans
|
||||
- No MFA on any admin access — biggest single gap
|
||||
- No change tickets, no incident response plan, no access reviews
|
||||
|
||||
**The user's instruction: "Keep these audits/controls in mind as we build these systems."**
|
||||
|
||||
This means: when deploying new services (DRE portal, Zabbix, Wazuh, Mautic), build with audit readiness in mind from day one:
|
||||
- Access logging on every service
|
||||
- Least-privilege by default on every new service
|
||||
- Documentation baked into deployment
|
||||
- Change tracking in the project log
|
||||
- MFA consideration at design time
|
||||
@@ -0,0 +1,98 @@
|
||||
---
|
||||
name: lynis-security-scan
|
||||
description: "Deploy and manage nightly Lynis security audits across all servers — package CVE scanning, misconfiguration detection, SSH brute-force monitoring. Silent when clean, alerts on new findings."
|
||||
version: 1.0.0
|
||||
author: Sho'Nuff
|
||||
tags: [security, lynis, audit, cve, scanning, cron]
|
||||
---
|
||||
|
||||
# Lynis Security Scan
|
||||
|
||||
Deploy the Lynis system audit tool across all servers. Nightly runs at 3 AM. Silent when no new warnings appear. Alerts if new findings emerge.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
apt-get install -y lynis
|
||||
lynis --version # should be 3.1.x+
|
||||
```
|
||||
|
||||
## First Run
|
||||
|
||||
```bash
|
||||
lynis audit system --quick
|
||||
```
|
||||
|
||||
Key output: Hardening index (0-100), Warnings count, Suggestions count.
|
||||
|
||||
Results are stored in `/var/log/lynis-report.dat` and `/var/log/lynis.log`.
|
||||
|
||||
## Cron Deployment
|
||||
|
||||
The script at `/root/.hermes/scripts/lynis-scan.sh`:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Nightly security audit via Lynis
|
||||
# Runs at 3 AM, outputs report, only alerts on new warnings
|
||||
LOG="/var/log/lynis-report.dat"
|
||||
PREV="/root/.hermes/lynis-prev.dat"
|
||||
|
||||
lynis audit system --quick > /dev/null 2>&1
|
||||
|
||||
# Count warnings
|
||||
WARNS=$(grep -c "^warning\[\]" "$LOG" 2>/dev/null || echo 0)
|
||||
|
||||
# Compare with previous run
|
||||
if [ -f "$PREV" ]; then
|
||||
OLD_WARNS=$(grep -c "^warning\[\]" "$PREV" 2>/dev/null || echo 0)
|
||||
else
|
||||
OLD_WARNS=0
|
||||
fi
|
||||
|
||||
cp "$LOG" "$PREV"
|
||||
|
||||
# Only output if something changed
|
||||
if [ "$WARNS" -ne "$OLD_WARNS" ]; then
|
||||
echo "Lynis: $WARNS warnings (was $OLD_WARNS)"
|
||||
grep "^warning\[\]" "$LOG" | sed 's/warning\[\]=//' | tr '|' ' '
|
||||
else
|
||||
echo "[SILENT]"
|
||||
fi
|
||||
```
|
||||
|
||||
Add as no_agent=True cron:
|
||||
```bash
|
||||
cronjob(action='create', name='lynis-scan', schedule='0 3 * * *',
|
||||
script='lynis-scan.sh', no_agent=True, deliver='origin')
|
||||
```
|
||||
|
||||
## Multi-Server Deployment
|
||||
|
||||
Copy the script to each server and add to crontab. **Use the server's native crontab** rather than the Hermes cron system — the Hermes cron scheduler only runs on the host box, so it cannot schedule jobs on remote servers. Use SSH + `crontab -l` to deploy remotely:
|
||||
|
||||
```bash
|
||||
scp /root/.hermes/scripts/lynis-scan.sh root@<server>:/usr/local/bin/
|
||||
ssh root@<server> 'chmod +x /usr/local/bin/lynis-scan.sh && \
|
||||
(crontab -l 2>/dev/null; echo "0 3 * * * /usr/local/bin/lynis-scan.sh") | crontab -'
|
||||
```
|
||||
|
||||
### Servers deployed
|
||||
|
||||
| Server | IP | Date deployed | Status |
|
||||
|--------|----|-------------|--------|
|
||||
| Core (netcup) | 152.53.192.33 | Jul 6, 2026 | ✅ Running via cron job |
|
||||
| wphost02 (Hetzner) | 5.161.62.38 | Jul 7, 2026 | ✅ Running via crontab |
|
||||
|
||||
**Watchdog cron jobs should be no_agent=True.** The script itself handles the decision logic (silent unless something changed). Wrapping it in an LLM agent adds unnecessary tokens and causes phantom typing indicators in Telegram. Set `no_agent=True` when creating/updating the cron job for this script.
|
||||
|
||||
## Baseline Scores
|
||||
|
||||
| Server | Score (out of 100) | Warnings |
|
||||
|--------|-------------------|----------|
|
||||
| Netcup (core) | 62 | 3 (vulnerable package, IPv6 DNS) |
|
||||
| Hetzner wphost02 | 63 | 3 (to review) |
|
||||
|
||||
Common warnings:
|
||||
- **PKGS-7392**: One or more vulnerable packages. Fix with `apt-get update && apt-get upgrade`.
|
||||
- **NETW-2704/NETW-2705**: Nameserver unreachable (often IPv6 DNS). Fix by disabling IPv6 DNS or adding working resolvers.
|
||||
@@ -0,0 +1,34 @@
|
||||
# Data Broker Removal — Action Plan (Jul 2026)
|
||||
|
||||
## Recommendation: Use a paid service
|
||||
|
||||
| Approach | Year 1 Cost | Time | Effectiveness |
|
||||
|----------|------------|------|---------------|
|
||||
| Manual DIY | $0 | 12-20 hrs initial + 2-4 hr/month | Medium |
|
||||
| Kanary | **$84/yr** | 2 hrs setup, done | High (continuous) |
|
||||
| DeleteMe | $129/yr | 2 hrs setup, done | High (premium) |
|
||||
|
||||
**Why paid:** Data respawns every 30-60 days. A service continuously re-scans and re-submits removals. Manual approaches fizzle because by the time you finish the 50-site list, the first sites have re-added you.
|
||||
|
||||
## Priority Tiers (50 sites)
|
||||
|
||||
### Priority 1 — The Big 10 (removal cascades to smaller sites)
|
||||
Spokeo, Whitepages, Intelius, BeenVerified, TruthFinder, PeopleFinders, Radaris, MyLife, Nuwber, PeekYou
|
||||
|
||||
### Priority 2 — 20 high-traffic people-search sites
|
||||
USSearch, ZabaSearch, CheckPeople, GoLookUp, PublicRecords, SocialCatfish, WhitePagesPrime, Addresses, PhoneNumber, InstantCheckmate, PeopleLooker, PeopleSearchNow, PeopleSmart, PublicReports, USPhoneBook, SearchPeopleFree, That'sThem, WhoIs, Yatedo, InfoTracer
|
||||
|
||||
### Priority 3 — 10 background check + public record sites
|
||||
CriminalPages, FederalPrison, Mugshots, BustedMugshots, JailBase, TexasTribune, VineLink, PACER, UniCourt, Trellis
|
||||
|
||||
### Priority 4 — 10 marketing data brokers
|
||||
Acxiom, Epsilon, Oracle Data Cloud, Experian Marketing, Neustar, Lotame, Merkle, Harte Hanks, BlueKai, Quantcast
|
||||
|
||||
## Breach Database Checks
|
||||
- HaveIBeenPwned — https://haveibeenpwned.com
|
||||
- Dehashed — https://dehashed.com (15B+ breach records)
|
||||
- Firefox Monitor — https://monitor.firefox.com
|
||||
- Google Dark Report — https://one.google.com
|
||||
|
||||
## Status
|
||||
On the todo list as `kanary-setup` (pending user authorization to sign up).
|
||||
@@ -0,0 +1,39 @@
|
||||
# ITGC Build-Time Checklist
|
||||
|
||||
Keep this checklist in mind whenever deploying a NEW service or making significant infrastructure changes. Not every item applies to every project — use judgement.
|
||||
|
||||
## Pre-deployment
|
||||
|
||||
- [ ] **Access logging** — Does this service log admin actions? Who did what when?
|
||||
- [ ] **Least privilege** — Is the API key / service account scoped to minimum permissions? Nothing more?
|
||||
- [ ] **MFA** — Can this service support MFA for admin access? If not, document the gap.
|
||||
- [ ] **Secrets** — Are credentials in `.env` (chmod 600), not hardcoded in config files?
|
||||
- [ ] **Backup** — Is the data backed up independently of the server? How often? To where?
|
||||
- [ ] **Restore test** — Has the backup been tested by actually restoring? (Quarterly minimum)
|
||||
|
||||
## Post-deployment
|
||||
|
||||
- [ ] **Document** — Is the deployment captured in the project log?
|
||||
- [ ] **Access list** — Who has access? (SSH keys, API tokens, admin accounts)
|
||||
- [ ] **Change ticket** — Is there a record of what was done, why, and by whom?
|
||||
- [ ] **Runbook** — If this service goes down, does someone know how to bring it back?
|
||||
|
||||
## Semi-annual
|
||||
|
||||
- [ ] **Access review** — Are all SSH keys still needed? Any terminated employees/contractors still have access?
|
||||
- [ ] **Token rotation** — Are long-lived API tokens still scoped correctly?
|
||||
- [ ] **Backup restore test** — Actually restore from backup, not just verify backup ran.
|
||||
|
||||
## Current gaps (Jul 2026)
|
||||
|
||||
These are documented so we can track closure over time:
|
||||
|
||||
| Gap | Target | Status |
|
||||
|-----|--------|--------|
|
||||
| MFA on SSH/admin | All admin access | ❌ Missing |
|
||||
| Access reviews | Quarterly | ❌ Missing |
|
||||
| Change tickets | Every infra change | ❌ Missing |
|
||||
| Incident response plan | Written + tested | ❌ Missing |
|
||||
| Backup restore tests | Quarterly | ❌ Never done |
|
||||
| Password policy | 14+ chars, rotated, MFA | ❌ Missing |
|
||||
| Encryption at rest | Sensitive data stores | ❌ Missing |
|
||||
Reference in New Issue
Block a user