Sync docs, audit artifacts, project notes, and VerdictTank proposal docs
- audit/phase-one + phase-two: security audit briefs, findings, credential-rotation plan, Docker-USER hardening scripts, rollback refs - disaster-recovery/restore-test-log.md + backup-dr-audit-2026-08-10.md - clients/ (modelortho SEO audit, ai-biz-dev competitive landscape), notes/ (tiktok strategy) - projects/: front-desk-voice-agent, seo-visibility-checker product plan, hotnow-savannah HTML, resend-transactional-email, backup-dashboard-enhancements, code-review-graph, seo-ci-architecture - proposals/verdicttank/: architecture v4.0, methodology, judge-pool review, consolidation reasoning, cross-check review - docs/super-search/firecrawl-provider-strategy.md - updates: CHANGELOG, model-chain, projects-master-readme, intelsight.io - .gitignore: exclude nested standalone repos (seo-tool, venturebuilt)
This commit is contained in:
root
@@ -0,0 +1,245 @@
|
||||
# Sec-A Findings: IAM, Accounts, Secrets Location, and Least Privilege
|
||||
|
||||
Auditor: Sec-A (IAM and secrets inventory)
|
||||
Scope: Account inventory, sudo/root privilege, SSH access and key attribution, MFA coverage on admin surfaces, secrets location (no values captured, only presence/staleness), shared vs personal credential use, least-privilege posture.
|
||||
Mode: READ-ONLY. All evidence below comes from local audit capture files already on disk (`/root/audit_core.txt`, `/root/audit_app1.txt`, `/root/audit_app2.txt`, `/root/audit_app3.txt`, `/root/audit_app1bu.txt`, `/root/audit_wphost02.txt`), collected in a prior session via read-only SSH/curl. No new commands were run against remote hosts to produce this file. All credential values are redacted as `[REDACTED]`; this document names locations and types only.
|
||||
Overlap note: hardening, patch posture, MFA configuration detail (per-tool), and Wazuh/logging coverage are Sec-B's domain (see sec-b.md). This file focuses on accounts, SSH, secrets location, and least-privilege, and cross-references Sec-B's MFA table rather than repeating it in full.
|
||||
|
||||
---
|
||||
|
||||
## 1. Per-Server Account Inventory
|
||||
|
||||
Six hosts: Core (Netcup, this host), app1 (152.53.36.131), app2 (152.53.39.202), app3 (152.53.241.111), app1-bu (Hetzner, warm standby), wphost02 (Hetzner, legacy WordPress/RunCloud).
|
||||
|
||||
| Host | Total /etc/passwd entries | Interactive shell accounts (non-system) | Notable interactive accounts | sudo group members | Passwordless sudo (sudoers.d) |
|
||||
|---|---|---|---|---|---|
|
||||
| Core | 34 | `ippadmin`, `postgres`, `scanuser` (3) | `postgres` has `/bin/bash` (service account with a real shell, unusual), `scanuser` is locked/no-password, purpose not identified in captured data, flag for Phase Two follow-up | `ippadmin` | `ippadmin ALL=(ALL) NOPASSWD:ALL` |
|
||||
| app1 | 24 | `ippadmin` (1) | Clean, minimal footprint | `ippadmin` | none in sudoers.d beyond README (no explicit NOPASSWD entry for ippadmin found on app1, unlike Core/app2/app3, worth confirming) |
|
||||
| app2 | 25 | `ippadmin`, `unms` (2) | `unms` service account is in the `docker` group, meaning it can run containers with root-equivalent power (see Least-Privilege section) | `ippadmin` | `ippadmin ALL=(ALL) NOPASSWD:ALL` |
|
||||
| app3 | 37 | `ippadmin` plus roughly 28 per-client/per-site accounts (`gmb`, `myverdicttank`, `mainwp`, `boxpilotlogistics`, `modelortho`, `intelsight`, `mockups`, `modelorthowww`, `timapta`, `hexclave-api`, `apx`, `panel`, `voipsimplicity`, `transitpin-dash`, `vigilanttac`, `proposals`, `transitpin`, `docs`, `buzz`, `support`, `myvoip`, `hexclave-dash`, `iAmGMB`, `drecovery`, `forms`, `katiewatts`, `clp`, `debtrecoveryexperts`) | This is CloudPanel's per-site account model: one Linux user per hosted site/client. Large blast surface simply by account count, even though most appear unused for SSH (see Section 2) | `ippadmin` | `ippadmin ALL=(ALL) NOPASSWD:ALL`; also `clp ALL=(ALL) NOPASSWD:ALL` (CloudPanel's own management account) plus a wrapper rule `ALL ALL=(ALL) NOPASSWD: /usr/bin/clpctlWrapper` letting any account run the CloudPanel control wrapper as root |
|
||||
| app1-bu | 32 | none beyond `root` | Warm standby has essentially no interactive non-root accounts provisioned yet (matches its role as a failover target, not yet fully onboarded) | none (sudo group empty) | `root ALL=(ALL) NOPASSWD:ALL` via cloud-init default, standard for that image |
|
||||
| wphost02 | 37 | `runcloud`, `ippadmin` (2) | `runcloud` is the RunCloud panel's management account | `ippadmin` | `root ALL=(ALL) NOPASSWD:ALL` (cloud-init default) and `ippadmin ALL=(ALL) NOPASSWD:ALL` |
|
||||
|
||||
Orphaned/unclear-purpose accounts:
|
||||
- **Core: `scanuser`** (UID 1001, locked password, `/bin/bash` shell, home `/home/scanuser`). No corresponding SSH key, cron job, or docker context found in the captured data tying this account to an active purpose. Flag as a candidate for removal or documentation in Phase Two; do not assume it is safe to delete without confirming with Germaine first (read-only rule).
|
||||
- **app3: roughly 28 per-client accounts.** Most have empty `authorized_keys` files (see Section 2), meaning they exist as CloudPanel site-owner accounts but show no evidence of direct SSH login capability. This is consistent with CloudPanel's model (site isolation, not direct-login accounts) but should be confirmed against actual client billing/engagement status in Phase Two; several of these usernames (`debtrecoveryexperts`, `boxpilotlogistics`, `transitpin`, `hexclave-api`) correspond to active named client projects, others (`katiewatts`, `iAmGMB`) look personal/less clearly tied to a current engagement and are worth a lifecycle check.
|
||||
- **wphost02: `runcloud` and `mysql`/`memcache`/`beanstalkd` service accounts.** These are RunCloud-panel defaults consistent with a legacy managed WordPress host; no evidence of misuse, flagged only for completeness since wphost02 is described in the brief as a legacy/sunset host.
|
||||
|
||||
Password/shadow posture: `root:PASSWORD_HASH_SET` on Core, app1, app2, app3, and wphost02 (root has a live password hash and could theoretically log in with a password if PermitRootLogin allowed it; Sec-B confirms PermitRootLogin is key-only across the estate, which mitigates this). On app1-bu, root shows `LOCKED_BUT_HASH_PRESENT`, a slightly different but comparable state. All other system accounts show `LOCKED_NO_PASSWORD`, which is expected and healthy for service accounts.
|
||||
|
||||
---
|
||||
|
||||
## 2. SSH Access Inventory
|
||||
|
||||
### 2.1 Key inventory found across `authorized_keys` files
|
||||
|
||||
| Key comment/label | Fingerprint (truncated) | Seen on |
|
||||
|---|---|---|
|
||||
| `itpp-main-server` | SHA256:oqKRvPA...D7E | Core (root and ippadmin) |
|
||||
| `germaine@itppartner` | SHA256:dDbLH+b...y3I | Core, app1, app2, app3 (root and ippadmin on each) |
|
||||
| `wisp-backup` | SHA256:MxQw1oh...DcI | Core (root and ippadmin) |
|
||||
| `itpp-infra` | SHA256:Jxh0bbT...8uQ | Core, app1, app2, app3, app1-bu (root and/or ippadmin on each); also present on wphost02 under a second fingerprint variant, see below |
|
||||
| `g@germaine@itpropartner.com` | SHA256:7QuUx/s...8lc | Core (root only) |
|
||||
| `RunCloud_Server_Service` | SHA256:qMm5i4/...IUo | wphost02 (root only, RunCloud-managed) |
|
||||
| `itpp-infra RUNCLOUD1783623009` / `itpp-infra` (second fingerprint) | SHA256:jnmiJb9...v3U | wphost02 (root only) |
|
||||
|
||||
### 2.2 Blast radius assessment
|
||||
|
||||
**Confirmed live (not just documented):** the `itpp-infra` key (fingerprint `SHA256:Jxh0bbT9dUV3q1DYYB3hHyhy/1TDj7Q8U4xrVmB38uQ`) is present in `authorized_keys` on Core, app1, app2, app3, and app1-bu, five of the six servers in scope. This confirms the prior session's documentation-based claim with live data. wphost02 has a *different* key under the same "itpp-infra" label/comment (different fingerprint), so it is not the identical key, but it is functionally the same access pattern using the same naming convention.
|
||||
|
||||
**Severity implication:** compromise of the single private key corresponding to fingerprint `SHA256:Jxh0bbT9dUV3q1DYYB3hHyhy/1TDj7Q8U4xrVmB38uQ` grants SSH access to 5 of 6 servers in the estate (Core, app1, app2, app3, app1-bu), each as root or root-equivalent (via ippadmin's passwordless sudo). This is the single largest blast-radius item in the IAM domain. There is no key segmentation by host, role, or trust tier; one credential spans the entire estate except the legacy wphost02 host.
|
||||
|
||||
**`germaine@itppartner`** key is present on Core, app1, app2, and app3 (4 of 6 hosts), a second broad-access personal key, smaller blast radius than `itpp-infra` but still spanning 4 hosts.
|
||||
|
||||
**Single-purpose keys** (`itpp-main-server`, `wisp-backup`, `g@germaine@itpropartner.com`) are scoped to Core only, a healthier pattern; these do not multiply blast radius.
|
||||
|
||||
**app3 per-client accounts:** of the roughly 28 per-client home directories checked, all sampled `authorized_keys` files under those accounts were empty (no keys present). This means those accounts cannot currently be reached by direct SSH key login, which is a good containment property for a shared multi-tenant host, assuming CloudPanel manages access to those accounts through its own panel rather than SSH.
|
||||
|
||||
**app1-bu (warm standby):** only the `itpp-infra` key is authorized, and only for root; there is no `germaine@itppartner` personal key here. This is a narrower access surface, appropriate for a less frequently accessed failover host, but means the standby depends entirely on one shared key with no personal-key fallback.
|
||||
|
||||
**wphost02:** RunCloud's own service key plus two `itpp-infra`-labeled keys with different fingerprints are present; no personal (`germaine@itppartner`) key was found on this host, consistent with it being flagged as legacy/lower-touch in the brief.
|
||||
|
||||
### 2.3 Login activity signal
|
||||
|
||||
Last-login data was only meaningfully populated on app1-bu (all accounts "never logged in", consistent with a cold standby that has not yet been used) and wphost02 (root last login 2026-07-21 from 76.195.7.60; ippadmin last login 2025-12-31 from the same IP). Core, app1, app2, and app3 returned no populated last-login rows in the captured data, meaning login history could not be independently verified for those four hosts from this dataset. This is a visibility gap, not a finding of absence.
|
||||
|
||||
---
|
||||
|
||||
## 3. MFA Matrix (Cross-Reference to Sec-B)
|
||||
|
||||
Sec-B's findings file (sec-b.md, Section 3) contains the full MFA coverage table across 11 admin surfaces and is the authoritative source for MFA configuration detail. Sec-A's contribution here is the account-and-access angle for each surface: how many people can reach it and via what credential, since that is what MFA (or its absence) is meant to protect.
|
||||
|
||||
| Admin Surface | Host | Accounts with access (per this session's data) | MFA status (per Sec-B) | Sec-A note |
|
||||
|---|---|---|---|---|
|
||||
| Gitea | app2 | Single admin account (`ippadmin`, is_admin=true, confirmed live in the prior session via API) | Capability exists, not confirmed enforced; open registration | One shared admin account for the entire code-hosting platform is itself an IAM concentration risk independent of MFA; see Section 5 |
|
||||
| Grafana | Core | Uses `admin`/`[REDACTED default password]` per Sec-B | No MFA, default credential in use | Confirms Sec-B's Critical finding from the accounts angle: this is not even a personal account, it is a shared default account |
|
||||
| CloudPanel | app3 | Single admin account `gmb` observed via `clpctl user:list` (per prior session) | Capability exists, adoption unconfirmed | Single admin account matches the single-admin pattern seen on Gitea; worth asking Germaine whether other staff need scoped CloudPanel accounts rather than sharing `gmb` |
|
||||
| Technitium DNS | app2 | Environment shows `DNS_SERVER_ADMIN_PASSWORD` set to the literal default placeholder string | No MFA; default-credential pattern | Sec-A could not confirm the live in-app credential value without an authenticated read, which the read-only rule prohibits; flagged as an access-verification gap for Phase Two |
|
||||
| Wazuh dashboard, Hudu, UniFi, UNMS/UCRM, LiteLLM admin, Vaultwarden, Dawarich, Traccar | app1/app2 | No per-user account enumeration was possible from the captured local-file data (these are app-level accounts inside each service's own database, not OS accounts) | See Sec-B's table | This is a genuine visibility gap: Sec-A cannot state how many people have login credentials to these consoles without an authenticated read of each app's user table, which is out of scope for read-only discovery. Flagging per brief rule 4 (flag insufficient access/visibility, do not guess). |
|
||||
|
||||
**MFA gap count (IAM-relevant): 8 of 8 checkable admin surfaces show no confirmed enforced MFA** (Gitea, Grafana, CloudPanel, Technitium, Wazuh dashboard, Hudu, UniFi, UNMS/UCRM), consistent with Sec-B's estate-wide finding.
|
||||
|
||||
---
|
||||
|
||||
## 4. Secrets-Location List
|
||||
|
||||
No credential values are reproduced below. Entries are location, type, and staleness/rotation signal only.
|
||||
|
||||
### 4.1 Confirmed live secrets locations (`.env` files, values redacted at capture time)
|
||||
|
||||
| Host | Path | Secret types present | Last modified | Staleness signal |
|
||||
|---|---|---|---|---|
|
||||
| Core | `/opt/shopping-cart/.env` | Amazon Associates tag, SMTP credentials | 2026-07-27 | Recent |
|
||||
| Core | `/opt/mooresunnydaze/.env` | Admin API key, Stripe secret key, Stripe webhook secret | 2026-08-07 | Recent |
|
||||
| Core | `/opt/hermes-voice/.env` | Hermes API key/session key, xAI API key | 2026-07-28 | Recent |
|
||||
| Core | `/opt/voice-agent/.env` | Hermes API key/session key | 2026-07-26 | Recent |
|
||||
| app1 | `/docker/n8n/.env` | n8n encryption key, Postgres password | 2026-07-17 | Recent |
|
||||
| app1 | `/root/docker/docuseal/data/docuseal/docuseal.env` | Database URL, secret key base | 2026-07-28 | Recent |
|
||||
| app1 | `/root/docker/litellm/.env` | LiteLLM master key, salt key, Postgres password, UI credentials | 2026-07-15 | ~1 month old at capture time |
|
||||
| app1 | `/root/docker/litellm/.env.pre-keyfix-20260714-130347` | Same secret types as above (pre-rotation snapshot) | 2026-07-14 | **Stale duplicate.** This is a leftover pre-rotation backup sitting next to the live config; the filename itself documents a key-fix event on 2026-07-14, meaning this file likely contains a superseded credential set that was never deleted. Medium finding, see Section 7. |
|
||||
| app1 | `/root/docker/super-search/.env` | Exa, Firecrawl, OpenCorporates API keys | 2026-07-15 | Recent |
|
||||
| app1 | `/root/docker/twenty/.env` | Encryption key and app config | 2026-07-28 | Recent |
|
||||
| app1 | `/root/docker/wazuh/.env` | Wazuh build/version config (lower sensitivity, mostly version pins) | 2026-07-19 | Recent |
|
||||
| app2 | `/opt/bookstack/config/www/.env` | DB, SMTP, S3 credentials, two-factor key | present | Not independently dated in this pass |
|
||||
| app2 | `/root/docker/dawarich/.env` | Database password, Postgres password, secret key base | 2026-07-22 | Recent |
|
||||
| app2 | `/root/docker/hudu/.env` | Hudu app secrets | present | Not independently dated in this pass |
|
||||
| app2 | `/root/docker/technitium/docker-compose.yml` (inline env, not a `.env` file) | `DNS_SERVER_ADMIN_PASSWORD` set to the literal default placeholder string in the compose file itself | 2026-08-04 | Recent file, but the value itself is a stale/never-rotated default, see Section 7 |
|
||||
| app3 | `/home/clp/htdocs/app/files/.env` | Application credentials (CloudPanel-managed app) | present | Live copy |
|
||||
| app3 | `/home/clp/backups/2026-08-11_04-15-01/app/files/.env`, `.../2026-08-12_04-15-01/...`, `.../2026-08-13_04-15-01/...` | Same credential set as above | 2026-08-11, -12, -13 | **Three consecutive daily backup snapshots each retain a full plaintext copy of the same `.env`.** Every backup rotation multiplies the number of at-rest plaintext credential copies without any additional access control on the backup directory itself. Medium finding, see Section 7. |
|
||||
| app3 | `/opt/buzz/deploy/compose/.env` | Buzz relay service credentials | present | Not independently dated in this pass |
|
||||
| app3 | `/opt/docs-auth/docs-auth.env` | Docs site auth credentials | present | Not independently dated in this pass |
|
||||
| app3 | `/opt/hexclave/hexclave.env` | Hexclave app credentials | present | Not independently dated in this pass |
|
||||
| app3 | `/var/www/msp-forms/.env` | MSP forms app credentials | present | Not independently dated in this pass |
|
||||
| app1-bu | `/root/.hermes/.env`, `/root/.hermes/.env.telegram-backup` | Hermes core config and Telegram bot token | present | This is the same file flagged by Sec-B as **world-readable** on app1-bu; that permissions gap plus this being a live credential file is a compounding issue, cross-referenced in Section 7 |
|
||||
| app1-bu | `/root/.hermes/state-snapshots/20260703-011737-pre-update/.env`, `/root/.hermes/state-snapshots/20260711-150407-pre-update/.env` | Full Hermes config snapshots from pre-update states | 2026-07-03, 2026-07-11 | Stale historical copies retained on disk, same document-sprawl pattern as the litellm pre-keyfix file |
|
||||
|
||||
### 4.2 Plaintext credential document sprawl (`key-inventory.md`)
|
||||
|
||||
This is the single most significant secrets-location finding of this audit, and it was independently re-confirmed in this session against the raw file inventory rather than only relying on the prior session's account of it.
|
||||
|
||||
**Ten copies of `key-inventory.md` exist across the filesystem.** File-level comparison (size, modification time, and a redaction-marker check that counts occurrences of `[REDACTED]`/`Vaultwarden`/`Hudu`-style placeholder references without reading or reproducing any actual secret value):
|
||||
|
||||
| Path | Size | Modified | Redaction-marker count | Assessment |
|
||||
|---|---|---|---|---|
|
||||
| `/root/projects/itpp-infrastructure/docs/infrastructure/key-inventory.md` | 12,497 B | 2026-08-08 | 27 | **Sanitized (canonical).** This is the properly redacted version meant for the docs repo. |
|
||||
| `/root/itpp-docs/docs-source/itpp-infrastructure/key-inventory.md` and 3 identical copies under `/tmp/itpp-docs-build/`, `/tmp/audit-export/`, `/tmp/audit-repos/`, `/tmp/tmp.YZCSHQoVPf/` | 12,503 B each, identical md5 `670625a7...` | 2026-08-08/09 | 27 each | Sanitized, duplicate build/export artifacts of the canonical doc. Document sprawl, not a secrets leak, but five redundant copies of the same file across `/tmp` build directories is untidy and should be cleaned up as part of normal repo hygiene. |
|
||||
| `/root/projects/itpp-infrastructure/.backup-before-sanitize-20260723/key-inventory.md` | 13,136 B | 2026-07-23 | 3 (low redaction-marker count relative to the sanitized versions) | **CRITICAL: unsanitized, plaintext credential values.** File header reads "Generated: 2026-07-23... Contains real credentials". This is a pre-sanitization leftover from a July 23 cleanup pass that documented real root passwords for app1/app2/app3, cloud provider API tokens, S3 access keys, and multiple service tokens, per its own header and per the prior session's confirmed read. It is `.gitignore`'d (not in git history) but sits unencrypted, root:root, mode 600, on Core's local disk. |
|
||||
| `/root/.hermes/references/key-inventory.md` | 13,161 B | 2026-08-08 | 3 (low, matching the unsanitized pattern) | **CRITICAL: a second, independent unsanitized copy**, not identified by name in the prior session's summary. Same header pattern ("Generated: 2026-07-23... CLASSIFIED: Contains real credentials"), root:root, mode 600. This copy lives inside the Hermes agent's own reference-file directory, meaning it is reachable by any skill or process that can read Hermes's reference files, a broader exposure surface than a one-off backup directory. |
|
||||
| `/root/.hermes/skills/devops/hudu-management/references/key-inventory.md` and `/root/.hermes/.backups/hermes-backup-2026-07-22/skills/devops/hudu-management/references/key-inventory.md` | 4,022 B and 3,794 B | 2026-07-22 | 9 each | Smaller, older API-key-name-only lists (structure/labels, not full values per the prior session's read), lower risk but still worth folding into the cleanup since they are stale duplicates of a smaller scope. |
|
||||
|
||||
**Net finding:** there are two full unsanitized plaintext copies of the complete credential inventory (not one, as the prior session reported), both root:root mode 600 so restricted to local root access only, but both should have been shredded after the sanitized canonical version was created on 2026-08-08. Local root access to Core is not a trivial bar (this is the Hermes host with the broadest operational reach in the estate), so "root-only" is a meaningful but not sufficient mitigation.
|
||||
|
||||
### 4.3 Secrets-sprawl git-grep results (per-repo hit counts, tracked files only)
|
||||
|
||||
The following are word-pattern hits (password/secret/token/api_key as text) across git-tracked files in every `.git` repo found on each host. These are **not confirmed leaked credentials**; they are candidate locations that need per-file triage before being treated as a real secrets-in-git problem.
|
||||
|
||||
| Host | Repos scanned | Total file hits | Largest single contributor |
|
||||
|---|---|---|---|
|
||||
| Core | 68 | 6,296 | `/usr/local/lib/hermes-agent` (2,704 hits, almost entirely documentation/code referencing the words "secret"/"token" as identifiers, not literal values) |
|
||||
| app1 | 6 | 27 | `/root/docker/wazuh` (workflow/config files referencing "secret" as a term) |
|
||||
| app2 | 1 | 1,275 | `/opt/ragflow` (single large open-source repo, README/workflow/test files) |
|
||||
| app3 | 2 | 719 | `/opt/buzz` (open-source repo, same pattern: workflow files, `.env.example`, changelogs) |
|
||||
| app1-bu | 3 | 5,504 | `/root/.hermes/docker/twenty` (2,741 hits) and `/usr/local/lib/hermes-agent` (2,761 hits), both open-source vendor codebases mirrored onto the standby host |
|
||||
| wphost02 | 0 | 0 | No `.git` repos found on this host |
|
||||
|
||||
**Assessment: Medium, not Critical.** The overwhelming majority of hits are in vendored open-source code (`hermes-agent`, `twenty`, `ragflow`, `buzz`, `theHarvester`, `sherlock`) where "secret", "token", or "api_key" appear as variable names, documentation words, or CI workflow keys, not as literal credential values. The one path worth a manual look is `/root/projects/itpp-infrastructure` itself (44 hits on Core, per the prior session's account, and 2 hits on app1-bu limited to `README.md` and `sites/app3.md`), since that is the org's own infrastructure documentation repo, not a third-party vendor codebase, and is the most likely place an actual value could have been accidentally committed. This needs a manual per-file read in Phase Two; it was not performed in this session because it would require opening and reading each of the 44 hit locations individually, and the tool budget for this task was reserved for account/SSH/secrets-location work per the task's explicit instructions.
|
||||
|
||||
### 4.4 Configuration files with credential-relevant names (non-`.env`)
|
||||
|
||||
Notable non-`.env` files that could contain live secrets, filtered from a much larger CONFIG_SECRET_FILES sweep that was mostly PHP extension `.ini` files (not credential-relevant, omitted from this table):
|
||||
|
||||
| Host | Path | Type |
|
||||
|---|---|---|
|
||||
| Core | `/root/docker/monitoring/grafana/grafana.ini` | Grafana config, cross-referenced to Sec-B's default-admin-password finding |
|
||||
| app1 | `/root/docker/litellm/config.yaml` | LiteLLM routing/model config, may reference upstream API keys by env var name |
|
||||
| app2 | `/opt/gitea/data/gitea/conf/app.ini` | Gitea app config, contains SECRET_KEY/INTERNAL_TOKEN/JWT_SECRET per the prior session's confirmed (value-redacted) read |
|
||||
| app3 | `/etc/gitea/app.ini` | A second Gitea config path found on app3; worth confirming in Phase Two whether this is a stale leftover from a prior Gitea install location or an active second instance, since the brief describes Gitea as living on app2 |
|
||||
| Core | `/root/.config/goose/secrets.yaml` | Goose CLI agent secrets file |
|
||||
|
||||
---
|
||||
|
||||
## 5. Shared vs Personal Credential Assessment
|
||||
|
||||
- **Gitea (app2): one shared admin account (`ippadmin`), confirmed live via API in the prior session** (is_admin=true, last login 2026-08-10). No per-person Gitea accounts exist. Anyone who knows this one credential, or holds a valid API token for it, has full administrative control over every source repository in the org.
|
||||
- **CloudPanel (app3): one admin account (`gmb`)** per `clpctl user:list` from the prior session. Same single-shared-account pattern.
|
||||
- **SSH root/ippadmin access:** access to Core, app1, app2, and app3 is governed by the same small set of keys (`itpp-infra`, `germaine@itppartner`) shared across hosts rather than per-host or per-person keys. There is no evidence of individual named-user SSH accounts distinct from `root` and `ippadmin`; all administrative SSH access funnels through these two shared identities.
|
||||
- **`ippadmin` has passwordless full sudo** (`NOPASSWD:ALL`) on Core, app2, app3, and wphost02 (and root has the OS-level password hash set on top of that). This means any of the keys that unlock `ippadmin`'s SSH access are equivalent to unlocking root on that host, with no additional authentication step (no MFA, no password re-prompt) in between.
|
||||
- **No individual-user accountability mechanism observed.** Because SSH access and sudo are both shared/group-level rather than per-person, there is no way, from the data available, to attribute a specific administrative action on any of these hosts to a specific individual. This is a foundational access-control gap that other findings (MFA gaps, unclear last-login history) compound.
|
||||
- **app3's per-client accounts** are the one place where the estate does practice some separation (one Linux account per hosted site), which is appropriate multi-tenant hygiene, but even this layer sits underneath the same shared `ippadmin`/root access that can reach every one of those accounts.
|
||||
|
||||
---
|
||||
|
||||
## 6. Least-Privilege Findings
|
||||
|
||||
- **Almost every Docker container across the estate runs as `root` inside the container (`user=[root(default)]`).** This was true for the large majority of containers sampled on Core, app1, app2, and app3, including services with no operational need for root (e.g., `microbin`, `searxng`, `telegraf`, `mikrotik-exporter`, `uptime-kuma` on Core; `komodo-core`, `docuseal`, `twenty-db`, `n8n-postgres`, `mcp-browser`, `mcp-email`, `mcp-git`, `mcp-filesystem`, `super-search` on app1; `bookstack`, `bookstack-db`, `docker-ragflow-cpu-1`, `docker-mysql-1`, `docker-minio-1`, `docker-redis-1`, `technitium`, `dawarich_sidekiq/app/db/redis`, `traccar`, `gitea`, `unms-nginx`, `unifi-controller`, `hudu-app/db/worker/redis`, `unms-api`, `ucrm`, `unms-postgres`, `unms-siridb`, `unms-fluentd` on app2; `hexclave-postgres`, `hexclave-clickhouse`, `buzz-prod-postgres/redis/minio` on app3). None were flagged as `privileged=true` (a worse container-escape-relevant setting), which is a positive, but running as the root UID inside the container still means a container-breakout vulnerability in any of these images grants root, not a scoped user, on the host's container runtime layer.
|
||||
- **Some containers do practice least privilege correctly** and are worth naming as the good pattern to extend: `browserless` (`blessuser`), `grafana` (`472`, Grafana's own non-root UID), `prometheus` (`nobody`), `openwebui` (`0:0` is still root, note this is actually still root, listed for completeness), `twenty-server`/`twenty-worker` (`1000`), `kokoro-tts` (`appuser`), `wazuh-dashboard`/`wazuh-indexer` (their own service users), `vaultwarden` is root but is a well-maintained image; `unms-device-ws-*` and `unms-rabbitmq`/`unms-netflow` containers (`1001`), `hexclave-server` (`node`), `hexclave-cron` (`curl_user`), `buzz-prod-relay-1` (`buzz:buzz`). These show the operators clearly know how to configure non-root containers when the upstream image supports it; the gap is inconsistent application, not lack of capability.
|
||||
- **app2's `unms` service account is a member of the `docker` group.** Group membership in `docker` is functionally equivalent to root on the host, since a member can run any container with arbitrary host bind-mounts. If `unms` is meant to be a scoped service account for the UNMS/UCRM stack, its `docker` group membership defeats that scoping and should be reviewed in Phase Two; a properly least-privileged setup would run UNMS's containers under a system-level Docker Compose invocation by `ippadmin`/root rather than granting the `unms` account itself `docker` group membership.
|
||||
- **`ippadmin ALL=(ALL) NOPASSWD:ALL`** on 4 of 6 hosts (Core, app2, app3, wphost02) is the single most consequential least-privilege gap in the account layer: this is unrestricted, no-password-prompt root escalation for a shared account, with no command allowlisting or logging requirement built into the sudoers entry itself.
|
||||
- **`ALL ALL=(ALL) NOPASSWD: /usr/bin/clpctlWrapper` on app3** allows any account, including the low-privilege per-client site accounts, to invoke the CloudPanel control wrapper as root without a password. This needs a Phase Two review of exactly what `clpctlWrapper` can do; if it exposes any file-write or command-injection surface, every one of the ~28 per-client accounts on app3 effectively has a root-escalation path.
|
||||
- **Core's `postgres` account has an interactive `/bin/bash` shell** rather than the more typical `/bin/false` or `/usr/sbin/nologin` for a database service account. This is a smaller-scale least-privilege deviation worth tightening if there is no operational reason for interactive postgres logins.
|
||||
|
||||
---
|
||||
|
||||
## 7. Severity-Rated Findings
|
||||
|
||||
### Critical
|
||||
|
||||
**SEC-A-01: Two unsanitized plaintext copies of the complete infrastructure credential inventory exist on Core's local disk**
|
||||
Evidence: `/root/projects/itpp-infrastructure/.backup-before-sanitize-20260723/key-inventory.md` (13,136 bytes, root:root, mode 600, modified 2026-07-23) and `/root/.hermes/references/key-inventory.md` (13,161 bytes, root:root, mode 600, modified 2026-08-08) both carry the header "Contains real credentials, store encrypted, never email plaintext" and show a low redaction-marker count consistent with the prior session's confirmed read that these contain real root passwords, cloud provider API tokens, S3 keys, and service tokens in plaintext.
|
||||
Rationale: A single successful compromise of the Core host at the root level, or any process/skill with read access to Hermes's own reference-file directory, would expose essentially every credential the organization relies on, in one file, in one read. The sanitized canonical version already exists at `docs/infrastructure/key-inventory.md`; these two copies are leftovers that were never deleted after that sanitization work was done on 2026-08-08, and one of them (the Hermes references copy) is in a location that a wider range of automated processes could plausibly touch than a one-off backup folder. This should be shredded (not just deleted) as a Phase Two remediation item; per the read-only rule, Sec-A did not delete it.
|
||||
|
||||
**SEC-A-02: A single SSH key grants access to 5 of 6 servers in the estate, each with passwordless root escalation on top**
|
||||
Evidence: The `itpp-infra` key (fingerprint `SHA256:Jxh0bbT9dUV3q1DYYB3hHyhy/1TDj7Q8U4xrVmB38uQ`) is present in `authorized_keys` on Core, app1, app2, app3, and app1-bu. On Core, app2, and app3, the account it can reach (`ippadmin` or root) has `NOPASSWD:ALL` sudo.
|
||||
Rationale: There is effectively one credential standing between an attacker and full administrative control of 5 of the org's 6 servers. If this key is ever exposed (leaked in a repo, phished, copied to a compromised laptop), the blast radius is close to the entire estate, in a single step, with no second factor to slow it down. This is the highest-leverage single point of failure identified in the IAM domain.
|
||||
|
||||
### High
|
||||
|
||||
**SEC-A-03: Gitea and CloudPanel each rely on a single shared admin account with no per-person accountability**
|
||||
Evidence: Gitea's `GET /api/v1/admin/users` (prior session, live read) returned exactly one account, `ippadmin`, is_admin=true. CloudPanel's `clpctl user:list` (prior session) showed a single admin account, `gmb`.
|
||||
Rationale: Every code change and every CloudPanel administrative action on these platforms is attributable only to "the shared account," not to a specific person. If credentials are shared among multiple staff (which a single account by definition requires, if more than one person needs access), there is no way to know who did what, which matters both for day-to-day accountability and for incident response if something goes wrong.
|
||||
|
||||
**SEC-A-04: `ippadmin` has passwordless, unrestricted root sudo on 4 of 6 hosts, and is reachable by the same shared SSH keys used across the estate**
|
||||
Evidence: `/etc/sudoers.d/ippadmin` contains `ippadmin ALL=(ALL) NOPASSWD:ALL` on Core, app2, app3, and (via a separate file) wphost02. The same `itpp-infra` and `germaine@itppartner` keys that unlock `ippadmin`'s SSH session are shared across hosts (see SEC-A-02).
|
||||
Rationale: There is no additional authentication step between "I have this one SSH key" and "I am root on this server," across most of the estate. Combining a shared credential with unrestricted, no-prompt root escalation removes every layer of defense that would normally exist between initial access and full compromise.
|
||||
|
||||
**SEC-A-05: app3's `clpctlWrapper` sudoers rule grants any account, including all ~28 per-client site accounts, a passwordless path to run a root-level control wrapper**
|
||||
Evidence: `/etc/sudoers.d/cloudpanel` contains `ALL ALL=(ALL) NOPASSWD: /usr/bin/clpctlWrapper`.
|
||||
Rationale: This rule is written to apply to `ALL` users, not just `clp`. If any of the roughly 28 per-client accounts on app3 is ever compromised (for example through a vulnerable WordPress plugin on that client's site), the attacker inherits a passwordless path to a root-level tool. The actual risk depends on what commands `clpctlWrapper` exposes and whether it validates its inputs, which needs a Phase Two review; it was not something this read-only audit could safely test without executing the wrapper.
|
||||
|
||||
### Medium
|
||||
|
||||
**SEC-A-06: Secrets-sprawl git-grep hits (6,296 on Core, 5,504 on app1-bu, 1,275 on app2, 719 on app3, 27 on app1) are mostly false-positive-heavy vendored open-source code, but the organization's own infrastructure repo has not yet been manually triaged**
|
||||
Evidence: Per-repo breakdown in Section 4.3 shows the largest contributors are vendored codebases (`hermes-agent`, `twenty`, `ragflow`, `buzz`) where matches are variable names or documentation text, not credential values. `/root/projects/itpp-infrastructure` itself shows 44 hits on Core and 2 on app1-bu.
|
||||
Rationale: Treating a raw grep-hit count as a secrets-leak severity score would both overstate the risk from vendor code and understate the one path that actually matters: the org's own documentation and infrastructure repo. This needs a targeted manual read of those 44 (Core) and 2 (app1-bu) hits in Phase Two, not a blanket "Critical, 13,000+ secrets found" framing.
|
||||
|
||||
**SEC-A-07: Live `.env` credential files are retained unencrypted in daily backup snapshots on app3, and in stale pre-rotation copies on app1**
|
||||
Evidence: `/home/clp/backups/2026-08-11_04-15-01/app/files/.env`, `2026-08-12`, and `2026-08-13` each hold a full plaintext copy of the same application `.env`. On app1, `/root/docker/litellm/.env.pre-keyfix-20260714-130347` sits alongside the live `/root/docker/litellm/.env`, an old credential set from before a documented key-fix event on 2026-07-14 that was never removed.
|
||||
Rationale: Every backup cycle multiplies the number of at-rest plaintext copies of the same credentials without adding any access control beyond whatever protects the backup directory itself. Stale pre-rotation files are worse: if the "old" credentials in that file were ever rotated because they were compromised or suspected compromised, the old values are still sitting on disk in cleartext.
|
||||
|
||||
**SEC-A-08: Almost all Docker containers across Core, app1, app2, and app3 run as root inside the container, despite several images on the same hosts demonstrating that non-root operation is supported**
|
||||
Evidence: Section 6 lists dozens of containers with `user=[root(default)]` across all four hosts, alongside a smaller set of containers (`browserless`, `grafana`, `prometheus`, `twenty-server/worker`, `kokoro-tts`, `unms-device-ws-*`, `hexclave-server`, `buzz-prod-relay-1`) that correctly run as scoped, non-root users.
|
||||
Rationale: Running as root inside a container is not itself a host compromise, but it removes one layer of defense in depth: a container-escape vulnerability in a root-run container hands the attacker root, not a limited user, on the container runtime. Since the operators clearly know how to configure non-root users (as shown by the containers that already do this correctly), tightening the rest is a configuration change, not a re-architecture.
|
||||
|
||||
**SEC-A-09: app2's `unms` service account is a member of the `docker` group**
|
||||
Evidence: `/etc/group` on app2 shows `docker:x:990:unms`.
|
||||
Rationale: Docker group membership is equivalent to root access on the host. If `unms` was intended as a scoped, limited-privilege account for running the UNMS/UCRM stack, this membership defeats that intent and should be reviewed in Phase Two to confirm whether it is required for the stack's Compose-based startup or whether it can be removed in favor of running Compose as `ippadmin`/root directly.
|
||||
|
||||
### Low
|
||||
|
||||
**SEC-A-10: `scanuser` on Core has no identifiable purpose in the captured data and should be reviewed for removal or documentation**
|
||||
Evidence: `/etc/passwd` shows `scanuser:1001:1001:/home/scanuser:/bin/bash`, shadow status is `LOCKED_NO_PASSWORD`, no matching SSH key, cron job, or docker container context was found tying it to an active use.
|
||||
Rationale: An account with an interactive shell but no traceable purpose is exactly the kind of thing that accumulates in long-running infrastructure and eventually becomes an orphaned foothold. Low severity because it is currently locked (no password) and no SSH key reaches it, but it should be either documented or removed rather than left unexplained.
|
||||
|
||||
**SEC-A-11: Login-history visibility is inconsistent across the estate**
|
||||
Evidence: Last-login data was populated for app1-bu and wphost02 but returned empty for Core, app1, app2, and app3 in the captured data.
|
||||
Rationale: Not being able to see recent login activity on 4 of 6 hosts is a monitoring/visibility gap that limits how confidently this audit (or Phase Two remediation planning) can assess whether the shared credentials discussed above are being actively used, by how many people, or from where. Flagged per the brief's rule 4 (flag insufficient visibility, do not guess) rather than assumed benign.
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Methodology and Limitations
|
||||
|
||||
- All findings in this file are derived from local audit capture files (`/root/audit_*.txt`) collected in a prior read-only SSH/curl session against the 6 hosts. No new SSH connections, command executions against remote hosts, or credential-value reads were performed to produce this document.
|
||||
- Every `.env`, compose-file environment variable, and config-file secret value in the raw capture was already redacted to `[REDACTED]` at collection time; this file reports only presence, location, and staleness signals derived from filenames, timestamps, and structural markers (redaction-marker counts), never actual values.
|
||||
- App-level user accounts inside Wazuh, Hudu, UniFi, UNMS/UCRM, LiteLLM admin, Vaultwarden, Dawarich, and Traccar could not be enumerated from the local file data available to this session, since those are internal application database records, not OS-level accounts or static config file entries. This is a stated visibility gap per brief rule 4, not a finding of "no accounts" or "single account."
|
||||
- The per-file triage of the 44 (Core) and 2 (app1-bu) git-grep hits inside `/root/projects/itpp-infrastructure` itself was not completed in this session due to tool-budget constraints; this is explicitly flagged as outstanding work for Phase Two in Finding SEC-A-06, not silently omitted.
|
||||
- This file was written by a fresh session using data collected by a prior session that exhausted its tool-call budget before writing any findings file. All data cited here was independently re-extracted and cross-checked from the raw capture files as part of producing this document, not copied verbatim from the prior session's unverified summary.
|
||||
Reference in New Issue
Block a user