hermes-skills/skills/creative/image-processing/SKILL.md

name, description, version, author, platforms, metadata

name	description	version	author	platforms	metadata
image-processing	Process images via CLI — background removal, transparency, resize/upscale, text cleanup, format conversion. For images sent by the user or extracted from screenshots/emails.	1.0.0	ShoNuff	linux	hermes tags image processing background-removal transparency png resize

Image Processing

Process images from the command line using Python PIL/Pillow and scientific packages (numpy, scipy, rembg).

Background Removal

For solid-color backgrounds (known color)

When the background is a uniform color (e.g. #A82835 red), use simple threshold-based removal — it's fast, deterministic, and doesn't need the rembg model.

from PIL import Image
import numpy as np

img = Image.open('input.jpg').convert('RGBA')
pixels = np.array(img)
h, w = pixels.shape[:2]

r, g, b = pixels[:,:,0].astype(float), pixels[:,:,1].astype(float), pixels[:,:,2].astype(float)

# Background color + threshold
bg_r, bg_g, bg_b = 168, 40, 53
dist = np.sqrt((r - bg_r)**2 + (g - bg_g)**2 + (b - bg_b)**2)
bg_mask = dist < 40

# Remove red shading on the subject (face highlights, etc.)
red_shading = (r > 100) & (g < 80) & (b < 80)

alpha = np.ones((h, w), dtype=np.uint8) * 255
alpha[bg_mask | red_shading] = 0

# Clean dark text/lineart to pure black
dark = (r < 40) & (g < 40) & (b < 40)
pixels[dark] = [0, 0, 0, 255]

pixels[:,:,3] = alpha
Image.fromarray(pixels).save('output.png')

Threshold tuning:

• dist < 40 — typical for solid backgrounds; increase for wider color tolerance
• (r > 100) & (g < 80) & (b < 80) — catches red-dominant subject shading
• (r < 40) & (g < 40) & (b < 40) — preserves black text/lineart

Solid red background — removing red-dominant shading on the subject

When the user's image has a solid red background AND the subject also has red shading (e.g. a character's face has red highlights), a single color-threshold pass will erase parts of the subject. Use a two-pass approach:

1. Remove background red first — target the specific background RGB (e.g. 168, 40, 53): dist < 40
2. Remove only red-dominant shading on the subject — target pixels that are very red AND low green/blue: (r > 100) & (g < 80) & (b < 80)
3. Restore black lineart — convert any dark pixels to pure black regardless of their original shade: pixels[(r < 40) & (g < 40) & (b < 40)] = [0, 0, 0, 255]

This is the opposite of the standard threshold approach: instead of trying to keep red subject areas, accept that red shading on the character will be lost and rely on the black lineart to define the character shape. This works because the source image defines the character through bold black outlines, not through the red fill.

Identifying the exact background color

# Sample corner pixels to determine background color
print(f"Corner: {pixels[0,0]}")
print(f"Bottom: {pixels[-1,0]}")

For pure white/off-white backgrounds (~248,248,248), threshold on all three channels:

bg = (r > 235) & (g > 235) & (b > 235)

Signature / watermark removal

When an image has a small text signature in a corner, paint it transparent by area:

# Target bottom-right corner for signature removal
sig_area = pixels[h-100:, w-150:, :]
sig_mask = (sig_area[:,:,0] < 100) & (sig_area[:,:,1] < 100) & (sig_area[:,:,2] < 100)
pixels[h-100:, w-150:][sig_mask, 3] = 0

Adjust offset (h-100, w-150) and color threshold based on signature location. The same pattern removes logos, timestamps, and watermarks.

Edge softening

After threshold-based removal, the alpha edge is a hard cut. For a softer edge:

from scipy.ndimage import gaussian_filter
alpha_float = alpha.astype(float)
alpha_blurred = gaussian_filter(alpha_float, sigma=1.0)
alpha = np.clip(alpha_blurred, 0, 255).astype(np.uint8)

Keep sigma between 0.5 and 2.0 — higher values cause visible halos.

For general photos (complex backgrounds)

Use rembg with the u2net model:

pip3 install "rembg[cpu]"

from rembg import remove, new_session
session = new_session("u2net")
output_data = remove(input_data, session=session)

Upscaling / Anti-aliasing

When the source image is small and text looks jagged after background removal:

from PIL import Image, ImageFilter

scale = 3  # 2x or 3x
img = Image.open('input.png').convert('RGBA')
new_w = img.width * scale
new_h = img.height * scale
img_large = img.resize((new_w, new_h), Image.Resampling.LANCZOS)
img_large = img_large.filter(ImageFilter.SMOOTH)
img_large.save('output.png')

LANCZOS resampling is the highest quality for text/lineart. SMOOTH filter after resize cleans up remaining aliasing.

Text Edge Cleanup

After background removal, text edges can look "jagged" from the hard alpha cut:

1. Fix by upscaling 3x with LANCZOS + SMOOTH filter
2. If that's insufficient, apply a light gaussian blur on the alpha channel then re-threshold
3. For really small source images (< 500px), upscaling before background removal gives better edge fidelity

Partial transparency cleanup (residual artifacts)

After threshold-based background removal, the image may still have semi-transparent pixels from the edge blending. A second pass that clears all semi-transparent pixels not adjacent to fully opaque ones removes ghost artifacts:

full_opaque = alpha > 240
semi = (alpha > 5) & (alpha < 200)
h, w = alpha.shape

# Keep semi-transparent pixels that border a fully opaque pixel (edge of subject)
keep = np.zeros_like(semi)
for y in range(1, h-1):
    for x in range(1, w-1):
        if semi[y,x]:
            if full_opaque[y-1:y+2, x-1:x+2].any():
                keep[y,x] = True

clear = semi & ~keep
pixels[clear, 3] = 0

This preserves the soft edge of the character/logo while cleaning up isolated star/dot artifacts left behind by the AI model.

For simpler cases — just clear all semi-transparent pixels:

pixels[(alpha > 5) & (alpha < 200), 3] = 0

Quick Size Check

python3 -c "from PIL import Image; img = Image.open('file.jpg'); print(f'{img.size[0]}x{img.size[1]}')"

Tool dependencies

Tool	Install	Used for
Pillow	Comes with Python	All image ops
numpy	Comes with Python	Pixel array manipulation
rembg[cpu]	pip3 install "rembg[cpu]"	AI background removal (complex photos)
scipy	pip3 install scipy	Optional: gaussian blur on alpha, connected-component cleanup

Sequence of operations for a solid-color background removal

The recommended sequence when starting from a JPG with solid red/white background and wanting a clean transparent PNG:

1. Load as RGBA: Image.open('input.jpg').convert('RGBA')
2. Threshold-based removal for the solid background color
3. Second pass: clear residual semi-transparent artifacts not bordering opaque pixels
4. Convert dark maroon/red pixels to pure black (clean up subject shading)
5. If text looks jagged: upscale 3x with LANCZOS, apply SMOOTH filter
6. Save as PNG

This avoids the rembg 176MB download entirely for simple solid-background images. Use rembg only for complex/photographic backgrounds.

Adding an email extraction tool for IMAP-based image uploads

When the user sends you a raw image that was emailed to them (e.g. a character illustration from a friend), you cannot vision_analyze it from inside an execute_code block because the image is in IMAP, not on disk. The pattern is:

1. Pull the email attachment from IMAP
2. Write it to a local path under /root/.hermes/cache/images/
3. Then do your image processing on that local file

When the user says they used a different tool and sent the result as an image, you can reference it at /root/.hermes/cache/images/img_<hash>.jpg without needing to re-download it from IMAP.

Pitfalls

• Jagged text after background removal — The source image was probably low-resolution. Upscale 3x with LANCZOS before or after removal, then apply SMOOTH filter.
• rembg leaves semi-transparent artifacts — Use the threshold-based removal instead for solid-color backgrounds. If you must use rembg, do a second pass: pixels[alpha < 200, 3] = 0 to clear ghosts.
• rembg first download is 176MB — It downloads u2net.onnx on first run. The smaller u2netp (4.5MB) is faster but slightly less accurate. Use new_session("u2netp").
• Telegram compresses images — An image sent through Telegram may be smaller or resampled. Ask for the original file if edge quality matters.
• Alpha channel comparison is False by default — Use np.where(pixels[:,:,3] > threshold).
• Character art with visible signature — If the user wants to remove a visible text signature from the bottom corner, use a separate area-limited pass. Target the corner zone and apply a dark-pixel threshold there, rather than trying to remove it globally or asking the user to do it themselves.
• Multiple iterations are expected — When the user says a processed image is not good, do not ask for permission to try again. Just try a different approach (upscale first, different threshold, different artifact cleanup) and deliver the new version. The user will tell you when it is good.
• Do not delegate image editing to subagents — Image processing (background removal, transparency, resize, cleanup) should be done directly with Python/PIL in terminal, not delegated. Subagents cannot see the images and produce SVG attempts instead, which wastes time and produces poor results. The user will reject SVG attempts and you'll end up doing it yourself anyway.
• Solid background removal is the only technique needed for character art — rembg (176MB model download) is unnecessary and slow for solid-background images. Threshold-based removal on the known background color is faster, better quality, and doesn't need dependencies. Only use rembg for complex photographic backgrounds.
• When the user sends a new image to replace a failed attempt, process it immediately — Do not ask "should I try again?" or offer options. The new image IS the answer. Remove the background, clean up, and deliver.
• User preference: the image should look like the reference they sent, not a crude SVG recreation — SVG is NOT the right tool for reproducing character art with complex shading, gradients, and organic shapes. The user explicitly rejected two SVG attempts and sent a new image from a different tool that they were happy with. If a subagent delivers an SVG and the user rejects it, do NOT re-delegate — do it yourself with Python/PIL.
• Deliver via MEDIA: path, not an email or link — After processing, send the image directly in the chat using the MEDIA: protocol. This lets the user see it right away in Telegram. Do not email the result or ask them to check a URL.
• When the user says an image is from a 'different tool' and sends the result, there is no background to remove — The image they send may already be a finished character design on a white/off-white background. Just remove the solid background and deliver. Do not try to recreate it or improve the design itself.
• Clean the signature first, then the background — When the user says they used a different tool, the resulting image may have a visible watermark/signature (e.g. "B. Martin") in the bottom-right corner. Remove the signature (pixel-area targeting in the corner) AND the background in a single pass. If you deliver a version with the signature still visible, they'll ask you to remove it anyway. Do both in one go.
• Upscale BEFORE background removal when source is small — For images under 500px where text will be jagged after removal, upscale 3x with LANCZOS first, THEN do the background removal. The edge quality is significantly better than removing first and upscaling after.
• Bold/stylized text on the character image counts as 'text' — The same jagged-edge cleanup techniques apply to in-image text as to the character outline.
• Multiple iterations are expected — When the user says a processed image is not good, do not ask for permission to try again. Just try a different approach (upscale first, different threshold, different artifact cleanup) and deliver the new version. The user will tell you when it is good.
• file command may not be installed on minimal servers — Use python3 -c to check image properties instead.
• User images land in /root/.hermes/cache/images/ — When the user sends an image as a message attachment, it's stored under this directory with a hash-based filename like img_398afabc6905.jpg. Do NOT ask for a URL or re-send — check that directory first.
• Subagent SVG is never the right answer for character art — SVG attempts at reproducing existing character illustrations are consistently rejected.
• User preference: deliver via MEDIA: path, not email or URL — After processing the image, send it directly in the chat using the MEDIA: protocol. This lets the user see it right away in Telegram. Do not email the result or ask them to check a URL.
• User preference: do not offer options or ask "should I try again" — When the user says an image is not good, just try a different approach without asking permission. Multiple iterations are expected and do not require signaling.