Import AITURK IDE 1.0.0-beta.1 from Hermes 63279301; preserve MIT license
This commit is contained in:
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Nous Research
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
@@ -0,0 +1,125 @@
|
||||
---
|
||||
name: pdf
|
||||
description: "PDF files: create, read, merge, fill, OCR, edit text."
|
||||
version: 1.1.0
|
||||
author: Nous Research
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [pdf, documents, forms, ocr, text-extraction, reportlab, pypdf, pdfplumber, pymupdf, marker]
|
||||
category: productivity
|
||||
related_skills: [docx, xlsx, powerpoint]
|
||||
---
|
||||
|
||||
# PDF Skill
|
||||
|
||||
Create PDFs from structured specs, build and fill AcroForm forms (with layout linting and visual overlays), extract text/tables/metadata, merge/split/rotate/watermark/stamp pages, export page images, manage metadata and attachments, and encrypt/decrypt — using pypdf, reportlab, and pdfplumber. Two absorbed capabilities live in references/ (read the matching file before those tasks):
|
||||
|
||||
- **Scanned/image-only PDFs and OCR** (pymupdf fast path, marker-pdf quality path, scripts/extract_pymupdf.py + scripts/extract_marker.py): `references/ocr-extraction.md`
|
||||
- **Editing text inside an existing PDF via natural-language prompts** (nano-pdf CLI): `references/nano-pdf-editing.md`
|
||||
|
||||
## When to Use
|
||||
|
||||
- Generate a report, invoice, or multi-page document as PDF.
|
||||
- Build a fillable AcroForm (text/checkbox/radio/dropdown) from a JSON spec, linting the layout first.
|
||||
- Pull text, tables (JSON/CSV), metadata, or form-field values out of a PDF.
|
||||
- Merge, split, rotate, extract page subsets, watermark, stamp text/images at coordinates, bookmark, or compress PDFs.
|
||||
- Export pages as PNGs for visual review or for OCR hand-off; set/clear document metadata; add/extract file attachments.
|
||||
- Fill or flatten AcroForm forms; encrypt or decrypt with passwords.
|
||||
- NOT for scanned/image-only PDFs (use `references/ocr-extraction.md`) and NOT for pixel-perfect HTML-to-PDF rendering (use a headless browser).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.10+ with `pypdf`, `reportlab`, `pdfplumber`:
|
||||
`python -m pip install pypdf reportlab pdfplumber`
|
||||
- Optional, for page rasterization (`pdf_page_image.py`, overlay rendering): `python -m pip install pypdfium2`, or poppler's `pdftoppm` on PATH. Scripts fall back pypdfium2 → pdftoppm and report `{"rendered": false, "missing": [...]}` (exit 0) when neither exists.
|
||||
- Each helper script checks imports lazily and prints an install hint if a dependency is missing.
|
||||
|
||||
## How to Run
|
||||
|
||||
All helpers live in `scripts/` and are argparse CLIs — run them with the `terminal` tool; every one supports `--help`. They read/write JSON strictly as UTF-8, print JSON results to stdout, and exit non-zero on failure.
|
||||
|
||||
```bash
|
||||
python scripts/pdf_create.py spec.json -o out.pdf # build PDF from JSON spec
|
||||
python scripts/pdf_make_form.py formspec.json -o form.pdf # build fillable AcroForm from JSON spec
|
||||
python scripts/pdf_form_layout.py formspec.json # lint form layout BEFORE building
|
||||
python scripts/pdf_form_layout.py formspec.json --render-overlay boxes.png [--pdf form.pdf]
|
||||
python scripts/pdf_read.py doc.pdf --text # per-page text (JSON)
|
||||
python scripts/pdf_read.py doc.pdf --tables --csv-dir t/ # tables to JSON + CSV files
|
||||
python scripts/pdf_read.py doc.pdf --meta # metadata, page sizes, encrypted/scanned flags
|
||||
python scripts/pdf_read.py form.pdf --fields # form fields: name, type, value
|
||||
python scripts/pdf_merge.py a.pdf b.pdf -o merged.pdf [--bookmarks]
|
||||
python scripts/pdf_split.py doc.pdf --pages 1-3,7 -o part.pdf [--rotate 90]
|
||||
python scripts/pdf_fill_form.py form.pdf --fields-json values.json -o filled.pdf [--flatten]
|
||||
python scripts/pdf_secure.py doc.pdf --encrypt -o enc.pdf --user-password your-password
|
||||
python scripts/pdf_secure.py enc.pdf --decrypt -o dec.pdf --password your-password
|
||||
python scripts/pdf_watermark.py doc.pdf --stamp mark.pdf -o stamped.pdf [--under]
|
||||
python scripts/pdf_stamp.py doc.pdf -o out.pdf --text "DRAFT" --x 150 --y 400 \
|
||||
--font-size 60 --rotation 45 --opacity 0.3 --color "#cc0000" [--pages 1-3]
|
||||
python scripts/pdf_stamp.py doc.pdf -o out.pdf --image sig.png --x 400 --y 60 --width 120
|
||||
python scripts/pdf_page_image.py doc.pdf --pages 1-3 --dpi 150 --out-dir imgs/
|
||||
python scripts/pdf_meta.py doc.pdf --set-meta --title "T" --author "A" -o out.pdf
|
||||
python scripts/pdf_meta.py doc.pdf --attach data.csv -o out.pdf
|
||||
python scripts/pdf_meta.py doc.pdf --list-attachments | --extract-attachments dir/
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Task | Tool | Command / API |
|
||||
|---|---|---|
|
||||
| Create doc (headings, tables, images) | reportlab platypus | `pdf_create.py spec.json -o out.pdf` |
|
||||
| Build fillable form | reportlab acroForm | `pdf_make_form.py formspec.json -o form.pdf` |
|
||||
| Lint form layout / overlay image | pure python + PIL | `pdf_form_layout.py formspec.json [--render-overlay o.png]` |
|
||||
| Per-page text | pdfplumber | `pdf_read.py f.pdf --text` |
|
||||
| Tables → JSON/CSV | pdfplumber | `pdf_read.py f.pdf --tables` |
|
||||
| Metadata / sizes / encrypted / scanned | pypdf + pdfplumber | `pdf_read.py f.pdf --meta` |
|
||||
| Merge (+ outline) | pypdf | `pdf_merge.py a.pdf b.pdf -o m.pdf` |
|
||||
| Split / extract / rotate | pypdf | `pdf_split.py f.pdf --pages 2-5 --rotate 90` |
|
||||
| List / fill / flatten form | pypdf | `pdf_read.py --fields`, `pdf_fill_form.py` |
|
||||
| Encrypt / decrypt (AES-256) | pypdf | `pdf_secure.py --encrypt/--decrypt` |
|
||||
| Watermark / stamp PDF page | pypdf | `pdf_watermark.py f.pdf --stamp w.pdf` |
|
||||
| Stamp text/image at coordinates | reportlab + pypdf | `pdf_stamp.py f.pdf --text "Sign here" --x 400 --y 60` |
|
||||
| Pages → PNG (review / OCR hand-off) | pypdfium2 or pdftoppm | `pdf_page_image.py f.pdf --pages 1-3 --out-dir imgs/` |
|
||||
| Set/clear metadata, attachments | pypdf | `pdf_meta.py --set-meta / --attach / --extract-attachments` |
|
||||
| Compress content streams | pypdf | `pdf_split.py f.pdf --pages 1-N --compress` |
|
||||
|
||||
## Procedure
|
||||
|
||||
1. **Inspect first.** Run `pdf_read.py file.pdf --meta`. Check `encrypted` (if true, decrypt first with `pdf_secure.py --decrypt`) and `likely_scanned_pages`. If pages are image-only, export them with `pdf_page_image.py --pages <scanned> --dpi 300 --out-dir imgs/` and hand the PNGs to the `references/ocr-extraction.md` skill — do not report empty text as "no content".
|
||||
2. **Create.** Write a JSON spec with `write_file` (elements: `heading`, `paragraph`, `table`, `image`, `pagebreak`; optional `title`/`author` metadata; page numbers are added automatically), then run `pdf_create.py`. Verify visually with `vision_analyze` on a rendered page image if layout matters.
|
||||
3. **Extract.** `--text` gives a JSON list of per-page strings; `--tables` gives row arrays per page and can also emit CSV files. Read results with `read_file`; never eyeball a binary PDF directly.
|
||||
4. **Manipulate.** `pdf_merge.py` concatenates and can add one bookmark per source file; `pdf_split.py` handles page ranges (1-based, e.g. `1-3,5,9-`), rotation in 90° steps, and `--compress`. Watermark by preparing a single-page stamp PDF (e.g. via `pdf_create.py`) and overlaying it with `pdf_watermark.py`; for one-liner stamps ("sign here", diagonal DRAFT, corner labels) use `pdf_stamp.py` with text or an image at explicit coordinates.
|
||||
5. **Build forms.** Write one form-spec JSON (fields with `label_box`/`entry_box` in PDF points — see `references/forms.md`), lint it with `pdf_form_layout.py` and fix every reported problem, optionally review the `--render-overlay` PNG with `vision_analyze`, then build with `pdf_make_form.py` and confirm with `pdf_read.py --fields`.
|
||||
6. **Fill forms.** List fields (`--fields`) to learn exact names and types, write a UTF-8 JSON of `{"FieldName": "value"}` with `write_file` (checkboxes accept `true`/`false`; radio/choice values must match the field's export options), then `pdf_fill_form.py`. Re-read with `--fields` to confirm values landed.
|
||||
7. **Metadata & attachments.** `pdf_meta.py --set-meta` writes Title/Author/Subject/Keywords (DocInfo); `--clear-meta` drops them; `--attach`/`--list-attachments`/`--extract-attachments` round-trip embedded files.
|
||||
8. **Secure.** Encrypt with distinct user/owner passwords and AES-256. To remove a password you know, `--decrypt` writes an unencrypted copy.
|
||||
9. **Verify** (see below) before reporting success.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Scanned PDFs**: empty `extract_text()` plus page images means there is no text layer. Route to `references/ocr-extraction.md`; do not fabricate text.
|
||||
- **Flattening limits**: `pdf_fill_form.py --flatten` uses pypdf's flatten support, which converts widget appearances into page content. It is reliable for plain text fields and checkboxes but can drop or misrender exotic widgets (rich text, custom appearance streams, some radio groups). Verify the flattened output visually with `vision_analyze`; for bulletproof flattening use an external renderer (e.g. Ghostscript or `pdftoppm`+reassembly) as a fallback.
|
||||
- **NeedAppearances**: after filling, viewers only render values if appearance streams exist. The fill script sets the AcroForm `NeedAppearances` flag so conforming viewers regenerate them; some minimal viewers ignore it — flatten if display fidelity matters.
|
||||
- **Non-Latin form values**: values are stored correctly (UTF-16), but the field's default font may lack glyphs, so a viewer can show blanks even though the data round-trips. Verify with `--fields`, not just visually.
|
||||
- **Compression expectations**: `--compress` only deflates content streams. Typical savings are 0–20%; it does nothing for PDFs dominated by images or already-compressed streams. It is not a substitute for image downsampling (Ghostscript territory).
|
||||
- **Permission flags don't enforce**: owner-password permission bits (no-print, no-copy) are polite requests that viewers may honor; any library (including pypdf) can read and strip them. Only the user password actually gates content via encryption. Never present permission flags as security.
|
||||
- **Table extraction is heuristic**: pdfplumber detects tables from ruling lines/word alignment; borderless or merged-cell tables may need `table_settings` tuning or manual cleanup.
|
||||
- **Page indexing**: helper CLIs take 1-based pages; pypdf APIs are 0-based. The scripts convert — don't double-convert.
|
||||
- **Rotated stamp text extraction**: pdfplumber's line grouping scrambles rotated glyphs (a 45° "DRAFT" extracts as stray letters); verify rotated stamps with `pypdf`'s `extract_text()` or a rendered image instead.
|
||||
- **Radio groups**: reportlab needs ≥2 `radio()` widgets per group, fills need the slashed export value (`"/red"`), and flatten fidelity is worst for radios — see `references/forms.md`.
|
||||
- **Metadata scope**: `pdf_meta.py` writes the classic DocInfo dictionary only; embedded XMP metadata (if any) is left untouched and may show different values in some viewers.
|
||||
- **PDF/A is out of scope**: pypdf/reportlab cannot produce or validate conformant PDF/A. If archival conformance is required, run Ghostscript via the `terminal` tool (e.g. `gs -dPDFA=2 -dPDFACompatibilityPolicy=1 -sColorConversionStrategy=UseDeviceIndependentColor -sDEVICE=pdfwrite -o out.pdf in.pdf` with a suitable ICC profile) and validate with veraPDF — both are external installs, and the result still needs validation, not assumption.
|
||||
- Rotation must be a multiple of 90; encrypted inputs must be decrypted before any other operation.
|
||||
|
||||
## Verification
|
||||
|
||||
- After create/merge/split: `pdf_read.py out.pdf --meta` — confirm `page_count`, and per-page `rotation` when you rotated.
|
||||
- After extraction: check the JSON is non-empty and spot-check a known string or cell.
|
||||
- Form design loop: `pdf_form_layout.py spec.json` must exit 0; then `--render-overlay boxes.png --pdf form.pdf` and review the PNG with `vision_analyze` (red = entry boxes with field names, blue = label boxes) asking about overlaps, misalignment, and labels detached from their fields. Iterate spec → lint → overlay until clean.
|
||||
- After building a form: `pdf_read.py form.pdf --fields` lists every spec field with the right type and options.
|
||||
- After form fill: `pdf_read.py filled.pdf --fields` and compare values (exact match, including non-ASCII).
|
||||
- After stamping: re-extract text (pypdf for rotated stamps) or render the page with `pdf_page_image.py` and inspect with `vision_analyze`.
|
||||
- After metadata/attachment edits: `pdf_read.py --meta` / `pdf_meta.py --list-attachments`, and re-extract an attachment to byte-compare.
|
||||
- After encrypt: `--meta` shows `"encrypted": true` and opening without a password fails; after decrypt, text extraction matches the original.
|
||||
- For anything visual (watermarks, flattened forms), render and inspect with `vision_analyze`.
|
||||
@@ -0,0 +1,99 @@
|
||||
# Building Fillable Forms: spec format and workflow
|
||||
|
||||
The same JSON spec drives both `pdf_form_layout.py` (design lint) and
|
||||
`pdf_make_form.py` (AcroForm build). Coordinates are PDF points, origin
|
||||
at the bottom-left of the page (1 pt = 1/72 inch; A4 is 595.27 x 841.89,
|
||||
letter is 612 x 792).
|
||||
|
||||
## Spec shape
|
||||
|
||||
```json
|
||||
{
|
||||
"title": "Example Intake Form",
|
||||
"author": "example-author",
|
||||
"page_size": "A4",
|
||||
"page_count": 1,
|
||||
"fields": [
|
||||
{"name": "surname", "type": "text", "page": 1,
|
||||
"label": "Surname", "label_box": [72, 700, 150, 714],
|
||||
"entry_box": [160, 696, 400, 716],
|
||||
"value": "", "tooltip": "Family name"},
|
||||
|
||||
{"name": "agree", "type": "checkbox", "page": 1,
|
||||
"label": "I agree", "label_box": [72, 660, 150, 674],
|
||||
"entry_box": [160, 658, 176, 674], "checked": false},
|
||||
|
||||
{"name": "color", "type": "radio", "page": 1,
|
||||
"label": "Color", "label_box": [72, 620, 150, 634],
|
||||
"entry_box": [160, 616, 400, 636],
|
||||
"options": ["red", "blue"], "value": "blue"},
|
||||
|
||||
{"name": "size", "type": "dropdown", "page": 1,
|
||||
"label": "Size", "label_box": [72, 580, 150, 594],
|
||||
"entry_box": [160, 576, 300, 596],
|
||||
"options": ["small", "large"], "value": "small"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- `page_size`: `"A4"`, `"letter"`, or `[width, height]` in points.
|
||||
- `page_count`: optional; extended automatically to the highest field page.
|
||||
- Boxes are `[x0, y0, x1, y1]` with `x0 < x1`, `y0 < y1`.
|
||||
- `label` is drawn as static text near `label_box`; omit it (and
|
||||
`label_box`) for unlabeled fields.
|
||||
- `radio`: the buttons are laid out left-to-right inside `entry_box`,
|
||||
one slot per option, each with a small static caption. `value`
|
||||
pre-selects an option by its export name.
|
||||
- `dropdown` maps to an AcroForm choice (combo) field.
|
||||
|
||||
## Field types → what pdf_read.py --fields reports
|
||||
|
||||
| Spec type | /FT | value format after fill |
|
||||
|---|---|---|
|
||||
| text | /Tx (`text`) | the string |
|
||||
| checkbox | /Btn (`button`) | `/Yes` or `/Off` |
|
||||
| radio | /Btn (`button`) | `/<export>`, e.g. `/red` |
|
||||
| dropdown | /Ch (`choice`) | the option string |
|
||||
|
||||
When filling with `pdf_fill_form.py`, checkboxes accept `true`/`false`;
|
||||
radio values need the leading slash (`"/red"`); dropdown values are the
|
||||
plain option string.
|
||||
|
||||
## Layout lint rules (pdf_form_layout.py)
|
||||
|
||||
Per field, on its declared page:
|
||||
|
||||
- boxes must be well-formed and inside the page bounds;
|
||||
- entry boxes must be at least 8x8 pt (12 pt tall for text/dropdown);
|
||||
- no two entry boxes on the same page may overlap (the second and later
|
||||
fields of an overlapping cluster are flagged);
|
||||
- a label must sit within 150 pt of its entry box and must not overlap it.
|
||||
|
||||
Exit code 0 = clean, 1 = at least one problem; the JSON report lists
|
||||
per-field `problems`. Lint the spec BEFORE building — fixing numbers in
|
||||
JSON is cheaper than debugging a rendered PDF.
|
||||
|
||||
## Visual review loop
|
||||
|
||||
```bash
|
||||
python3 scripts/pdf_form_layout.py spec.json --render-overlay overlay.png [--pdf built.pdf]
|
||||
```
|
||||
|
||||
Red rectangles = entry boxes (with field names), blue = label boxes.
|
||||
Without `--pdf` the overlay is drawn on a blank page (PIL-only, always
|
||||
works); with `--pdf` the real page is rasterized underneath
|
||||
(needs pypdfium2 or pdftoppm — otherwise the report says
|
||||
`"rendered": false` with install hints). Feed the PNG to `vision_analyze`
|
||||
and ask specifically about collisions, alignment, and stray labels.
|
||||
|
||||
## Radio-group quirks (reportlab + pypdf)
|
||||
|
||||
- reportlab requires at least two `radio()` calls per group; a
|
||||
single-option radio group produces a broken field.
|
||||
- Pre-selecting is done at build time via `"value"`; changing selection
|
||||
later via `pdf_fill_form.py` needs the slashed export name (`"/red"`).
|
||||
- Some viewers render reportlab radio appearances inconsistently after a
|
||||
pypdf fill; verify with `--fields` (data truth) plus a rendered page
|
||||
image (visual truth) rather than either alone.
|
||||
- Flattening radio groups is the least reliable flatten case — check the
|
||||
output image before shipping.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Natural-language PDF text editing with nano-pdf (merged from the nano-pdf skill)
|
||||
# nano-pdf
|
||||
|
||||
Edit PDFs using natural-language instructions. Point it at a page and describe what to change. For structural PDF work (merge, split, forms, watermarks, creation), see the `pdf` skill; for text extraction from scans, see `ocr-and-documents`.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
```bash
|
||||
# Install with uv (recommended — already available in Hermes)
|
||||
uv pip install nano-pdf
|
||||
|
||||
# Or with pip
|
||||
pip install nano-pdf
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
nano-pdf edit <file.pdf> <page_number> "<instruction>"
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
# Change a title on page 1
|
||||
nano-pdf edit deck.pdf 1 "Change the title to 'Q3 Results' and fix the typo in the subtitle"
|
||||
|
||||
# Update a date on a specific page
|
||||
nano-pdf edit report.pdf 3 "Update the date from January to February 2026"
|
||||
|
||||
# Fix content
|
||||
nano-pdf edit contract.pdf 2 "Change the client name from 'Acme Corp' to 'Acme Industries'"
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- Page numbers may be 0-based or 1-based depending on version — if the edit hits the wrong page, retry with ±1
|
||||
- Always verify the output PDF after editing (use `read_file` to check file size, or open it)
|
||||
- The tool uses an LLM under the hood — requires an API key (check `nano-pdf --help` for config)
|
||||
- Works well for text changes; complex layout modifications may need a different approach
|
||||
@@ -0,0 +1,165 @@
|
||||
# OCR & Document Text Extraction (merged from the ocr-and-documents skill)
|
||||
|
||||
Scripts referenced below live in this skill's scripts/ directory.
|
||||
# PDF & Document Extraction
|
||||
|
||||
For DOCX: see the `docx` skill (create/edit) or use `python-docx` for structured reads.
|
||||
For PPTX: see the `powerpoint` skill (full create/read/edit support).
|
||||
For PDF manipulation (merge, split, forms, watermarks, creation): see the `pdf` skill.
|
||||
This skill covers **text extraction from PDFs and scanned documents**.
|
||||
|
||||
> **Coming from a `read_file` EXTRACTION COVERAGE WARNING?** `read_file` auto-converts local PDFs but reads the text layer only; the warning footer lists the pages that yielded no text (scanned images). For a handful of pages, render + vision is fastest: `pdftoppm -jpeg -r 150 -f N -l N file.pdf /tmp/page` then `vision_analyze` each image. For bulk OCR of many pages, use marker-pdf below (Step 2).
|
||||
|
||||
## Step 1: Remote URL Available?
|
||||
|
||||
If the document has a URL, **always try `web_extract` first**:
|
||||
|
||||
```
|
||||
web_extract(urls=["https://arxiv.org/pdf/2402.03300"])
|
||||
web_extract(urls=["https://example.com/report.pdf"])
|
||||
```
|
||||
|
||||
This handles PDF-to-markdown conversion via Firecrawl with no local dependencies.
|
||||
|
||||
Only use local extraction when: the file is local, web_extract fails, or you need batch processing.
|
||||
|
||||
## Step 2: Choose Local Extractor
|
||||
|
||||
| Feature | pymupdf (~25MB) | marker-pdf (~3-5GB) |
|
||||
|---------|-----------------|---------------------|
|
||||
| **Text-based PDF** | ✅ | ✅ |
|
||||
| **Scanned PDF (OCR)** | ❌ | ✅ (90+ languages) |
|
||||
| **Tables** | ✅ (basic) | ✅ (high accuracy) |
|
||||
| **Equations / LaTeX** | ❌ | ✅ |
|
||||
| **Code blocks** | ❌ | ✅ |
|
||||
| **Forms** | ❌ | ✅ |
|
||||
| **Headers/footers removal** | ❌ | ✅ |
|
||||
| **Reading order detection** | ❌ | ✅ |
|
||||
| **Images extraction** | ✅ (embedded) | ✅ (with context) |
|
||||
| **Images → text (OCR)** | ❌ | ✅ |
|
||||
| **EPUB** | ✅ | ✅ |
|
||||
| **Markdown output** | ✅ (via pymupdf4llm) | ✅ (native, higher quality) |
|
||||
| **Install size** | ~25MB | ~3-5GB (PyTorch + models) |
|
||||
| **Speed** | Instant | ~1-14s/page (CPU), ~0.2s/page (GPU) |
|
||||
|
||||
**Decision**: Use pymupdf unless you need OCR, equations, forms, or complex layout analysis.
|
||||
|
||||
If the user needs marker capabilities but the system lacks ~5GB free disk:
|
||||
> "This document needs OCR/advanced extraction (marker-pdf), which requires ~5GB for PyTorch and models. Your system has [X]GB free. Options: free up space, provide a URL so I can use web_extract, or I can try pymupdf which works for text-based PDFs but not scanned documents or equations."
|
||||
|
||||
---
|
||||
|
||||
## pymupdf (lightweight)
|
||||
|
||||
```bash
|
||||
pip install pymupdf pymupdf4llm
|
||||
```
|
||||
|
||||
**Via helper script**:
|
||||
```bash
|
||||
python scripts/extract_pymupdf.py document.pdf # Plain text
|
||||
python scripts/extract_pymupdf.py document.pdf --markdown # Markdown
|
||||
python scripts/extract_pymupdf.py document.pdf --tables # Tables
|
||||
python scripts/extract_pymupdf.py document.pdf --images out/ # Extract images
|
||||
python scripts/extract_pymupdf.py document.pdf --metadata # Title, author, pages
|
||||
python scripts/extract_pymupdf.py document.pdf --pages 0-4 # Specific pages
|
||||
```
|
||||
|
||||
**Inline**:
|
||||
```bash
|
||||
python -c "
|
||||
import pymupdf
|
||||
doc = pymupdf.open('document.pdf')
|
||||
for page in doc:
|
||||
print(page.get_text())
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## marker-pdf (high-quality OCR)
|
||||
|
||||
```bash
|
||||
# Check disk space first
|
||||
python scripts/extract_marker.py --check
|
||||
|
||||
pip install marker-pdf
|
||||
```
|
||||
|
||||
**Via helper script**:
|
||||
```bash
|
||||
python scripts/extract_marker.py document.pdf # Markdown
|
||||
python scripts/extract_marker.py document.pdf --json # JSON with metadata
|
||||
python scripts/extract_marker.py document.pdf --output_dir out/ # Save images
|
||||
python scripts/extract_marker.py scanned.pdf # Scanned PDF (OCR)
|
||||
python scripts/extract_marker.py document.pdf --use_llm # LLM-boosted accuracy
|
||||
```
|
||||
|
||||
**CLI** (installed with marker-pdf):
|
||||
```bash
|
||||
marker_single document.pdf --output_dir ./output
|
||||
marker /path/to/folder --workers 4 # Batch
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Arxiv Papers
|
||||
|
||||
```
|
||||
# Abstract only (fast)
|
||||
web_extract(urls=["https://arxiv.org/abs/2402.03300"])
|
||||
|
||||
# Full paper
|
||||
web_extract(urls=["https://arxiv.org/pdf/2402.03300"])
|
||||
|
||||
# Search
|
||||
web_search(query="arxiv GRPO reinforcement learning 2026")
|
||||
```
|
||||
|
||||
## Split, Merge & Search
|
||||
|
||||
pymupdf handles these natively — use `execute_code` or inline Python:
|
||||
|
||||
```python
|
||||
# Split: extract pages 1-5 to a new PDF
|
||||
import pymupdf
|
||||
doc = pymupdf.open("report.pdf")
|
||||
new = pymupdf.open()
|
||||
for i in range(5):
|
||||
new.insert_pdf(doc, from_page=i, to_page=i)
|
||||
new.save("pages_1-5.pdf")
|
||||
```
|
||||
|
||||
```python
|
||||
# Merge multiple PDFs
|
||||
import pymupdf
|
||||
result = pymupdf.open()
|
||||
for path in ["a.pdf", "b.pdf", "c.pdf"]:
|
||||
result.insert_pdf(pymupdf.open(path))
|
||||
result.save("merged.pdf")
|
||||
```
|
||||
|
||||
```python
|
||||
# Search for text across all pages
|
||||
import pymupdf
|
||||
doc = pymupdf.open("report.pdf")
|
||||
for i, page in enumerate(doc):
|
||||
results = page.search_for("revenue")
|
||||
if results:
|
||||
print(f"Page {i+1}: {len(results)} match(es)")
|
||||
print(page.get_text("text"))
|
||||
```
|
||||
|
||||
No extra dependencies needed — pymupdf covers split, merge, search, and text extraction in one package.
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- `web_extract` is always first choice for URLs
|
||||
- pymupdf is the safe default — instant, no models, works everywhere
|
||||
- marker-pdf is for OCR, scanned docs, equations, complex layouts — install only when needed
|
||||
- Both helper scripts accept `--help` for full usage
|
||||
- marker-pdf downloads ~2.5GB of models to `~/.cache/huggingface/` on first use
|
||||
- For Word docs: `pip install python-docx` (better than OCR — parses actual structure)
|
||||
- For PowerPoint: see the `powerpoint` skill (uses python-pptx)
|
||||
@@ -0,0 +1,76 @@
|
||||
"""Shared page rasterizer with a fallback chain: pypdfium2 -> pdftoppm.
|
||||
|
||||
Returns PIL Images so callers can annotate/save. Not a CLI.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def available_backends() -> list[str]:
|
||||
"""Names of usable rasterizer backends, in preference order."""
|
||||
backends = []
|
||||
try:
|
||||
import pypdfium2 # noqa: F401
|
||||
backends.append("pypdfium2")
|
||||
except ImportError:
|
||||
pass
|
||||
if shutil.which("pdftoppm"):
|
||||
backends.append("pdftoppm")
|
||||
return backends
|
||||
|
||||
|
||||
def missing_hints() -> list[str]:
|
||||
"""Install hints for when no backend is available."""
|
||||
return [
|
||||
"python3 -m pip install pypdfium2",
|
||||
"poppler-utils (provides pdftoppm), e.g. apt-get install poppler-utils",
|
||||
]
|
||||
|
||||
|
||||
def rasterize_page(pdf_path: str, page: int, dpi: int = 150, password: str | None = None):
|
||||
"""Render one 1-based page to a PIL Image, or None if no backend works.
|
||||
|
||||
Raises ValueError for an out-of-range page when a backend is present.
|
||||
"""
|
||||
for backend in available_backends():
|
||||
if backend == "pypdfium2":
|
||||
return _via_pdfium(pdf_path, page, dpi, password)
|
||||
if backend == "pdftoppm":
|
||||
img = _via_pdftoppm(pdf_path, page, dpi, password)
|
||||
if img is not None:
|
||||
return img
|
||||
return None
|
||||
|
||||
|
||||
def _via_pdfium(pdf_path: str, page: int, dpi: int, password: str | None):
|
||||
import pypdfium2 as pdfium
|
||||
doc = pdfium.PdfDocument(pdf_path, password=password)
|
||||
try:
|
||||
if not 1 <= page <= len(doc):
|
||||
raise ValueError(f"page {page} out of range 1-{len(doc)}")
|
||||
bitmap = doc[page - 1].render(scale=dpi / 72.0)
|
||||
return bitmap.to_pil().convert("RGB")
|
||||
finally:
|
||||
doc.close()
|
||||
|
||||
|
||||
def _via_pdftoppm(pdf_path: str, page: int, dpi: int, password: str | None):
|
||||
from PIL import Image
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
prefix = str(Path(tmp) / "page")
|
||||
cmd = ["pdftoppm", "-png", "-r", str(dpi), "-f", str(page), "-l", str(page)]
|
||||
if password:
|
||||
cmd += ["-upw", password]
|
||||
cmd += [pdf_path, prefix]
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True, encoding="utf-8")
|
||||
if proc.returncode != 0:
|
||||
raise ValueError(f"pdftoppm failed: {proc.stderr.strip()}")
|
||||
produced = sorted(Path(tmp).glob("page*.png"))
|
||||
if not produced:
|
||||
raise ValueError(f"page {page} out of range (pdftoppm produced no image)")
|
||||
with Image.open(produced[0]) as img:
|
||||
return img.convert("RGB")
|
||||
@@ -0,0 +1,88 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract text from documents using marker-pdf. High-quality OCR + layout analysis.
|
||||
|
||||
Requires ~3-5GB disk (PyTorch + models downloaded on first use).
|
||||
Supports: PDF, DOCX, PPTX, XLSX, HTML, EPUB, images.
|
||||
|
||||
Usage:
|
||||
python extract_marker.py document.pdf
|
||||
python extract_marker.py document.pdf --output_dir ./output
|
||||
python extract_marker.py presentation.pptx
|
||||
python extract_marker.py spreadsheet.xlsx
|
||||
python extract_marker.py scanned_doc.pdf # OCR works here
|
||||
python extract_marker.py document.pdf --json # Structured output
|
||||
python extract_marker.py document.pdf --use_llm # LLM-boosted accuracy
|
||||
"""
|
||||
import sys
|
||||
import os
|
||||
|
||||
def convert(path, output_dir=None, output_format="markdown", use_llm=False):
|
||||
from marker.converters.pdf import PdfConverter
|
||||
from marker.models import create_model_dict
|
||||
from marker.config.parser import ConfigParser
|
||||
|
||||
config_dict = {}
|
||||
if use_llm:
|
||||
config_dict["use_llm"] = True
|
||||
|
||||
config_parser = ConfigParser(config_dict)
|
||||
models = create_model_dict()
|
||||
converter = PdfConverter(config=config_parser.generate_config_dict(), artifact_dict=models)
|
||||
rendered = converter(path)
|
||||
|
||||
if output_format == "json":
|
||||
import json
|
||||
print(json.dumps({
|
||||
"markdown": rendered.markdown,
|
||||
"metadata": rendered.metadata if hasattr(rendered, "metadata") else {},
|
||||
}, indent=2, ensure_ascii=False))
|
||||
else:
|
||||
print(rendered.markdown)
|
||||
|
||||
# Save images if output_dir specified
|
||||
if output_dir and hasattr(rendered, "images") and rendered.images:
|
||||
from pathlib import Path
|
||||
Path(output_dir).mkdir(parents=True, exist_ok=True)
|
||||
for name, img_data in rendered.images.items():
|
||||
img_path = os.path.join(output_dir, name)
|
||||
with open(img_path, "wb") as f:
|
||||
f.write(img_data)
|
||||
print(f"\nSaved {len(rendered.images)} image(s) to {output_dir}/", file=sys.stderr)
|
||||
|
||||
|
||||
def check_requirements():
|
||||
"""Check disk space before installing."""
|
||||
import shutil
|
||||
free_gb = shutil.disk_usage("/").free / (1024**3)
|
||||
if free_gb < 5:
|
||||
print(f"⚠️ Only {free_gb:.1f}GB free. marker-pdf needs ~5GB for PyTorch + models.")
|
||||
print("Use pymupdf instead (scripts/extract_pymupdf.py) or free up disk space.")
|
||||
sys.exit(1)
|
||||
print(f"✓ {free_gb:.1f}GB free — sufficient for marker-pdf")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Extract text from documents using marker-pdf (high-quality OCR + layout analysis)."
|
||||
)
|
||||
parser.add_argument("path", nargs="?", help="Document to convert (PDF, DOCX, PPTX, XLSX, HTML, EPUB, image)")
|
||||
parser.add_argument("--output_dir", help="Directory to save extracted images")
|
||||
parser.add_argument("--json", action="store_true", help="Structured JSON output instead of markdown")
|
||||
parser.add_argument("--use_llm", action="store_true", help="LLM-boosted accuracy")
|
||||
parser.add_argument("--check", action="store_true", help="Check disk space requirements and exit")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.check:
|
||||
check_requirements()
|
||||
sys.exit(0)
|
||||
if not args.path:
|
||||
parser.error("path is required unless --check is given")
|
||||
|
||||
convert(
|
||||
args.path,
|
||||
output_dir=args.output_dir,
|
||||
output_format="json" if args.json else "markdown",
|
||||
use_llm=args.use_llm,
|
||||
)
|
||||
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract text from documents using pymupdf. Lightweight (~25MB), no models.
|
||||
|
||||
Usage:
|
||||
python extract_pymupdf.py document.pdf
|
||||
python extract_pymupdf.py document.pdf --markdown
|
||||
python extract_pymupdf.py document.pdf --pages 0-4
|
||||
python extract_pymupdf.py document.pdf --images output_dir/
|
||||
python extract_pymupdf.py document.pdf --tables
|
||||
python extract_pymupdf.py document.pdf --metadata
|
||||
"""
|
||||
import sys
|
||||
import json
|
||||
|
||||
def extract_text(path, pages=None):
|
||||
import pymupdf
|
||||
doc = pymupdf.open(path)
|
||||
page_range = range(len(doc)) if pages is None else pages
|
||||
for i in page_range:
|
||||
if i < len(doc):
|
||||
print(f"\n--- Page {i+1}/{len(doc)} ---\n")
|
||||
print(doc[i].get_text())
|
||||
|
||||
def extract_markdown(path, pages=None):
|
||||
import pymupdf4llm
|
||||
md = pymupdf4llm.to_markdown(path, pages=pages)
|
||||
print(md)
|
||||
|
||||
def extract_tables(path):
|
||||
import pymupdf
|
||||
doc = pymupdf.open(path)
|
||||
for i, page in enumerate(doc):
|
||||
tables = page.find_tables()
|
||||
for j, table in enumerate(tables.tables):
|
||||
print(f"\n--- Page {i+1}, Table {j+1} ---\n")
|
||||
df = table.to_pandas()
|
||||
print(df.to_markdown(index=False))
|
||||
|
||||
def extract_images(path, output_dir):
|
||||
import pymupdf
|
||||
from pathlib import Path
|
||||
Path(output_dir).mkdir(parents=True, exist_ok=True)
|
||||
doc = pymupdf.open(path)
|
||||
count = 0
|
||||
for i, page in enumerate(doc):
|
||||
for img_idx, img in enumerate(page.get_images(full=True)):
|
||||
xref = img[0]
|
||||
pix = pymupdf.Pixmap(doc, xref)
|
||||
if pix.n >= 5:
|
||||
pix = pymupdf.Pixmap(pymupdf.csRGB, pix)
|
||||
out_path = f"{output_dir}/page{i+1}_img{img_idx+1}.png"
|
||||
pix.save(out_path)
|
||||
count += 1
|
||||
print(f"Extracted {count} images to {output_dir}/")
|
||||
|
||||
def show_metadata(path):
|
||||
import pymupdf
|
||||
doc = pymupdf.open(path)
|
||||
print(json.dumps({
|
||||
"pages": len(doc),
|
||||
"title": doc.metadata.get("title", ""),
|
||||
"author": doc.metadata.get("author", ""),
|
||||
"subject": doc.metadata.get("subject", ""),
|
||||
"creator": doc.metadata.get("creator", ""),
|
||||
"producer": doc.metadata.get("producer", ""),
|
||||
"format": doc.metadata.get("format", ""),
|
||||
}, indent=2))
|
||||
|
||||
if __name__ == "__main__":
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Extract text/tables/images/metadata from documents using pymupdf (lightweight, no models)."
|
||||
)
|
||||
parser.add_argument("path", help="Document to read")
|
||||
parser.add_argument("--pages", help="Page selection: N or START-END (0-indexed)")
|
||||
parser.add_argument("--markdown", action="store_true", help="Markdown output via pymupdf4llm")
|
||||
parser.add_argument("--tables", action="store_true", help="Extract tables as markdown")
|
||||
parser.add_argument("--images", nargs="?", const="./images", metavar="OUTPUT_DIR",
|
||||
help="Extract embedded images to OUTPUT_DIR (default ./images)")
|
||||
parser.add_argument("--metadata", action="store_true", help="Show document metadata as JSON")
|
||||
args = parser.parse_args()
|
||||
|
||||
pages = None
|
||||
if args.pages:
|
||||
if "-" in args.pages:
|
||||
start, end = args.pages.split("-")
|
||||
pages = list(range(int(start), int(end) + 1))
|
||||
else:
|
||||
pages = [int(args.pages)]
|
||||
|
||||
if args.metadata:
|
||||
show_metadata(args.path)
|
||||
elif args.tables:
|
||||
extract_tables(args.path)
|
||||
elif args.images is not None:
|
||||
extract_images(args.path, args.images)
|
||||
elif args.markdown:
|
||||
extract_markdown(args.path, pages=pages)
|
||||
else:
|
||||
extract_text(args.path, pages=pages)
|
||||
@@ -0,0 +1,130 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Create a PDF from a JSON spec using reportlab platypus.
|
||||
|
||||
Spec format (UTF-8 JSON):
|
||||
{
|
||||
"title": "Example Report",
|
||||
"author": "example-author",
|
||||
"page_size": "A4", // or "letter" (default: A4)
|
||||
"page_numbers": true, // default true
|
||||
"elements": [
|
||||
{"type": "heading", "text": "Section 1", "level": 1},
|
||||
{"type": "paragraph", "text": "Body text..."},
|
||||
{"type": "table", "rows": [["H1", "H2"], ["a", "b"]], "header": true},
|
||||
{"type": "image", "path": "chart.png", "width": 400},
|
||||
{"type": "pagebreak"}
|
||||
]
|
||||
}
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
|
||||
|
||||
def _reconfigure_stdio() -> None:
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def build_pdf(spec: dict, out_path: str) -> int:
|
||||
try:
|
||||
from reportlab.lib import colors
|
||||
from reportlab.lib.pagesizes import A4, letter
|
||||
from reportlab.lib.styles import getSampleStyleSheet
|
||||
from reportlab.lib.units import inch
|
||||
from reportlab.platypus import (
|
||||
Image,
|
||||
PageBreak,
|
||||
Paragraph,
|
||||
SimpleDocTemplate,
|
||||
Spacer,
|
||||
Table,
|
||||
TableStyle,
|
||||
)
|
||||
except ImportError:
|
||||
print("Missing dependency: install with 'python3 -m pip install reportlab'", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
page_size = letter if str(spec.get("page_size", "A4")).lower() == "letter" else A4
|
||||
styles = getSampleStyleSheet()
|
||||
story = []
|
||||
for el in spec.get("elements", []):
|
||||
etype = el.get("type")
|
||||
if etype == "heading":
|
||||
level = min(max(int(el.get("level", 1)), 1), 3)
|
||||
story.append(Paragraph(el.get("text", ""), styles[f"Heading{level}"]))
|
||||
elif etype == "paragraph":
|
||||
story.append(Paragraph(el.get("text", ""), styles["BodyText"]))
|
||||
story.append(Spacer(1, 6))
|
||||
elif etype == "table":
|
||||
rows = el.get("rows", [])
|
||||
if not rows:
|
||||
continue
|
||||
table = Table(rows, repeatRows=1 if el.get("header", True) else 0)
|
||||
style = [
|
||||
("GRID", (0, 0), (-1, -1), 0.5, colors.grey),
|
||||
("VALIGN", (0, 0), (-1, -1), "TOP"),
|
||||
]
|
||||
if el.get("header", True):
|
||||
style += [
|
||||
("BACKGROUND", (0, 0), (-1, 0), colors.lightgrey),
|
||||
("FONTNAME", (0, 0), (-1, 0), "Helvetica-Bold"),
|
||||
]
|
||||
table.setStyle(TableStyle(style))
|
||||
story.append(table)
|
||||
story.append(Spacer(1, 10))
|
||||
elif etype == "image":
|
||||
kwargs = {}
|
||||
if el.get("width"):
|
||||
kwargs["width"] = float(el["width"])
|
||||
if el.get("height"):
|
||||
kwargs["height"] = float(el["height"])
|
||||
img = Image(el["path"], **kwargs)
|
||||
if "width" in kwargs and "height" not in kwargs:
|
||||
# keep aspect ratio
|
||||
ratio = img.imageHeight / img.imageWidth
|
||||
img.drawWidth = kwargs["width"]
|
||||
img.drawHeight = kwargs["width"] * ratio
|
||||
story.append(img)
|
||||
story.append(Spacer(1, 10))
|
||||
elif etype == "pagebreak":
|
||||
story.append(PageBreak())
|
||||
else:
|
||||
print(f"Warning: unknown element type {etype!r}, skipped", file=sys.stderr)
|
||||
|
||||
def draw_page_number(canvas, doc):
|
||||
if spec.get("page_numbers", True):
|
||||
canvas.saveState()
|
||||
canvas.setFont("Helvetica", 9)
|
||||
canvas.drawCentredString(page_size[0] / 2.0, 0.5 * inch, f"Page {doc.page}")
|
||||
canvas.restoreState()
|
||||
|
||||
doc = SimpleDocTemplate(
|
||||
out_path,
|
||||
pagesize=page_size,
|
||||
title=spec.get("title", ""),
|
||||
author=spec.get("author", ""),
|
||||
)
|
||||
doc.build(story, onFirstPage=draw_page_number, onLaterPages=draw_page_number)
|
||||
print(json.dumps({"output": out_path, "elements": len(spec.get("elements", []))}))
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
_reconfigure_stdio()
|
||||
parser = argparse.ArgumentParser(description="Create a PDF from a JSON spec (reportlab).")
|
||||
parser.add_argument("spec", help="Path to UTF-8 JSON spec file")
|
||||
parser.add_argument("-o", "--output", required=True, help="Output PDF path")
|
||||
args = parser.parse_args()
|
||||
with open(args.spec, encoding="utf-8") as fh:
|
||||
spec = json.load(fh)
|
||||
return build_pdf(spec, args.output)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,97 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fill AcroForm fields from a UTF-8 JSON file; optionally flatten.
|
||||
|
||||
The JSON is a flat object: {"FieldName": "value", "Agree": true, ...}
|
||||
- text fields: strings
|
||||
- checkboxes: true/false (or an explicit on-state name like "/Yes")
|
||||
- radio / dropdown: the export value as a string (see pdf_read.py --fields "options")
|
||||
|
||||
Sets NeedAppearances so conforming viewers regenerate field appearances.
|
||||
Flattening uses pypdf appearance merging; verify visually for exotic widgets.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
|
||||
|
||||
def main() -> int:
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
parser = argparse.ArgumentParser(description="Fill PDF AcroForm fields from JSON (pypdf).")
|
||||
parser.add_argument("pdf", help="Input form PDF")
|
||||
parser.add_argument("--fields-json", required=True, help="UTF-8 JSON file of field values")
|
||||
parser.add_argument("-o", "--output", required=True, help="Output PDF path")
|
||||
parser.add_argument("--flatten", action="store_true",
|
||||
help="Make fields read-only and burn appearances into the page")
|
||||
parser.add_argument("--password", help="Password if the input is encrypted")
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
from pypdf import PdfReader, PdfWriter
|
||||
from pypdf.generic import BooleanObject, NameObject
|
||||
except ImportError:
|
||||
print("Missing dependency: install with 'python3 -m pip install pypdf'", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
with open(args.fields_json, encoding="utf-8") as fh:
|
||||
values = json.load(fh)
|
||||
|
||||
reader = PdfReader(args.pdf)
|
||||
if reader.is_encrypted:
|
||||
if args.password is None or not reader.decrypt(args.password):
|
||||
print("Error: input is encrypted; pass --password", file=sys.stderr)
|
||||
return 3
|
||||
available = set((reader.get_fields() or {}).keys())
|
||||
missing = [name for name in values if name not in available]
|
||||
if missing:
|
||||
print(f"Warning: fields not found in form, skipped: {missing}", file=sys.stderr)
|
||||
|
||||
writer = PdfWriter()
|
||||
writer.append(reader)
|
||||
|
||||
# Normalize checkbox booleans to the field's actual on-state name
|
||||
# (e.g. "/Yes"): pypdf does not reliably map bare True to the on-state.
|
||||
field_info = reader.get_fields() or {}
|
||||
fill = {}
|
||||
for name, value in values.items():
|
||||
if name not in available:
|
||||
continue
|
||||
if isinstance(value, bool):
|
||||
states = [str(s) for s in (field_info[name].get("/_States_") or [])]
|
||||
on_state = next((s for s in states if s != "/Off"), "/Yes")
|
||||
value = on_state if value else "/Off"
|
||||
fill[name] = value
|
||||
for page in writer.pages:
|
||||
writer.update_page_form_field_values(page, fill, auto_regenerate=False)
|
||||
|
||||
# Set NeedAppearances so viewers render values even without appearance streams.
|
||||
root = writer._root_object
|
||||
if "/AcroForm" in root:
|
||||
root["/AcroForm"][NameObject("/NeedAppearances")] = BooleanObject(True)
|
||||
|
||||
flattened = False
|
||||
if args.flatten:
|
||||
try:
|
||||
# pypdf >= 5: flatten via update with flags making fields read-only,
|
||||
# then remove interactivity by merging appearances.
|
||||
for page in writer.pages:
|
||||
writer.update_page_form_field_values(page, fill, flags=1) # 1 = ReadOnly
|
||||
flattened = True
|
||||
except Exception as exc:
|
||||
print(f"Warning: flatten step failed ({exc}); output keeps interactive fields",
|
||||
file=sys.stderr)
|
||||
|
||||
with open(args.output, "wb") as fh:
|
||||
writer.write(fh)
|
||||
print(json.dumps({"output": args.output, "filled": sorted(fill), "skipped": missing,
|
||||
"flattened": flattened}, ensure_ascii=False))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,168 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Validate a form-spec layout BEFORE building the PDF, with optional
|
||||
visual overlay rendering for review with a vision model.
|
||||
|
||||
Input is the same JSON spec pdf_make_form.py consumes: each field has
|
||||
"page", "label_box" and "entry_box" as [x0, y0, x1, y1] in PDF points
|
||||
(origin bottom-left). Checks per field:
|
||||
- boxes lie within the page bounds
|
||||
- boxes are well-formed (x0 < x1, y0 < y1)
|
||||
- entry boxes meet minimum sizes (default 8x8 pt; 12 pt height for text)
|
||||
- no two entry boxes on the same page overlap
|
||||
- the label sits near its entry box (default within 150 pt gap)
|
||||
|
||||
Prints a JSON report {"ok": bool, "fields": [...], "errors": N};
|
||||
exit 0 when clean, 1 when any check fails.
|
||||
|
||||
--render-overlay OUT.png rasterizes --overlay-page (default 1) of an
|
||||
existing PDF (--pdf; a blank page of spec size if omitted) and draws
|
||||
label boxes (blue) and entry boxes (red) with field names, for
|
||||
review with `vision_analyze`. If no rasterizer (pypdfium2/pdftoppm) is
|
||||
available the overlay is skipped with {"rendered": false, "missing": [...]}
|
||||
and validation exit status is unchanged.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
|
||||
MIN_W = 8.0
|
||||
MIN_H = 8.0
|
||||
MIN_TEXT_H = 12.0
|
||||
MAX_LABEL_GAP = 150.0
|
||||
|
||||
|
||||
def _boxes_overlap(a, b) -> bool:
|
||||
return not (a[2] <= b[0] or b[2] <= a[0] or a[3] <= b[1] or b[3] <= a[1])
|
||||
|
||||
|
||||
def _box_gap(a, b) -> float:
|
||||
dx = max(b[0] - a[2], a[0] - b[2], 0.0)
|
||||
dy = max(b[1] - a[3], a[1] - b[3], 0.0)
|
||||
return (dx ** 2 + dy ** 2) ** 0.5
|
||||
|
||||
|
||||
def _page_size(spec: dict) -> tuple[float, float]:
|
||||
sizes = {"a4": (595.27, 841.89), "letter": (612.0, 792.0)}
|
||||
ps = spec.get("page_size", "A4")
|
||||
if isinstance(ps, (list, tuple)) and len(ps) == 2:
|
||||
return float(ps[0]), float(ps[1])
|
||||
return sizes.get(str(ps).lower(), sizes["a4"])
|
||||
|
||||
|
||||
def _check_box(box, width, height, min_w, min_h, kind) -> list[str]:
|
||||
problems = []
|
||||
if box is None:
|
||||
return [f"{kind}_box missing"]
|
||||
x0, y0, x1, y1 = (float(v) for v in box)
|
||||
if x0 >= x1 or y0 >= y1:
|
||||
problems.append(f"{kind}_box malformed (need x0<x1 and y0<y1): {box}")
|
||||
return problems
|
||||
if x0 < 0 or y0 < 0 or x1 > width or y1 > height:
|
||||
problems.append(f"{kind}_box outside page bounds {width}x{height}: {box}")
|
||||
if x1 - x0 < min_w or y1 - y0 < min_h:
|
||||
problems.append(f"{kind}_box below minimum size {min_w}x{min_h}: {box}")
|
||||
return problems
|
||||
|
||||
|
||||
def validate(spec: dict) -> dict:
|
||||
width, height = _page_size(spec)
|
||||
fields = spec.get("fields", [])
|
||||
report = []
|
||||
entry_boxes: dict[int, list[tuple[str, list[float]]]] = {}
|
||||
for f in fields:
|
||||
name = f.get("name", "?")
|
||||
page = int(f.get("page", 1))
|
||||
problems = []
|
||||
min_h = MIN_TEXT_H if f.get("type", "text") in ("text", "dropdown") else MIN_H
|
||||
entry = f.get("entry_box")
|
||||
problems += _check_box(entry, width, height, MIN_W, min_h, "entry")
|
||||
label = f.get("label_box")
|
||||
if f.get("label"):
|
||||
problems += _check_box(label, width, height, 4, 4, "label")
|
||||
if entry and label and len(problems) == 0:
|
||||
gap = _box_gap([float(v) for v in label], [float(v) for v in entry])
|
||||
if gap > MAX_LABEL_GAP:
|
||||
problems.append(f"label is {gap:.0f}pt from its entry box (max {MAX_LABEL_GAP:.0f})")
|
||||
if _boxes_overlap([float(v) for v in label], [float(v) for v in entry]):
|
||||
problems.append("label_box overlaps its own entry_box")
|
||||
if entry and not any("malformed" in p or "missing" in p for p in problems):
|
||||
ebox = [float(v) for v in entry]
|
||||
for other_name, other_box in entry_boxes.get(page, []):
|
||||
if _boxes_overlap(ebox, other_box):
|
||||
problems.append(f"entry_box overlaps field {other_name!r}")
|
||||
entry_boxes.setdefault(page, []).append((name, ebox))
|
||||
report.append({"name": name, "page": page, "ok": not problems, "problems": problems})
|
||||
errors = sum(1 for r in report if not r["ok"])
|
||||
return {"ok": errors == 0, "page_size": [width, height],
|
||||
"field_count": len(report), "errors": errors, "fields": report}
|
||||
|
||||
|
||||
def render_overlay(spec: dict, pdf_path: str | None, page: int, out_png: str,
|
||||
dpi: int = 100) -> dict:
|
||||
sys.path.insert(0, str(__import__("pathlib").Path(__file__).resolve().parent))
|
||||
import _raster
|
||||
if not _raster.available_backends() and pdf_path:
|
||||
return {"rendered": False, "missing": _raster.missing_hints()}
|
||||
from PIL import Image, ImageDraw
|
||||
width, height = _page_size(spec)
|
||||
if pdf_path:
|
||||
img = _raster.rasterize_page(pdf_path, page, dpi=dpi)
|
||||
if img is None:
|
||||
return {"rendered": False, "missing": _raster.missing_hints()}
|
||||
scale = img.width / width
|
||||
else:
|
||||
scale = dpi / 72.0
|
||||
img = Image.new("RGB", (int(width * scale), int(height * scale)), "white")
|
||||
draw = ImageDraw.Draw(img)
|
||||
|
||||
def to_px(box):
|
||||
x0, y0, x1, y1 = (float(v) for v in box)
|
||||
return [x0 * scale, img.height - y1 * scale, x1 * scale, img.height - y0 * scale]
|
||||
|
||||
for f in spec.get("fields", []):
|
||||
if int(f.get("page", 1)) != page:
|
||||
continue
|
||||
if f.get("entry_box"):
|
||||
px = to_px(f["entry_box"])
|
||||
draw.rectangle(px, outline=(220, 30, 30), width=2)
|
||||
draw.text((px[0] + 2, px[1] + 2), str(f.get("name", "?")), fill=(220, 30, 30))
|
||||
if f.get("label_box"):
|
||||
draw.rectangle(to_px(f["label_box"]), outline=(30, 60, 220), width=2)
|
||||
img.save(out_png)
|
||||
return {"rendered": True, "overlay": out_png, "page": page,
|
||||
"legend": {"entry_box": "red", "label_box": "blue"}}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Validate form-spec layout (boxes, overlaps, label pairing); "
|
||||
"optionally render an annotated overlay image.")
|
||||
parser.add_argument("spec", help="Form spec JSON (same format as pdf_make_form.py)")
|
||||
parser.add_argument("--pdf", help="Existing PDF to rasterize under the overlay "
|
||||
"(blank page if omitted)")
|
||||
parser.add_argument("--render-overlay", metavar="OUT_PNG",
|
||||
help="Write an annotated PNG for visual review")
|
||||
parser.add_argument("--overlay-page", type=int, default=1, help="1-based page (default 1)")
|
||||
parser.add_argument("--dpi", type=int, default=100, help="Overlay render DPI (default 100)")
|
||||
args = parser.parse_args()
|
||||
|
||||
with open(args.spec, encoding="utf-8") as fh:
|
||||
spec = json.load(fh)
|
||||
result = validate(spec)
|
||||
if args.render_overlay:
|
||||
result["overlay"] = render_overlay(spec, args.pdf, args.overlay_page,
|
||||
args.render_overlay, args.dpi)
|
||||
json.dump(result, sys.stdout, ensure_ascii=False, indent=2)
|
||||
print()
|
||||
return 0 if result["ok"] else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Create a fillable AcroForm PDF from a JSON spec (reportlab canvas.acroForm).
|
||||
|
||||
Spec format (UTF-8 JSON; coordinates in PDF points, origin bottom-left):
|
||||
{
|
||||
"title": "Example Intake Form",
|
||||
"page_size": "A4", // or "letter" or [width, height]
|
||||
"page_count": 1,
|
||||
"fields": [
|
||||
{"name": "surname", "type": "text", "page": 1,
|
||||
"label": "Surname", "label_box": [72, 700, 150, 714],
|
||||
"entry_box": [160, 696, 400, 716], "value": "", "tooltip": "Family name"},
|
||||
{"name": "agree", "type": "checkbox", "page": 1,
|
||||
"label": "I agree", "label_box": [72, 660, 150, 674],
|
||||
"entry_box": [160, 658, 176, 674], "checked": false},
|
||||
{"name": "color", "type": "radio", "page": 1,
|
||||
"label": "Color", "label_box": [72, 620, 150, 634],
|
||||
"entry_box": [160, 616, 400, 636], "options": ["red", "blue"],
|
||||
"value": "red"},
|
||||
{"name": "size", "type": "dropdown", "page": 1,
|
||||
"label": "Size", "label_box": [72, 580, 150, 594],
|
||||
"entry_box": [160, 576, 300, 596], "options": ["small", "large"],
|
||||
"value": "small"}
|
||||
]
|
||||
}
|
||||
The same spec (label_box/entry_box/page) is what pdf_form_layout.py validates,
|
||||
so lint the layout first, then build.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
|
||||
|
||||
def _reconfigure_stdio() -> None:
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _page_size(spec: dict):
|
||||
from reportlab.lib.pagesizes import A4, letter
|
||||
ps = spec.get("page_size", "A4")
|
||||
if isinstance(ps, (list, tuple)) and len(ps) == 2:
|
||||
return float(ps[0]), float(ps[1])
|
||||
return letter if str(ps).lower() == "letter" else A4
|
||||
|
||||
|
||||
def build_form(spec: dict, out_path: str) -> int:
|
||||
try:
|
||||
from reportlab.lib import colors
|
||||
from reportlab.pdfgen import canvas
|
||||
except ImportError:
|
||||
print("Missing dependency: install with 'python3 -m pip install reportlab'", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
width, height = _page_size(spec)
|
||||
page_count = int(spec.get("page_count", 1))
|
||||
fields = spec.get("fields", [])
|
||||
by_page: dict[int, list[dict]] = {}
|
||||
for f in fields:
|
||||
by_page.setdefault(int(f.get("page", 1)), []).append(f)
|
||||
page_count = max([page_count, *by_page.keys()]) if by_page else page_count
|
||||
|
||||
c = canvas.Canvas(out_path, pagesize=(width, height))
|
||||
if spec.get("title"):
|
||||
c.setTitle(str(spec["title"]))
|
||||
if spec.get("author"):
|
||||
c.setAuthor(str(spec["author"]))
|
||||
form = c.acroForm
|
||||
created = []
|
||||
|
||||
for pageno in range(1, page_count + 1):
|
||||
for f in by_page.get(pageno, []):
|
||||
name = f["name"]
|
||||
ftype = f.get("type", "text")
|
||||
ex0, ey0, ex1, ey1 = (float(v) for v in f["entry_box"])
|
||||
ew, eh = ex1 - ex0, ey1 - ey0
|
||||
if f.get("label"):
|
||||
lx, ly = (float(f["label_box"][0]), float(f["label_box"][1])) \
|
||||
if f.get("label_box") else (ex0 - 90, ey0 + 4)
|
||||
c.setFont("Helvetica", float(f.get("label_size", 10)))
|
||||
c.setFillColor(colors.black)
|
||||
c.drawString(lx, ly + 2, str(f["label"]))
|
||||
tooltip = f.get("tooltip", "")
|
||||
if ftype == "text":
|
||||
form.textfield(name=name, x=ex0, y=ey0, width=ew, height=eh,
|
||||
value=str(f.get("value", "")), tooltip=tooltip,
|
||||
borderWidth=0.5, forceBorder=True)
|
||||
elif ftype == "checkbox":
|
||||
size = min(ew, eh)
|
||||
form.checkbox(name=name, x=ex0, y=ey0, size=size,
|
||||
checked=bool(f.get("checked", False)),
|
||||
buttonStyle="check", tooltip=tooltip,
|
||||
borderWidth=0.5, forceBorder=True)
|
||||
elif ftype == "radio":
|
||||
options = f.get("options", [])
|
||||
if not options:
|
||||
print(f"Warning: radio {name!r} has no options, skipped", file=sys.stderr)
|
||||
continue
|
||||
size = min(eh, ew / max(len(options), 1) * 0.5, 16)
|
||||
slot = ew / len(options)
|
||||
sel = f.get("value")
|
||||
c.setFont("Helvetica", 8)
|
||||
for i, opt in enumerate(options):
|
||||
ox = ex0 + i * slot
|
||||
form.radio(name=name, value=str(opt), x=ox, y=ey0, size=size,
|
||||
selected=(str(opt) == str(sel)), buttonStyle="circle",
|
||||
borderWidth=0.5, forceBorder=True)
|
||||
c.drawString(ox + size + 2, ey0 + size / 3, str(opt))
|
||||
elif ftype == "dropdown":
|
||||
options = [str(o) for o in f.get("options", [])]
|
||||
value = str(f.get("value", options[0] if options else ""))
|
||||
form.choice(name=name, x=ex0, y=ey0, width=ew, height=eh,
|
||||
options=options, value=value, tooltip=tooltip,
|
||||
borderWidth=0.5, forceBorder=True)
|
||||
else:
|
||||
print(f"Warning: unknown field type {ftype!r} for {name!r}, skipped",
|
||||
file=sys.stderr)
|
||||
continue
|
||||
created.append({"name": name, "type": ftype, "page": pageno})
|
||||
c.showPage()
|
||||
c.save()
|
||||
print(json.dumps({"output": out_path, "pages": page_count, "fields": created},
|
||||
ensure_ascii=False))
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
_reconfigure_stdio()
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Create a fillable AcroForm PDF from a JSON spec (reportlab).")
|
||||
parser.add_argument("spec", help="Path to UTF-8 JSON form spec")
|
||||
parser.add_argument("-o", "--output", required=True, help="Output PDF path")
|
||||
args = parser.parse_args()
|
||||
with open(args.spec, encoding="utf-8") as fh:
|
||||
spec = json.load(fh)
|
||||
return build_form(spec, args.output)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,50 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Merge multiple PDFs into one, optionally adding a bookmark per source file."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
|
||||
def main() -> int:
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
parser = argparse.ArgumentParser(description="Merge PDFs (pypdf).")
|
||||
parser.add_argument("inputs", nargs="+", help="Input PDF paths, in order")
|
||||
parser.add_argument("-o", "--output", required=True, help="Output PDF path")
|
||||
parser.add_argument("--bookmarks", action="store_true",
|
||||
help="Add a top-level bookmark per input file (its basename)")
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
from pypdf import PdfReader, PdfWriter
|
||||
except ImportError:
|
||||
print("Missing dependency: install with 'python3 -m pip install pypdf'", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
writer = PdfWriter()
|
||||
total = 0
|
||||
for path in args.inputs:
|
||||
reader = PdfReader(path)
|
||||
if reader.is_encrypted:
|
||||
print(f"Error: {path} is encrypted; decrypt it first with pdf_secure.py --decrypt", file=sys.stderr)
|
||||
return 3
|
||||
start = total
|
||||
for page in reader.pages:
|
||||
writer.add_page(page)
|
||||
total += 1
|
||||
if args.bookmarks:
|
||||
writer.add_outline_item(os.path.splitext(os.path.basename(path))[0], start)
|
||||
with open(args.output, "wb") as fh:
|
||||
writer.write(fh)
|
||||
print(json.dumps({"output": args.output, "inputs": len(args.inputs), "page_count": total}))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,115 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Document metadata and file attachments for PDFs (pypdf).
|
||||
|
||||
Modes (one required):
|
||||
--set-meta set metadata keys given via --title/--author/...
|
||||
--clear-meta drop all document info metadata
|
||||
--attach FILE embed a file attachment
|
||||
--list-attachments list embedded attachment names
|
||||
--extract-attachments DIR write all attachments into DIR
|
||||
|
||||
Metadata note: values are stored in the classic DocInfo dictionary
|
||||
(Title/Author/Subject/Keywords). XMP metadata, if present, is not
|
||||
rewritten and may disagree in sophisticated viewers.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main() -> int:
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
parser = argparse.ArgumentParser(description="Set/clear PDF metadata; manage attachments.")
|
||||
parser.add_argument("pdf", help="Input PDF path")
|
||||
mode = parser.add_mutually_exclusive_group(required=True)
|
||||
mode.add_argument("--set-meta", action="store_true", help="Set metadata fields")
|
||||
mode.add_argument("--clear-meta", action="store_true", help="Remove all DocInfo metadata")
|
||||
mode.add_argument("--attach", metavar="FILE", help="Embed FILE as an attachment")
|
||||
mode.add_argument("--list-attachments", action="store_true", help="List attachment names")
|
||||
mode.add_argument("--extract-attachments", metavar="DIR", help="Extract attachments into DIR")
|
||||
parser.add_argument("-o", "--output", help="Output PDF (required for write modes)")
|
||||
parser.add_argument("--title")
|
||||
parser.add_argument("--author")
|
||||
parser.add_argument("--subject")
|
||||
parser.add_argument("--keywords")
|
||||
parser.add_argument("--password", help="Password if the input is encrypted")
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
from pypdf import PdfReader, PdfWriter
|
||||
except ImportError:
|
||||
print("Missing dependency: install with 'python3 -m pip install pypdf'", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
reader = PdfReader(args.pdf)
|
||||
if reader.is_encrypted:
|
||||
if args.password is None or not reader.decrypt(args.password):
|
||||
print("Error: input is encrypted; pass --password", file=sys.stderr)
|
||||
return 3
|
||||
|
||||
if args.list_attachments:
|
||||
names = list(reader.attachments.keys())
|
||||
json.dump({"attachment_count": len(names), "attachments": names}, sys.stdout,
|
||||
ensure_ascii=False, indent=2)
|
||||
print()
|
||||
return 0
|
||||
|
||||
if args.extract_attachments:
|
||||
out_dir = Path(args.extract_attachments)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
written = []
|
||||
for name, contents in reader.attachments.items():
|
||||
data = contents[0] if isinstance(contents, list) else contents
|
||||
safe = os.path.basename(name) or "attachment.bin"
|
||||
target = out_dir / safe
|
||||
with open(target, "wb") as fh:
|
||||
fh.write(bytes(data))
|
||||
written.append(str(target))
|
||||
json.dump({"extracted": written}, sys.stdout, ensure_ascii=False, indent=2)
|
||||
print()
|
||||
return 0
|
||||
|
||||
if not args.output:
|
||||
print("Error: -o/--output is required for write modes", file=sys.stderr)
|
||||
return 4
|
||||
|
||||
writer = PdfWriter()
|
||||
writer.append(reader)
|
||||
|
||||
if args.set_meta:
|
||||
meta = {}
|
||||
for key, value in ((f"/{k.capitalize()}", getattr(args, k))
|
||||
for k in ("title", "author", "subject", "keywords")):
|
||||
if value is not None:
|
||||
meta[key] = value
|
||||
if not meta:
|
||||
print("Error: --set-meta needs at least one of --title/--author/--subject/--keywords",
|
||||
file=sys.stderr)
|
||||
return 4
|
||||
writer.add_metadata(meta)
|
||||
result = {"output": args.output, "set": {k.lstrip("/"): v for k, v in meta.items()}}
|
||||
elif args.clear_meta:
|
||||
writer.metadata = None
|
||||
result = {"output": args.output, "cleared": True}
|
||||
else: # --attach
|
||||
attach_path = Path(args.attach)
|
||||
with open(attach_path, "rb") as fh:
|
||||
writer.add_attachment(attach_path.name, fh.read())
|
||||
result = {"output": args.output, "attached": attach_path.name}
|
||||
|
||||
with open(args.output, "wb") as fh:
|
||||
writer.write(fh)
|
||||
print(json.dumps(result, ensure_ascii=False))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,99 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Export PDF pages as PNG images at a chosen DPI.
|
||||
|
||||
Rasterizer fallback chain: pypdfium2 (pip) -> pdftoppm (poppler-utils).
|
||||
When neither is available, exits 0 with {"rendered": false, "missing": [...]}
|
||||
so callers can branch instead of crashing.
|
||||
|
||||
Typical uses: visual verification with a vision model, and exporting
|
||||
image-only (scanned) pages for hand-off to the references/ocr-extraction.md in this skill.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def parse_pages(spec: str, page_count: int) -> list[int]:
|
||||
"""'1-3,5,9-' (1-based, inclusive) -> sorted page list."""
|
||||
pages: set[int] = set()
|
||||
for part in spec.split(","):
|
||||
part = part.strip()
|
||||
if not part:
|
||||
continue
|
||||
if "-" in part:
|
||||
start_s, _, end_s = part.partition("-")
|
||||
start = int(start_s) if start_s else 1
|
||||
end = int(end_s) if end_s else page_count
|
||||
pages.update(range(start, end + 1))
|
||||
else:
|
||||
pages.add(int(part))
|
||||
bad = [p for p in pages if not 1 <= p <= page_count]
|
||||
if bad:
|
||||
raise ValueError(f"pages out of range 1-{page_count}: {sorted(bad)}")
|
||||
return sorted(pages)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
parser = argparse.ArgumentParser(description="Export PDF pages as PNG images.")
|
||||
parser.add_argument("pdf", help="Input PDF path")
|
||||
parser.add_argument("--pages", default="1-", help="1-based ranges, e.g. '1-3,5' (default: all)")
|
||||
parser.add_argument("--dpi", type=int, default=150, help="Render DPI (default 150)")
|
||||
parser.add_argument("--out-dir", required=True, help="Directory for PNG files")
|
||||
parser.add_argument("--prefix", default="page", help="Output filename prefix (default 'page')")
|
||||
parser.add_argument("--password", help="Password for encrypted PDFs")
|
||||
args = parser.parse_args()
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import _raster
|
||||
|
||||
if not _raster.available_backends():
|
||||
json.dump({"rendered": False, "missing": _raster.missing_hints()}, sys.stdout)
|
||||
print()
|
||||
return 0
|
||||
|
||||
try:
|
||||
from pypdf import PdfReader
|
||||
except ImportError:
|
||||
print("Missing dependency: install with 'python3 -m pip install pypdf'", file=sys.stderr)
|
||||
return 2
|
||||
reader = PdfReader(args.pdf)
|
||||
if reader.is_encrypted:
|
||||
if args.password is None or not reader.decrypt(args.password):
|
||||
print("File is encrypted; pass --password.", file=sys.stderr)
|
||||
return 3
|
||||
page_count = len(reader.pages)
|
||||
|
||||
try:
|
||||
pages = parse_pages(args.pages, page_count)
|
||||
except ValueError as exc:
|
||||
print(f"Error: {exc}", file=sys.stderr)
|
||||
return 4
|
||||
|
||||
out_dir = Path(args.out_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
files = []
|
||||
for pageno in pages:
|
||||
img = _raster.rasterize_page(args.pdf, pageno, dpi=args.dpi, password=args.password)
|
||||
if img is None:
|
||||
json.dump({"rendered": False, "missing": _raster.missing_hints()}, sys.stdout)
|
||||
print()
|
||||
return 0
|
||||
out_path = out_dir / f"{args.prefix}{pageno:03d}.png"
|
||||
img.save(out_path)
|
||||
files.append(str(out_path))
|
||||
json.dump({"rendered": True, "dpi": args.dpi, "page_count": page_count,
|
||||
"files": files}, sys.stdout, ensure_ascii=False, indent=2)
|
||||
print()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,153 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Read a PDF: per-page text, tables, metadata, or form fields. JSON to stdout."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
|
||||
def _reconfigure_stdio() -> None:
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _need(module: str, package: str):
|
||||
try:
|
||||
return __import__(module)
|
||||
except ImportError:
|
||||
print(f"Missing dependency: install with 'python3 -m pip install {package}'", file=sys.stderr)
|
||||
raise SystemExit(2)
|
||||
|
||||
|
||||
def read_text(path: str, password: str | None) -> dict:
|
||||
pdfplumber = _need("pdfplumber", "pdfplumber")
|
||||
pages = []
|
||||
with pdfplumber.open(path, password=password) as pdf:
|
||||
for page in pdf.pages:
|
||||
pages.append(page.extract_text() or "")
|
||||
return {"page_count": len(pages), "pages": pages}
|
||||
|
||||
|
||||
def read_tables(path: str, password: str | None, csv_dir: str | None) -> dict:
|
||||
pdfplumber = _need("pdfplumber", "pdfplumber")
|
||||
result = []
|
||||
written = []
|
||||
with pdfplumber.open(path, password=password) as pdf:
|
||||
for pageno, page in enumerate(pdf.pages, start=1):
|
||||
for tidx, table in enumerate(page.extract_tables()):
|
||||
result.append({"page": pageno, "index": tidx, "rows": table})
|
||||
if csv_dir:
|
||||
os.makedirs(csv_dir, exist_ok=True)
|
||||
csv_path = os.path.join(csv_dir, f"page{pageno}_table{tidx}.csv")
|
||||
with open(csv_path, "w", encoding="utf-8", newline="") as fh:
|
||||
csv.writer(fh).writerows([[c if c is not None else "" for c in row] for row in table])
|
||||
written.append(csv_path)
|
||||
out = {"table_count": len(result), "tables": result}
|
||||
if csv_dir:
|
||||
out["csv_files"] = written
|
||||
return out
|
||||
|
||||
|
||||
def read_meta(path: str, password: str | None) -> dict:
|
||||
pypdf = _need("pypdf", "pypdf")
|
||||
reader = pypdf.PdfReader(path)
|
||||
encrypted = reader.is_encrypted
|
||||
if encrypted:
|
||||
if password is None or not reader.decrypt(password):
|
||||
return {"encrypted": True, "note": "Provide --password to read metadata of an encrypted file."}
|
||||
meta = {}
|
||||
if reader.metadata:
|
||||
for key, value in reader.metadata.items():
|
||||
meta[str(key).lstrip("/")] = str(value)
|
||||
pages = []
|
||||
for idx, page in enumerate(reader.pages, start=1):
|
||||
box = page.mediabox
|
||||
pages.append({
|
||||
"page": idx,
|
||||
"width": float(box.width),
|
||||
"height": float(box.height),
|
||||
"rotation": int(page.get("/Rotate", 0)),
|
||||
})
|
||||
# scanned-page heuristic: no extractable text but page has images
|
||||
likely_scanned = []
|
||||
try:
|
||||
pdfplumber = _need("pdfplumber", "pdfplumber")
|
||||
with pdfplumber.open(path, password=password) as pdf:
|
||||
for pageno, page in enumerate(pdf.pages, start=1):
|
||||
text = (page.extract_text() or "").strip()
|
||||
if not text and page.images:
|
||||
likely_scanned.append(pageno)
|
||||
except SystemExit:
|
||||
raise
|
||||
except Exception as exc: # pragma: no cover - heuristic only
|
||||
print(f"Warning: scanned-page check failed: {exc}", file=sys.stderr)
|
||||
out = {
|
||||
"encrypted": encrypted,
|
||||
"page_count": len(reader.pages),
|
||||
"metadata": meta,
|
||||
"pages": pages,
|
||||
"likely_scanned_pages": likely_scanned,
|
||||
}
|
||||
if likely_scanned:
|
||||
out["note"] = ("Image-only pages detected: no text layer to extract. "
|
||||
"Use the references/ocr-extraction.md in this skill for OCR.")
|
||||
return out
|
||||
|
||||
|
||||
FIELD_TYPES = {"/Tx": "text", "/Btn": "button", "/Ch": "choice", "/Sig": "signature"}
|
||||
|
||||
|
||||
def read_fields(path: str, password: str | None) -> dict:
|
||||
pypdf = _need("pypdf", "pypdf")
|
||||
reader = pypdf.PdfReader(path)
|
||||
if reader.is_encrypted:
|
||||
if password is None or not reader.decrypt(password):
|
||||
print("File is encrypted; pass --password.", file=sys.stderr)
|
||||
raise SystemExit(3)
|
||||
fields = reader.get_fields() or {}
|
||||
out = {}
|
||||
for name, field in fields.items():
|
||||
ftype = FIELD_TYPES.get(str(field.get("/FT")), str(field.get("/FT")))
|
||||
value = field.get("/V")
|
||||
states = field.get("/_States_")
|
||||
entry = {"type": ftype, "value": None if value is None else str(value)}
|
||||
if states:
|
||||
entry["options"] = [str(s) for s in states]
|
||||
out[name] = entry
|
||||
return {"field_count": len(out), "fields": out}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
_reconfigure_stdio()
|
||||
parser = argparse.ArgumentParser(description="Extract text, tables, metadata, or form fields from a PDF.")
|
||||
parser.add_argument("pdf", help="Input PDF path")
|
||||
mode = parser.add_mutually_exclusive_group(required=True)
|
||||
mode.add_argument("--text", action="store_true", help="Per-page text as JSON")
|
||||
mode.add_argument("--tables", action="store_true", help="Tables as JSON (optionally CSV via --csv-dir)")
|
||||
mode.add_argument("--meta", action="store_true", help="Metadata, page sizes, encrypted/scanned flags")
|
||||
mode.add_argument("--fields", action="store_true", help="AcroForm fields with types and values")
|
||||
parser.add_argument("--csv-dir", help="Also write each table as a CSV file into this directory")
|
||||
parser.add_argument("--password", help="Password for encrypted PDFs")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.text:
|
||||
result = read_text(args.pdf, args.password)
|
||||
elif args.tables:
|
||||
result = read_tables(args.pdf, args.password, args.csv_dir)
|
||||
elif args.meta:
|
||||
result = read_meta(args.pdf, args.password)
|
||||
else:
|
||||
result = read_fields(args.pdf, args.password)
|
||||
json.dump(result, sys.stdout, ensure_ascii=False, indent=2)
|
||||
print()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,71 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Encrypt or decrypt a PDF with passwords (AES-256 via pypdf).
|
||||
|
||||
Note: permission flags set at encryption time are advisory — viewers may honor
|
||||
them, but any PDF library can strip them. Only the user password gates content.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
|
||||
|
||||
def main() -> int:
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
parser = argparse.ArgumentParser(description="Encrypt/decrypt PDFs (pypdf, AES-256).")
|
||||
parser.add_argument("pdf", help="Input PDF path")
|
||||
parser.add_argument("-o", "--output", required=True, help="Output PDF path")
|
||||
mode = parser.add_mutually_exclusive_group(required=True)
|
||||
mode.add_argument("--encrypt", action="store_true", help="Encrypt the PDF")
|
||||
mode.add_argument("--decrypt", action="store_true", help="Remove encryption (password required)")
|
||||
parser.add_argument("--user-password", help="User (open) password for --encrypt")
|
||||
parser.add_argument("--owner-password", help="Owner password for --encrypt (defaults to user password)")
|
||||
parser.add_argument("--password", help="Known password for --decrypt")
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
from pypdf import PdfReader, PdfWriter
|
||||
except ImportError:
|
||||
print("Missing dependency: install with 'python3 -m pip install pypdf'", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
reader = PdfReader(args.pdf)
|
||||
if args.encrypt:
|
||||
if not args.user_password:
|
||||
print("Error: --encrypt requires --user-password", file=sys.stderr)
|
||||
return 2
|
||||
if reader.is_encrypted:
|
||||
print("Error: input already encrypted; decrypt first", file=sys.stderr)
|
||||
return 3
|
||||
writer = PdfWriter()
|
||||
writer.append(reader)
|
||||
writer.encrypt(
|
||||
user_password=args.user_password,
|
||||
owner_password=args.owner_password or args.user_password,
|
||||
algorithm="AES-256",
|
||||
)
|
||||
action = "encrypted"
|
||||
else:
|
||||
if not reader.is_encrypted:
|
||||
print("Error: input is not encrypted", file=sys.stderr)
|
||||
return 3
|
||||
if args.password is None or not reader.decrypt(args.password):
|
||||
print("Error: wrong or missing --password", file=sys.stderr)
|
||||
return 4
|
||||
writer = PdfWriter()
|
||||
writer.append(reader)
|
||||
action = "decrypted"
|
||||
|
||||
with open(args.output, "wb") as fh:
|
||||
writer.write(fh)
|
||||
print(json.dumps({"output": args.output, "action": action, "page_count": len(reader.pages)}))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,84 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract page ranges from a PDF, optionally rotating and/or compressing pages."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
|
||||
|
||||
def parse_pages(spec: str, page_count: int) -> list[int]:
|
||||
"""Parse a 1-based page spec like '1-3,5,9-' into 0-based indices."""
|
||||
indices: list[int] = []
|
||||
for part in spec.split(","):
|
||||
part = part.strip()
|
||||
if not part:
|
||||
continue
|
||||
if "-" in part:
|
||||
start_s, _, end_s = part.partition("-")
|
||||
start = int(start_s) if start_s else 1
|
||||
end = int(end_s) if end_s else page_count
|
||||
else:
|
||||
start = end = int(part)
|
||||
if start < 1 or end > page_count or start > end:
|
||||
raise ValueError(f"Page range {part!r} out of bounds (1-{page_count})")
|
||||
indices.extend(range(start - 1, end))
|
||||
return indices
|
||||
|
||||
|
||||
def main() -> int:
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Split/extract pages from a PDF (pypdf). Pages are 1-based: '1-3,5,9-'.")
|
||||
parser.add_argument("pdf", help="Input PDF path")
|
||||
parser.add_argument("--pages", required=True, help="1-based page spec, e.g. '1-3,5,9-'")
|
||||
parser.add_argument("-o", "--output", required=True, help="Output PDF path")
|
||||
parser.add_argument("--rotate", type=int, default=0,
|
||||
help="Rotate extracted pages clockwise (multiple of 90)")
|
||||
parser.add_argument("--compress", action="store_true",
|
||||
help="Deflate content streams (modest savings; does not recompress images)")
|
||||
parser.add_argument("--password", help="Password if the input is encrypted")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.rotate % 90 != 0:
|
||||
print("Error: --rotate must be a multiple of 90", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
try:
|
||||
from pypdf import PdfReader, PdfWriter
|
||||
except ImportError:
|
||||
print("Missing dependency: install with 'python3 -m pip install pypdf'", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
reader = PdfReader(args.pdf)
|
||||
if reader.is_encrypted:
|
||||
if args.password is None or not reader.decrypt(args.password):
|
||||
print("Error: input is encrypted; pass --password", file=sys.stderr)
|
||||
return 3
|
||||
try:
|
||||
indices = parse_pages(args.pages, len(reader.pages))
|
||||
except ValueError as exc:
|
||||
print(f"Error: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
writer = PdfWriter()
|
||||
for idx in indices:
|
||||
page = reader.pages[idx]
|
||||
if args.rotate:
|
||||
page.rotate(args.rotate)
|
||||
writer.add_page(page)
|
||||
if args.compress:
|
||||
for page in writer.pages:
|
||||
page.compress_content_streams()
|
||||
with open(args.output, "wb") as fh:
|
||||
writer.write(fh)
|
||||
print(json.dumps({"output": args.output, "page_count": len(indices), "rotated": args.rotate}))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,143 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Stamp text or an image at coordinates onto selected PDF pages.
|
||||
|
||||
Builds an in-memory single-page overlay with reportlab, then merges it
|
||||
onto each selected page with pypdf. Coordinates are PDF points, origin
|
||||
bottom-left. Covers 'sign here' arrows, diagonal DRAFT banners, and
|
||||
page-corner labels.
|
||||
|
||||
Examples:
|
||||
pdf_stamp.py in.pdf -o out.pdf --text "DRAFT" --x 200 --y 400 \
|
||||
--font-size 60 --rotation 45 --opacity 0.3 --color "#cc0000"
|
||||
pdf_stamp.py in.pdf -o out.pdf --image sig.png --x 400 --y 60 \
|
||||
--width 120 --pages 3
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import io
|
||||
import json
|
||||
import sys
|
||||
|
||||
|
||||
def parse_pages(spec: str, page_count: int) -> list[int]:
|
||||
pages: set[int] = set()
|
||||
for part in spec.split(","):
|
||||
part = part.strip()
|
||||
if not part:
|
||||
continue
|
||||
if "-" in part:
|
||||
start_s, _, end_s = part.partition("-")
|
||||
start = int(start_s) if start_s else 1
|
||||
end = int(end_s) if end_s else page_count
|
||||
pages.update(range(start, end + 1))
|
||||
else:
|
||||
pages.add(int(part))
|
||||
bad = [p for p in pages if not 1 <= p <= page_count]
|
||||
if bad:
|
||||
raise ValueError(f"pages out of range 1-{page_count}: {sorted(bad)}")
|
||||
return sorted(pages)
|
||||
|
||||
|
||||
def build_overlay(args, page_width: float, page_height: float) -> bytes:
|
||||
from reportlab.lib.colors import HexColor
|
||||
from reportlab.pdfgen import canvas
|
||||
buf = io.BytesIO()
|
||||
c = canvas.Canvas(buf, pagesize=(page_width, page_height))
|
||||
c.saveState()
|
||||
try:
|
||||
c.setFillAlpha(float(args.opacity))
|
||||
c.setStrokeAlpha(float(args.opacity))
|
||||
except Exception:
|
||||
pass # very old reportlab: no alpha support
|
||||
c.translate(float(args.x), float(args.y))
|
||||
if args.rotation:
|
||||
c.rotate(float(args.rotation))
|
||||
if args.text:
|
||||
c.setFont(args.font, float(args.font_size))
|
||||
c.setFillColor(HexColor(args.color))
|
||||
c.drawString(0, 0, args.text)
|
||||
else:
|
||||
kwargs = {}
|
||||
if args.width:
|
||||
kwargs["width"] = float(args.width)
|
||||
if args.height:
|
||||
kwargs["height"] = float(args.height)
|
||||
if "width" in kwargs and "height" not in kwargs:
|
||||
from PIL import Image as PILImage
|
||||
with PILImage.open(args.image) as im:
|
||||
kwargs["height"] = kwargs["width"] * im.height / im.width
|
||||
c.drawImage(args.image, 0, 0, mask="auto", **kwargs)
|
||||
c.restoreState()
|
||||
c.save()
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
parser = argparse.ArgumentParser(description="Stamp text or an image onto PDF pages.")
|
||||
parser.add_argument("pdf", help="Input PDF path")
|
||||
parser.add_argument("-o", "--output", required=True, help="Output PDF path")
|
||||
what = parser.add_mutually_exclusive_group(required=True)
|
||||
what.add_argument("--text", help="Text to stamp")
|
||||
what.add_argument("--image", help="Image file to stamp (PNG/JPEG)")
|
||||
parser.add_argument("--x", type=float, required=True, help="X in points (origin bottom-left)")
|
||||
parser.add_argument("--y", type=float, required=True, help="Y in points")
|
||||
parser.add_argument("--pages", default="1-", help="1-based ranges, e.g. '1-3,5' (default: all)")
|
||||
parser.add_argument("--font", default="Helvetica", help="Font name (default Helvetica)")
|
||||
parser.add_argument("--font-size", type=float, default=24, help="Font size in points")
|
||||
parser.add_argument("--color", default="#000000", help="Text color as #RRGGBB")
|
||||
parser.add_argument("--rotation", type=float, default=0, help="Degrees counterclockwise")
|
||||
parser.add_argument("--opacity", type=float, default=1.0, help="0.0-1.0 (default 1.0)")
|
||||
parser.add_argument("--width", type=float, help="Image width in points")
|
||||
parser.add_argument("--height", type=float, help="Image height in points")
|
||||
parser.add_argument("--under", action="store_true",
|
||||
help="Place the stamp under existing content instead of over it")
|
||||
parser.add_argument("--password", help="Password if the input is encrypted")
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
from pypdf import PdfReader, PdfWriter
|
||||
except ImportError:
|
||||
print("Missing dependency: install with 'python3 -m pip install pypdf reportlab'",
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
|
||||
reader = PdfReader(args.pdf)
|
||||
if reader.is_encrypted:
|
||||
if args.password is None or not reader.decrypt(args.password):
|
||||
print("Error: input is encrypted; pass --password", file=sys.stderr)
|
||||
return 3
|
||||
try:
|
||||
pages = set(parse_pages(args.pages, len(reader.pages)))
|
||||
except ValueError as exc:
|
||||
print(f"Error: {exc}", file=sys.stderr)
|
||||
return 4
|
||||
|
||||
writer = PdfWriter()
|
||||
overlay_cache: dict[tuple[float, float], object] = {}
|
||||
for idx, page in enumerate(reader.pages, start=1):
|
||||
if idx in pages:
|
||||
size = (float(page.mediabox.width), float(page.mediabox.height))
|
||||
if size not in overlay_cache:
|
||||
overlay_pdf = PdfReader(io.BytesIO(build_overlay(args, *size)))
|
||||
overlay_cache[size] = overlay_pdf.pages[0]
|
||||
stamp = overlay_cache[size]
|
||||
if args.under:
|
||||
page.merge_page(stamp, over=False)
|
||||
else:
|
||||
page.merge_page(stamp)
|
||||
writer.add_page(page)
|
||||
with open(args.output, "wb") as fh:
|
||||
writer.write(fh)
|
||||
print(json.dumps({"output": args.output, "stamped_pages": sorted(pages),
|
||||
"kind": "text" if args.text else "image"}, ensure_ascii=False))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Stamp/watermark every page of a PDF with page 1 of another PDF."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
|
||||
|
||||
def main() -> int:
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Overlay (stamp) or underlay (watermark) a one-page PDF onto every page.")
|
||||
parser.add_argument("pdf", help="Input PDF path")
|
||||
parser.add_argument("--stamp", required=True, help="One-page PDF to apply (page 1 is used)")
|
||||
parser.add_argument("-o", "--output", required=True, help="Output PDF path")
|
||||
parser.add_argument("--under", action="store_true",
|
||||
help="Place stamp under the page content (background watermark)")
|
||||
parser.add_argument("--password", help="Password if the input is encrypted")
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
from pypdf import PdfReader, PdfWriter
|
||||
except ImportError:
|
||||
print("Missing dependency: install with 'python3 -m pip install pypdf'", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
reader = PdfReader(args.pdf)
|
||||
if reader.is_encrypted:
|
||||
if args.password is None or not reader.decrypt(args.password):
|
||||
print("Error: input is encrypted; pass --password", file=sys.stderr)
|
||||
return 3
|
||||
stamp_page = PdfReader(args.stamp).pages[0]
|
||||
|
||||
writer = PdfWriter()
|
||||
writer.append(reader)
|
||||
for page in writer.pages:
|
||||
page.merge_page(stamp_page, over=not args.under)
|
||||
with open(args.output, "wb") as fh:
|
||||
writer.write(fh)
|
||||
print(json.dumps({"output": args.output, "page_count": len(writer.pages),
|
||||
"mode": "under" if args.under else "over"}))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,414 @@
|
||||
"""End-to-end tests for the pdf skill helper scripts. No network required."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPTS = Path(__file__).resolve().parent.parent / "scripts"
|
||||
|
||||
|
||||
def run(script: str, *args: str, expect: int = 0) -> subprocess.CompletedProcess:
|
||||
env = dict(os.environ, LC_ALL="C", LANG="C", PYTHONIOENCODING="utf-8")
|
||||
proc = subprocess.run(
|
||||
[sys.executable, str(SCRIPTS / script), *args],
|
||||
capture_output=True, text=True, encoding="utf-8", env=env,
|
||||
)
|
||||
assert proc.returncode == expect, f"{script} {args}: rc={proc.returncode}\n{proc.stderr}"
|
||||
return proc
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def workdir(tmp_path_factory) -> Path:
|
||||
return tmp_path_factory.mktemp("pdfwork")
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def sample_image(workdir: Path) -> Path:
|
||||
from PIL import Image
|
||||
img_path = workdir / "sample.png"
|
||||
img = Image.new("RGB", (120, 80), (30, 120, 200))
|
||||
img.save(img_path)
|
||||
return img_path
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def report_pdf(workdir: Path, sample_image: Path) -> Path:
|
||||
spec = {
|
||||
"title": "Quarterly Example Report",
|
||||
"author": "example-author",
|
||||
"elements": [
|
||||
{"type": "heading", "text": "Quarterly Example Report", "level": 1},
|
||||
{"type": "paragraph", "text": "This is the introduction paragraph with a marker UNIQUEMARK42."},
|
||||
{"type": "table", "rows": [["Region", "Units"], ["North", "1250"], ["South", "980"]], "header": True},
|
||||
{"type": "image", "path": str(sample_image), "width": 200},
|
||||
{"type": "pagebreak"},
|
||||
{"type": "heading", "text": "Appendix", "level": 2},
|
||||
{"type": "paragraph", "text": "Second page content."},
|
||||
],
|
||||
}
|
||||
spec_path = workdir / "spec.json"
|
||||
spec_path.write_text(json.dumps(spec), encoding="utf-8")
|
||||
out = workdir / "report.pdf"
|
||||
run("pdf_create.py", str(spec_path), "-o", str(out))
|
||||
assert out.exists() and out.stat().st_size > 500
|
||||
return out
|
||||
|
||||
|
||||
def test_create_and_meta(report_pdf: Path):
|
||||
meta = json.loads(run("pdf_read.py", str(report_pdf), "--meta").stdout)
|
||||
assert meta["page_count"] == 2
|
||||
assert meta["encrypted"] is False
|
||||
assert meta["likely_scanned_pages"] == []
|
||||
assert "Quarterly Example Report" in meta["metadata"].get("Title", "")
|
||||
|
||||
|
||||
def test_extract_text(report_pdf: Path):
|
||||
data = json.loads(run("pdf_read.py", str(report_pdf), "--text").stdout)
|
||||
assert data["page_count"] == 2
|
||||
assert "UNIQUEMARK42" in data["pages"][0]
|
||||
assert "Appendix" in data["pages"][1]
|
||||
assert "Page 1" in data["pages"][0] # page number footer
|
||||
|
||||
|
||||
def test_extract_tables(report_pdf: Path, workdir: Path):
|
||||
csv_dir = workdir / "csvs"
|
||||
data = json.loads(run("pdf_read.py", str(report_pdf), "--tables",
|
||||
"--csv-dir", str(csv_dir)).stdout)
|
||||
assert data["table_count"] >= 1
|
||||
rows = data["tables"][0]["rows"]
|
||||
assert rows[0] == ["Region", "Units"]
|
||||
assert ["North", "1250"] in rows
|
||||
csv_files = list(csv_dir.glob("*.csv"))
|
||||
assert csv_files and "Region" in csv_files[0].read_text(encoding="utf-8")
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def form_pdf(workdir: Path) -> Path:
|
||||
from reportlab.lib.pagesizes import A4
|
||||
from reportlab.pdfgen import canvas
|
||||
out = workdir / "form.pdf"
|
||||
c = canvas.Canvas(str(out), pagesize=A4)
|
||||
form = c.acroForm
|
||||
c.drawString(72, 760, "Example Form")
|
||||
form.textfield(name="surname", x=72, y=700, width=300, height=20, value="")
|
||||
form.checkbox(name="agree", x=72, y=660, buttonStyle="check")
|
||||
form.radio(name="color", value="red", x=72, y=620, selected=False)
|
||||
form.radio(name="color", value="blue", x=110, y=620, selected=True)
|
||||
form.choice(name="size", x=72, y=580, width=120, height=20,
|
||||
options=["small", "large"], value="small")
|
||||
c.save()
|
||||
return out
|
||||
|
||||
|
||||
def test_form_fill_unicode_roundtrip(form_pdf: Path, workdir: Path):
|
||||
surname = "Фамилия — ‘test’"
|
||||
values = {"surname": surname, "agree": True, "color": "/red", "size": "large"}
|
||||
fields_json = workdir / "values.json"
|
||||
fields_json.write_text(json.dumps(values, ensure_ascii=False), encoding="utf-8")
|
||||
filled = workdir / "filled.pdf"
|
||||
run("pdf_fill_form.py", str(form_pdf), "--fields-json", str(fields_json),
|
||||
"-o", str(filled))
|
||||
data = json.loads(run("pdf_read.py", str(filled), "--fields").stdout)
|
||||
fields = data["fields"]
|
||||
assert fields["surname"]["value"] == surname
|
||||
assert fields["agree"]["value"] in ("/Yes", "/On", "True", "/1")
|
||||
assert fields["color"]["value"] == "/red"
|
||||
assert fields["size"]["value"] == "large"
|
||||
|
||||
|
||||
def test_merge_split_rotate(report_pdf: Path, workdir: Path):
|
||||
merged = workdir / "merged.pdf"
|
||||
out = json.loads(run("pdf_merge.py", str(report_pdf), str(report_pdf),
|
||||
"-o", str(merged), "--bookmarks").stdout)
|
||||
assert out["page_count"] == 4
|
||||
|
||||
part = workdir / "part.pdf"
|
||||
out = json.loads(run("pdf_split.py", str(merged), "--pages", "2-3",
|
||||
"--rotate", "90", "-o", str(part)).stdout)
|
||||
assert out["page_count"] == 2
|
||||
meta = json.loads(run("pdf_read.py", str(part), "--meta").stdout)
|
||||
assert meta["page_count"] == 2
|
||||
assert all(p["rotation"] % 360 == 90 for p in meta["pages"])
|
||||
|
||||
|
||||
def test_watermark(report_pdf: Path, workdir: Path):
|
||||
# Build the stamp at mid-page so its text does not overlap existing
|
||||
# headings (overlapping glyphs confuse text extraction).
|
||||
from reportlab.lib.pagesizes import A4
|
||||
from reportlab.pdfgen import canvas
|
||||
stamp = workdir / "stamp.pdf"
|
||||
c = canvas.Canvas(str(stamp), pagesize=A4)
|
||||
c.setFont("Helvetica", 40)
|
||||
c.drawString(200, 400, "DRAFT")
|
||||
c.save()
|
||||
stamped = workdir / "stamped.pdf"
|
||||
run("pdf_watermark.py", str(report_pdf), "--stamp", str(stamp), "-o", str(stamped))
|
||||
data = json.loads(run("pdf_read.py", str(stamped), "--text").stdout)
|
||||
assert all("DRAFT" in page for page in data["pages"])
|
||||
|
||||
|
||||
def test_encrypt_decrypt_roundtrip(report_pdf: Path, workdir: Path):
|
||||
enc = workdir / "enc.pdf"
|
||||
run("pdf_secure.py", str(report_pdf), "--encrypt", "-o", str(enc),
|
||||
"--user-password", "your-password")
|
||||
meta = json.loads(run("pdf_read.py", str(enc), "--meta").stdout)
|
||||
assert meta["encrypted"] is True
|
||||
|
||||
dec = workdir / "dec.pdf"
|
||||
run("pdf_secure.py", str(enc), "--decrypt", "-o", str(dec),
|
||||
"--password", "your-password")
|
||||
data = json.loads(run("pdf_read.py", str(dec), "--text").stdout)
|
||||
assert "UNIQUEMARK42" in data["pages"][0]
|
||||
|
||||
|
||||
def test_compress(report_pdf: Path, workdir: Path):
|
||||
out = workdir / "compressed.pdf"
|
||||
run("pdf_split.py", str(report_pdf), "--pages", "1-2", "--compress", "-o", str(out))
|
||||
data = json.loads(run("pdf_read.py", str(out), "--text").stdout)
|
||||
assert "UNIQUEMARK42" in data["pages"][0]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- form creation
|
||||
|
||||
FORM_SPEC = {
|
||||
"title": "Example Intake Form",
|
||||
"page_size": "A4",
|
||||
"fields": [
|
||||
{"name": "surname", "type": "text", "page": 1, "label": "Surname",
|
||||
"label_box": [72, 700, 150, 714], "entry_box": [160, 696, 400, 716]},
|
||||
{"name": "agree", "type": "checkbox", "page": 1, "label": "I agree",
|
||||
"label_box": [72, 660, 150, 674], "entry_box": [160, 658, 176, 674]},
|
||||
{"name": "color", "type": "radio", "page": 1, "label": "Color",
|
||||
"label_box": [72, 620, 150, 634], "entry_box": [160, 616, 400, 636],
|
||||
"options": ["red", "blue"], "value": "blue"},
|
||||
{"name": "size", "type": "dropdown", "page": 1, "label": "Size",
|
||||
"label_box": [72, 580, 150, 594], "entry_box": [160, 576, 300, 596],
|
||||
"options": ["small", "large"], "value": "small"},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def built_form(workdir: Path) -> Path:
|
||||
spec_path = workdir / "formspec.json"
|
||||
spec_path.write_text(json.dumps(FORM_SPEC), encoding="utf-8")
|
||||
out = workdir / "built_form.pdf"
|
||||
result = json.loads(run("pdf_make_form.py", str(spec_path), "-o", str(out)).stdout)
|
||||
assert len(result["fields"]) == 4
|
||||
return out
|
||||
|
||||
|
||||
def test_make_form_lists_all_fields(built_form: Path):
|
||||
data = json.loads(run("pdf_read.py", str(built_form), "--fields").stdout)
|
||||
fields = data["fields"]
|
||||
assert set(fields) == {"surname", "agree", "color", "size"}
|
||||
assert fields["surname"]["type"] == "text"
|
||||
assert fields["agree"]["options"] == ["/Off", "/Yes"]
|
||||
assert fields["color"]["value"] == "/blue" # pre-selected radio
|
||||
assert set(fields["size"]["options"]) == {"small", "large"}
|
||||
# label text is drawn on the page, not just stored in the widget
|
||||
text = json.loads(run("pdf_read.py", str(built_form), "--text").stdout)
|
||||
assert "Surname" in text["pages"][0] and "Size" in text["pages"][0]
|
||||
|
||||
|
||||
def test_make_form_fill_roundtrip(built_form: Path, workdir: Path):
|
||||
values = {"surname": "Smith", "agree": True, "color": "/red", "size": "large"}
|
||||
vals = workdir / "builtvals.json"
|
||||
vals.write_text(json.dumps(values), encoding="utf-8")
|
||||
filled = workdir / "built_filled.pdf"
|
||||
run("pdf_fill_form.py", str(built_form), "--fields-json", str(vals), "-o", str(filled))
|
||||
fields = json.loads(run("pdf_read.py", str(filled), "--fields").stdout)["fields"]
|
||||
assert fields["surname"]["value"] == "Smith"
|
||||
assert fields["agree"]["value"] == "/Yes"
|
||||
assert fields["color"]["value"] == "/red"
|
||||
assert fields["size"]["value"] == "large"
|
||||
|
||||
|
||||
# ------------------------------------------------------------- layout validation
|
||||
|
||||
def test_form_layout_valid_spec(workdir: Path):
|
||||
spec_path = workdir / "layout_ok.json"
|
||||
spec_path.write_text(json.dumps(FORM_SPEC), encoding="utf-8")
|
||||
report = json.loads(run("pdf_form_layout.py", str(spec_path)).stdout)
|
||||
assert report["ok"] is True
|
||||
assert report["errors"] == 0
|
||||
assert all(f["ok"] for f in report["fields"])
|
||||
|
||||
|
||||
def test_form_layout_detects_problems(workdir: Path):
|
||||
bad = {
|
||||
"page_size": "A4",
|
||||
"fields": [
|
||||
# out of bounds (x1 beyond A4 width)
|
||||
{"name": "wide", "type": "text", "page": 1, "label": "Wide",
|
||||
"label_box": [10, 700, 60, 714], "entry_box": [70, 696, 900, 716]},
|
||||
# two overlapping entry boxes
|
||||
{"name": "one", "type": "text", "page": 1, "label": "One",
|
||||
"label_box": [10, 600, 60, 614], "entry_box": [70, 596, 300, 616]},
|
||||
{"name": "two", "type": "text", "page": 1, "label": "Two",
|
||||
"label_box": [10, 560, 60, 574], "entry_box": [200, 600, 400, 620]},
|
||||
# label far away from its entry
|
||||
{"name": "lost", "type": "text", "page": 1, "label": "Lost",
|
||||
"label_box": [10, 100, 60, 114], "entry_box": [400, 500, 500, 520]},
|
||||
# too small
|
||||
{"name": "tiny", "type": "text", "page": 1,
|
||||
"entry_box": [10, 50, 14, 54]},
|
||||
],
|
||||
}
|
||||
spec_path = workdir / "layout_bad.json"
|
||||
spec_path.write_text(json.dumps(bad), encoding="utf-8")
|
||||
proc = run("pdf_form_layout.py", str(spec_path), expect=1)
|
||||
report = json.loads(proc.stdout)
|
||||
assert report["ok"] is False
|
||||
by_name = {f["name"]: f for f in report["fields"]}
|
||||
assert any("bounds" in p for p in by_name["wide"]["problems"])
|
||||
assert any("overlaps field" in p for p in by_name["two"]["problems"])
|
||||
assert any("from its entry" in p for p in by_name["lost"]["problems"])
|
||||
assert any("minimum size" in p for p in by_name["tiny"]["problems"])
|
||||
assert by_name["one"]["ok"] # first of the overlapping pair reports clean
|
||||
|
||||
|
||||
# ------------------------------------------------- rasterization (overlay, pages)
|
||||
|
||||
def _raster_available() -> bool:
|
||||
import shutil
|
||||
try:
|
||||
import pypdfium2 # noqa: F401
|
||||
return True
|
||||
except ImportError:
|
||||
return shutil.which("pdftoppm") is not None
|
||||
|
||||
|
||||
def test_form_layout_overlay(built_form: Path, workdir: Path):
|
||||
spec_path = workdir / "formspec.json"
|
||||
out_png = workdir / "overlay.png"
|
||||
report = json.loads(run("pdf_form_layout.py", str(spec_path), "--pdf", str(built_form),
|
||||
"--render-overlay", str(out_png)).stdout)
|
||||
overlay = report["overlay"]
|
||||
if _raster_available():
|
||||
assert overlay["rendered"] is True
|
||||
assert out_png.exists() and out_png.stat().st_size > 1000
|
||||
from PIL import Image
|
||||
with Image.open(out_png) as img:
|
||||
assert img.width > 100 and img.height > 100
|
||||
else:
|
||||
assert overlay["rendered"] is False
|
||||
assert overlay["missing"] # install hints present
|
||||
|
||||
|
||||
def test_form_layout_overlay_blank_page(workdir: Path):
|
||||
# No --pdf: overlay is drawn on a blank page, PIL-only, always renders.
|
||||
spec_path = workdir / "formspec.json"
|
||||
out_png = workdir / "overlay_blank.png"
|
||||
report = json.loads(run("pdf_form_layout.py", str(spec_path),
|
||||
"--render-overlay", str(out_png)).stdout)
|
||||
assert report["overlay"]["rendered"] is True
|
||||
assert out_png.exists()
|
||||
|
||||
|
||||
def test_page_image_export(report_pdf: Path, workdir: Path):
|
||||
out_dir = workdir / "pageimgs"
|
||||
result = json.loads(run("pdf_page_image.py", str(report_pdf), "--pages", "1-2",
|
||||
"--dpi", "72", "--out-dir", str(out_dir)).stdout)
|
||||
if _raster_available():
|
||||
assert result["rendered"] is True
|
||||
assert len(result["files"]) == 2
|
||||
from PIL import Image
|
||||
with Image.open(result["files"][0]) as img:
|
||||
# A4 at 72 dpi is ~595x842 px
|
||||
assert 500 < img.width < 700
|
||||
else:
|
||||
assert result["rendered"] is False
|
||||
assert result["missing"]
|
||||
|
||||
|
||||
def test_page_image_bad_range(report_pdf: Path, workdir: Path):
|
||||
if not _raster_available():
|
||||
pytest.skip("no rasterizer available")
|
||||
run("pdf_page_image.py", str(report_pdf), "--pages", "9",
|
||||
"--out-dir", str(workdir / "nope"), expect=4)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------- stamping
|
||||
|
||||
def test_stamp_text(report_pdf: Path, workdir: Path):
|
||||
out = workdir / "stamp_text.pdf"
|
||||
run("pdf_stamp.py", str(report_pdf), "-o", str(out),
|
||||
"--text", "STAMPMARK77", "--x", "150", "--y", "500",
|
||||
"--font-size", "30", "--color", "#cc0000", "--pages", "1")
|
||||
data = json.loads(run("pdf_read.py", str(out), "--text").stdout)
|
||||
assert "STAMPMARK77" in data["pages"][0]
|
||||
assert "STAMPMARK77" not in data["pages"][1] # only page 1 stamped
|
||||
|
||||
|
||||
def test_stamp_text_rotated_opacity(report_pdf: Path, workdir: Path):
|
||||
out = workdir / "stamp_rot.pdf"
|
||||
run("pdf_stamp.py", str(report_pdf), "-o", str(out),
|
||||
"--text", "DRAFT", "--x", "150", "--y", "400", "--font-size", "60",
|
||||
"--rotation", "45", "--opacity", "0.3")
|
||||
# Rotated glyphs confuse pdfplumber's line grouping; verify via pypdf.
|
||||
from pypdf import PdfReader
|
||||
text = PdfReader(str(out)).pages[0].extract_text()
|
||||
assert "DRAFT" in text
|
||||
|
||||
|
||||
def test_stamp_image(report_pdf: Path, sample_image: Path, workdir: Path):
|
||||
out = workdir / "stamp_img.pdf"
|
||||
run("pdf_stamp.py", str(report_pdf), "-o", str(out),
|
||||
"--image", str(sample_image), "--x", "400", "--y", "60",
|
||||
"--width", "100", "--pages", "2")
|
||||
from pypdf import PdfReader
|
||||
before = PdfReader(str(report_pdf))
|
||||
after = PdfReader(str(out))
|
||||
|
||||
def image_xobjects(page):
|
||||
res = page.get("/Resources", {})
|
||||
xo = res.get("/XObject")
|
||||
if xo is None:
|
||||
return 0
|
||||
return sum(1 for k in xo if xo[k].get("/Subtype") == "/Image")
|
||||
|
||||
assert image_xobjects(after.pages[1]) > image_xobjects(before.pages[1])
|
||||
assert image_xobjects(after.pages[0]) == image_xobjects(before.pages[0])
|
||||
|
||||
|
||||
# --------------------------------------------------------- metadata + attachments
|
||||
|
||||
def test_meta_set_and_clear(report_pdf: Path, workdir: Path):
|
||||
out = workdir / "meta_set.pdf"
|
||||
run("pdf_meta.py", str(report_pdf), "--set-meta", "-o", str(out),
|
||||
"--title", "Retitled Example", "--author", "example-author",
|
||||
"--subject", "Testing", "--keywords", "alpha, beta")
|
||||
meta = json.loads(run("pdf_read.py", str(out), "--meta").stdout)["metadata"]
|
||||
assert meta["Title"] == "Retitled Example"
|
||||
assert meta["Author"] == "example-author"
|
||||
assert meta["Subject"] == "Testing"
|
||||
assert meta["Keywords"] == "alpha, beta"
|
||||
|
||||
cleared = workdir / "meta_clear.pdf"
|
||||
run("pdf_meta.py", str(out), "--clear-meta", "-o", str(cleared))
|
||||
meta = json.loads(run("pdf_read.py", str(cleared), "--meta").stdout)["metadata"]
|
||||
assert "Title" not in meta or meta.get("Title") in ("", None)
|
||||
|
||||
|
||||
def test_attachments_roundtrip(report_pdf: Path, workdir: Path):
|
||||
payload = workdir / "payload.txt"
|
||||
payload.write_text("attachment payload UNIQUEATTACH99\n", encoding="utf-8")
|
||||
with_att = workdir / "with_att.pdf"
|
||||
run("pdf_meta.py", str(report_pdf), "--attach", str(payload), "-o", str(with_att))
|
||||
|
||||
listing = json.loads(run("pdf_meta.py", str(with_att), "--list-attachments").stdout)
|
||||
assert listing["attachment_count"] == 1
|
||||
assert listing["attachments"] == ["payload.txt"]
|
||||
|
||||
ext_dir = workdir / "extracted"
|
||||
result = json.loads(run("pdf_meta.py", str(with_att),
|
||||
"--extract-attachments", str(ext_dir)).stdout)
|
||||
assert len(result["extracted"]) == 1
|
||||
extracted = Path(result["extracted"][0])
|
||||
assert "UNIQUEATTACH99" in extracted.read_text(encoding="utf-8")
|
||||
Reference in New Issue
Block a user