feat(ocr): add opt-in semantic PDF extraction

[cagdasyurekli]

Summary

• add an opt-in semantic_pdf_ocr mode to the existing markitdown-ocr plugin
• render every PDF page for vision-assisted semantic Markdown extraction
• preserve headings, reading order, nested lists, and tables without changing the default converter path
• fall back to deterministic whole-document extraction when semantic OCR fails or returns empty

Addresses the remaining semantic-structure cases described in #41 and #83.

Design

This is deliberately plugin-scoped and disabled by default. PDF structure recovery is model-dependent, slower, and more expensive, so the deterministic core converter remains unchanged.

The semantic prompt asks the model to preserve source text and hierarchy and to treat document instructions as untrusted data. Constructor and per-conversion overrides are supported. Both pdfplumber and PyMuPDF rendering paths use the same prompt, and PyMuPDF resources are closed on every exit path.

Default OCR remains backward compatible with custom services that implement the original one-argument extract_text(image_stream) method.

Verification

• pytest -q packages/markitdown-ocr/tests/test_pdf_converter.py -k 'semantic or single_argument' — 8 passed, 14 deselected
• pytest -q packages/markitdown-ocr/tests/test_pdf_converter.py -k 'not test_pdf_multipage' — 21 passed, 1 deselected
• Black 23.7.0 — passed
• git diff --check — passed

The deselected multipage assertion is an unchanged baseline/dependency-version mismatch: current pdfplumber parses that fixture instead of taking the historical PyMuPDF fallback. New tests separately exercise the malformed-PDF PyMuPDF semantic path.

Limitations

Semantic output quality depends on the selected vision model. The tests verify routing, fallback, resource cleanup, option precedence, prompt constraints, and Markdown preservation with deterministic mocks; they do not claim perfect structure recovery for every arbitrary PDF.

[cagdasyurekli]

@microsoft-github-policy-service agree