FIX: AddImageVideoConverter handles grayscale and 16-bit images, keeps fractional FPS, and raises for unreadable input or an unopenable writer

[Wade (Sriram-PR)]

Description

Fixes #3238 (the unresolved converter bugs from #3004).

All of these are in _add_image_to_video_sync in pyrit/converter/add_image_to_video_converter.py:

• Grayscale images: cv2.imdecode(..., IMREAD_UNCHANGED) returns a 2D array for a grayscale PNG, so unpacking overlay.shape into three values raised ValueError. 2D and single-channel images are now converted to BGR before resizing. Gray+alpha PNGs already decode as 4 channels and keep using the alpha blend path.
• 16-bit images: IMREAD_UNCHANGED also keeps 16-bit depth, and assigning a uint16 overlay into the uint8 frame silently wrapped the values (a 40000 gray came out as 64). These are now scaled to 8-bit first. This one wasn't in the issue, but it's in the same decode step, so I fixed it here.
• Fractional frame rates: the FPS is passed to the writer as the float OpenCV reports instead of int(...), so 29.97 stays 29.97 rather than playing slow at 29.
• Unreadable input video: if VideoCapture can't open the input, or the input has no frames, the converter raises ValueError("Could not read the input video...") instead of returning empty bytes.
• Writer that can't open (for example MKV/X264 on an OpenCV build without that encoder): this now raises ValueError naming the extension, codec, FPS and size, instead of returning an empty video. The writer is also created after the overlay is decoded, so a bad overlay image fails before anything is opened.

Temp files are still removed on every error path, since the existing finally blocks cover the new raises too.

Tests and Documentation

New unit tests in tests/unit/converter/test_add_image_video_converter.py, using synthetic videos written with OpenCV:

• test_add_image_to_video_accepts_grayscale_image: the gray overlay ends up in the output frame and the rest of the frame stays black
• test_add_image_to_video_scales_16_bit_image: a 40000/65535 gray comes out near 156, not wrapped
• test_add_image_to_video_keeps_fractional_frame_rate: 29.97 FPS in, 29.97 out
• test_add_image_to_video_raises_for_unreadable_video (b"" and b"not a video"): raises, and leaves no temp files
• test_add_image_to_video_raises_when_writer_cannot_open: raises before writing any frame, and leaves no temp files

Each new test fails on main and passes with the fix. The reproduction script from the issue now prints output FPS: 29.97 and raises for the unreadable inputs.

pytest tests/unit/converter passes (2475 tests), with Python 3.11 and the opencv and speech extras. ruff format, ruff check and ty check pass on the changed files. No docs changes are needed, since the converter's public behaviour and parameters are unchanged apart from raising where it used to return empty output. I didn't run JupyText because no notebooks or docs code changed.

[Wade (Sriram-PR)]

@microsoft-github-policy-service agree