Multimodal Prompt Design

IntermediatecreativeMinimum 32K context

Writes prompts that combine images, video, documents, and text effectively. Covers ordering images relative to instructions, resolution and tiling tradeoffs that affect token cost and detail, grounding questions to specific regions or pages, requesting structured output from visual input, and reducing visual hallucination on charts, tables, and scanned documents.

multimodalvisionprompt-engineeringdocument-understandingocrstructured-output

Use cases

  • Extracting structured data from scanned invoices or forms
  • Asking precise questions about charts and diagrams
  • Ordering images and instructions for best accuracy
  • Cutting token cost on high-resolution image input

Example prompt

I need to extract line items from scanned invoices of varying quality into JSON.

Design the multimodal prompt: how to order the image and instructions, what resolution to send
and the cost tradeoff, how to define the output schema, how the model should signal low
confidence on unreadable fields instead of guessing, and how to handle multi-page invoices.
Include the prompt and a validation step for the returned JSON.

Recommended models

claude-opus-5gemini-3-6-flashqwen3-8-max

Compatible tools

claude-codekirogemini-cliany

Modalities

Input: text, image, video
→
Output: text, code

Related Skills

prompt-engineeringpdf-data-extractionstructured-output-design

Author

OpenModels Community

@openmodelsrun