Multimodal

Multimodal describes an AI model that takes in or produces more than one kind of data, such as text together with images, audio, or video. Older systems bolted separate components together: a speech recogniser fed text to a language model, which passed a caption to an image generator. Modern multimodal models project each modality into a shared representation space and process them in one network, so a single pass can reason across a chart, a paragraph, and a screenshot at once. That unification is what allows a model to answer a question about a photograph, read a slide deck, describe what happened in a clip, or hold a spoken conversation with natural timing. GPT-4o, Gemini, and Claude accept images alongside text, and models such as Whisper, Sora, and Veo handle speech and video specifically. Practical uses include reading documents and receipts, checking product images, and helping visually impaired users understand their surroundings. Vision capability is not visual accuracy, though. Models still misread dense tables, small text, and precise spatial relationships, and an image can carry a prompt injection, so anything safety-critical or financial needs a verification step.