Gemma 4 E4B is a 9.6 GB download. And when it finishes you’re holding an open-source multimodal LLM that reads text and images on hardware you own. The short answer for developers: Gemma 4 runs locally through Ollama v0.20.0 or later with two Terminal commands, handles text and images natively with audio and video variants in the family. And pairs with Pydantic for structured output in Python. For document-heavy work, Qwen2.5-VL plus Qdrant and ColPali gives you fully local multimodal RAG where nothing leaves your machine. Below is what each stack is genuinely good for, the exact commands to get started.
And where I still send work to a cloud API instead.
What Local Multimodal Actually Replaces
Multimodal used to be a plumbing project.
You wired a transcription service to a chat API to a vision endpoint, paid three vendors.
And debugged three failure modes every time a client changed the input format. One model that accepts text and images collapses that into a single dependency you can reason about.
That’s the real shift here.
And daily.dev’s Best Local LLM Models to Run in 2026 roundup frames Gemma 4 exactly that way: “good for chat, images.
And tool use; some models support 128K context.” Native multimodal support for text and images, with audio and video variants, positions it as a local-first option rather than a lab demo.
That 128K context figure matters more than it looks.
Whole documents, long transcripts, and multi-page scans fit without the chunk-and-pray pipeline that destroys context in traditional retrieval builds.
For a small operation, one model with a long window and image input replaces half a microservices diagram.
| Stack | Inputs | How it runs locally | Best first project |
|—|—|—|—|
| Gemma 4 (E4B, 12B) | Text and images natively; audio and video variants in the family | Ollama v0.20.0 or later; about 9.6 GB on disk for E4B | Chat, tool use, structured extraction |
| Qwen2.5-VL + Qdrant + ColPali | Document pages processed as images | Fully local: embed, store in Qdrant, query | Multimodal RAG over your own documents |
Two Commands and a Version Check
The barrier to entry is now disk space and a version flag, not infrastructure. A hands-on dev.to guide notes that Gemma 4 requires Ollama v0.20.0 or later. And its blunt advice is worth repeating: “if you’re on an older version, update first.” The default E4B model download lands at approximately 9.6 GB on disk. So clear the space before you start.
On macOS, running the 12B variant is two commands in Terminal:
“`
ollama pull gemma4:12b
ollama run gemma4:12b
“`
That’s the entire setup for a local model that reads images.
Compare that to the procurement conversation you’d have for an enterprise vision API.
And you can see why indie teams are prototyping faster than shops ten times their size. The version requirement is the only trap, and it announces itself loudly when you hit it.
Structured Output Is the Real Story
The part of this stack that actually earns money is the one nobody puts in the headline. A daily.dev walkthrough on building multimodal workflows with a local LLM shows Gemma 4 E4B running locally via Ollama, combined with Pydantic for structured output, all orchestrated in Python.
Here’s why that combination matters.
A model that chats is a toy; a model that returns a typed object is a pipeline. When you point a multimodal model at a screenshot, a scanned form, or a product photo and get back a validated Pydantic object, you’ve built extraction software with no parser to maintain. The schema becomes the contract, and Python enforces it before anything downstream sees the data.
This is how I’d judge every local multimodal model from here on: not by how clever it sounds in a chat window. But by how reliably it fills a schema when you feed it an image. Schema adherence is the difference between a weekend experiment and something you’d hand a client.
Fully Local Multimodal RAG, No Cloud in the Loop
The second workflow worth your time is document intelligence. A daily.dev build log on 100% local multimodal RAG using Qwen2.5-VL lays out the architecture: Qwen2.5-VL plus Qdrant and ColPali, where document pages are extracted as images, embedded, stored in a Qdrant vector database. And queried to generate responses. The whole thing runs locally.
Notice what’s missing: OCR. The classic pipeline of rasterize, recognize text, chunk, embed. And pray is gone, because the page itself is the input. Tables, charts, stamps, and handwriting stop being extraction failures and become part of the queryable image. For anyone who has watched a traditional OCR pipeline mangle an invoice layout, that’s not an incremental improvement, it’s a different architecture.
For small businesses, the stakes are concrete. Contracts, medical records, client files, and financial documents get processed without a single byte leaving the building. You also stop paying per token on volumes you already own, and no vendor deprecates your pipeline mid-quarter. Start with the document pile you’d least like to upload to someone else’s cloud, since that’s where local wins before you’ve benchmarked anything.
Where Cloud Still Wins
I’m not telling you to cancel your API keys.
Frontier-scale models still hold the quality edge on hard reasoning and high-volume production traffic. And a managed endpoint absorbs the operational pain of throughput, scaling, and uptime that your laptop will not. If a quality gap costs a client more than the monthly bill, pay the bill.
Local takes the other side of that ledger: sensitive data, unpredictable volumes where per-token pricing punishes you. And prototype work where you’ll iterate a hundred times before anything ships. My rule is simple. Anything a client would wince at sending outbound runs local. And anything where frontier quality is the product runs in the cloud.
The interesting part of 2026 is that the local column keeps growing.
Start Here This Week
Do this in order. Update Ollama to v0.20.0 or later, confirm you have the roughly 9.6 GB free for E4B, then pull and run the model with the two commands above. Build one Pydantic workflow against a real screenshot from your own work before you touch anything fancier. Then stand up the Qwen2.5-VL, Qdrant, and ColPali stack against a folder of documents you already have.
That’s a working local multimodal capability assembled in an afternoon, from open components, on hardware you own.
If you want a build like this scoped to your operation instead of a weekend project, that’s the work I do. Tell me which document pile or screenshot workflow you want running on your own machines. And I’ll map the stack.
