How I pitted local LLMs against each other, Gladiator style, and the final winner was me

The tournament arc, staple of every shonen manga and anime, is the arc where the stakes reach a peak for our protagonist early in the story, while letting the audience see the skills of the challengers and introducing new characters that will play instrumental roles in the story ahead. Our protagonist can lose (My Hero Academia, Hunter x Hunter), win (Dragon Ball, Dragon Ball Z), or the tournament might come to an abrupt end due to an attack from the villains (Naruto). But the thing that does remain is the excitement of the tournament, the expectation of upsets and the wonder of seeing new characters and their skill. Why am I telling you about the tournament arc? Because I recently ran a personal tournament, no, not with humans but with local LLMs.

You see, I had a problem. I am an avid Instagram screenshotter and reel saver, an article bookmarker. A connoisseur of culture, if I may.

My two Instagram accounts had upwards of 20 daily messages between them, which were all reels or posts that I found interesting. I had already vibe coded a personal app called Nook which would take on some of the preserving part, but it was coded to save links; there was no way to get a summary of the content I was saving. And by this time, I had become even more fascinated by vibe coding. I had already built a ton of apps, some useful, mostly not. This time I decided I will create a personal capture app, an app where I will share a screenshot and it will be read and the text extracted. I will have a neat summary to read as well as a ledger of all the things I found interesting in one place. The app is called “Sift”.

My agent of choice to build it was Anthropic’s Fable. We brainstormed and exchanged some ideas, ranging from backend APIs to phone-level OCRs to renting servers and even setting up a small personal server. The prices of most of these things made me realise I need to keep the scope small right now. And even then, I needed a proof of concept. I already had some experience with local LLMs, as I had experimented with Ollama. I already had Gemma4:e4b and Gemma4:e2b installed, which I had used a couple of times before but had been frustrated with the results as well as the speed. You see, the main constraint was having a machine with only 6 GB of VRAM. But it was enough to see if this was a feasible endeavour or not. So, Claude and I decided to explore.

We did a preliminary search. DeepSeek-OCR, Qwen3-VL:8b and MiniCPM-V4.5 became part of the first test, along with the original resident Gemma4:e4b and its smaller sibling Gemma4:e2b.

Our preliminary search had some limiters which I put on myself, and I should state them here first. First, whatever model we decided to use had to live comfortably inside a consumer 6 GB GPU while also not letting my laptop die from overheating (well, it’s the only machine I have got; even this is being written on it). Second, privacy: I didn’t want any of the personal images or any of my patient data to even accidentally go to an outside company. (Google is already enough of an overlord.)

In the limited initial tests we ran, DeepSeek was fast but invented fabricated URLs and altered words. Qwen 8B tied with MiniCPM on quality and lost only on speed. Both Gemmas answered “no image available” and then made up entire carousels instead of admitting blindness (here I learned that models need vision capability, it is not built in, and even when one has a vision tag the results are not guaranteed), and the only one that could complete it with acceptable speed and accuracy was MiniCPM-V4.5. It was still slow (taking over a minute for each image in some cases) but it worked, and it was a proof of concept that yes, the app is possible, and yes, it will run on my laptop. Albeit slowly.

I still didn’t know how big the local LLM universe was. We built our app, but one smart decision we took at that time was to keep the app model-non-specific, like a cheap harness into which any future model can plug and just work. And that one decision is what made this tournament possible and feasible, as well as entertaining to watch. Now the stage was set, the rules were in place and an initial champion was crowned: MiniCPM-V4.5.

The Morning Massacre

The next day, I had a bright smile. I had been through a brutal but satisfying operation theatre list, and had done a ton of Instagram scrolling as well, which had resulted in a lot of videos and screenshots to go through (I was under the assumption that I had a working app, you see). But the morning dump produced less than acceptable results: missing text, incomplete analysis, even simple images not being read. On diagnosing, it turned out that the pipeline to which the model attaches, the harness if I may, was broken. The stage had a broken floor.

We fixed the issues and the app got working again. But it left a shred of doubt in my mind: if the pipeline was broken, it might mean that the other models might work again, because the initial tests were not comprehensive. I was not exactly happy with a one-minute timeline on one image when there could be tens of images to go through. So, a decision was made: a rematch was needed, keeping the currently selected model as the champion and then comparing its results to the challengers’ results.

The next thing to do was some more research on the models that could exist for my use, and that led me to a vaster universe than the glimpse my preliminary search had given me the night before. With confidence that our harness was now not the limiting factor, after running the test bench with the original champion (MiniCPM) the results were encouraging; even its own output speed improved by about 30%. The quality improved, the results became more trustworthy. The tournament was decided, the challengers were named, and thus the event began.

Round 1: The Giant Who Just Kept Working

My favourite on paper was the newest Qwen model at 6.1 GB. Yes, I know it did not fit my VRAM, but I was hopeful and naive; the reality is that the maths doesn’t work that way. You see, Ollama split it 50/50 between my CPU and GPU. It started its first run and ran for 152 minutes, which I had to cancel mid-run. I had spent nearly 2.5 hours and learned nothing about the model, but I did learn that I needed to set up hard limits so that this does not happen again. So, a limit of 450 seconds per image was selected. Because I had learnt by now that sometimes it was not the model but the harness itself, I decided to give Claude a mandate to figure out who was the culprit. Because unlike the Roman Emperors I was not merciless; I had other goals, to find the best model for my own work. New rules were also set with some early hard failures, and some later models were fast but the results produced were either completely hallucinated or it would stamp an incomplete result as complete. The test got upgraded as we went through the tournament, and that I think was the real win.

Round 2: The Younger Sibling Comes to Take the Throne

MiniCPM-V4.6, the newer model in the same size class as the current champion, came with a vengeance. It was faster but had a flaw: it was too eager and too sneaky, it hid its wrong results, tried to pass failures as success. It lost and was deleted.

Round 3: The Thinking Mute

Qwen3-VL:4b at 3.3 GB, the smaller sibling of the giant, fit my VRAM. It produced two replies, both of them empty. A little probing found the words, nearly 500 tokens’ worth of them, all in the Ollama thinking channel. The run was tried again after trying to keep the thinking off, but the result was the same: it could not stop thinking. But it raised an interesting question about the giant. If it stopped thinking, could it have produced the results? Which led us to the next model.

Round 3b: The Thoughtless Butcher

Qwen3-VL:4b-instruct, the non-thinking version of the mute, emerged as the first real contender. It went through the test bench at an astonishing speed of about 22 seconds average against the champion’s 80-odd seconds, even after the harness fix, and it did so with no failures. It became one of the finalists. The idiom “Do, don’t think” applied here well.

Round 4: The Blabberer

DeepSeek-OCR:3b, the one built for exactly doing the OCR work, was the strangest: it blabbered our own prompt back at us. Deleted.

Round 5: The Featherweight

Qwen3-VL:2b-instruct, the first finalist’s sibling, was fast like a real featherweight boxer. It read through the bench at an astonishing 13 seconds average, with just one failure. Off it went to the finals.

Round 6: The Expert Who Just Made Things Up

Granite 3.2 Vision, the document specialist, was quite moody from the start. It straight out rejected every request because it was tokenising the same image and prompt to over the set token limit, which was raised especially for it, and even then it just made things up: left a workout log halfway, invented prices where there were none. Rejected and deleted with prejudice.

The Endgame

The early rounds were exploratory; they helped figure out the test as well as improve the harness and weed out the clear losers. The stage was now set, the harness was updated and, with the bracket complete, the finals had three models: the original champion MiniCPM-V4.5 and the two Qwen siblings (qwen3-vl:4b-instruct, qwen3-vl:2b-instruct). They were tested on a brand-new test bench of images and videos (with transcription from local Whisper for all), double the size of the original test bench, with the lessons in harness quality and prompting learnt over two days. All three were unleashed one by one on the test bench. MiniCPM showed promise; the harness improvements led to it being noticeably faster. But the champion confabulated Gurmukhi; it had resorted to hallucinations and was deemed unfit. The Featherweight was still the fastest but had ended up inventing whole scripts. The winner was middling on speed and perfect on honesty. It completed the test bench, had the best acceptable results and followed the prompt to a T. The cup had changed hands, and it was the Qwen 4b model.

The erstwhile champion and the second finalist were thanked for their participation and then promptly deleted. Qwen3-VL:4b-instruct became the model for my app.

The Champion Defends Its Crown

Four days after the crown initially passed, NVIDIA announced its Nemotron model, which intrigued me, and I realised there may be some models that I missed. Nemotron was ruled out because it was a text-only model. It had an OCR sibling, which I ruled out because it meant setting up a whole different CUDA pipeline. The sweep spat out two promising candidates: Qianfan-OCR 4b and Gemma3:4b. Now, with the improved harness and prompts, the three-way match was underway, on an even newer and larger test bench made of real screenshots and synthetic images created using Claude to check the limits of the models. Qianfan came close, really close: faster than the champion on the real images and more faithful than it on the synthetic images, but it was slightly worse on the analysis and missed more than Qwen did. Gemma was even faster but missed too much and hallucinated on the photos that were part of the test.

The crown was defended and the tournament has ended, as for now.

The Summit

A handwritten date from a junior resident’s ball pen, 8/8/26, confounded the champion and both our finalists. Still undefeated, waiting for a better or more powerful model to come and read it.

The Crown: Why the Final Winner Was Me

No model won in reality. I did. Every lesson learnt, every process created belongs to me, not the contestants. For my use for Sift, Qwen3-VL:4b-instruct retains the crown currently. The app currently works as intended. It needs some real work and improvement, but it is now part of my daily workflow and I am content with its current work.

The gladiators fought for a crown. I kept the arena and the prize.