Antonio G. Di Benedetto has begun a testing series on local AI agents. He ran Hermes Agent with a large Qwen model on a Mac Studio with 256GB of unified memory and found it handy for jobs he would not give to a cloud service. He also found it flaky, and says the "magical assistant" idea does not hold up in daily use. His account, published by The Verge on 11 October 2026, is a first-person diary rather than a security report.
Why he is testing local models
Di Benedetto says he has mostly steered clear of regular AI use because he did not want to hand personal data to cloud providers. He now wants to find out whether spending on hardware that runs models locally pays off. He calls himself a non-expert and frames the piece as part of an ongoing series.
His test bench includes an M5 Ultra Mac Studio plus other Mac and Windows systems. He writes that Apple is pitching its new Mac desktops as good local-AI machines, and that RTX Spark Windows PCs with up to 128GB of RAM, aimed at agentic AI, are "landing imminently." Both are vendor and market claims he relays, not outcomes of his own tests.
The setup
- Agent: Hermes Agent, which he describes as an open-source, self-hosted desktop app for macOS, Windows and Linux. By his account it is free as long as it runs local models.
- Hardware: a Mac Studio with 256GB of unified memory.
- Model: Qwen 3.8 Flash Next, which he describes as a 125-billion-parameter model of about 105GB.
- Phone access: a Telegram bot that lets him control Hermes remotely.
Next he plans to try smaller Qwen models on an M6 Mac Mini, an M5 MacBook Air and an Asus TUF Gaming A14 with an AMD Strix Halo processor. RTX Spark hardware is on the list for later.
What the agent did
Daily briefing. A cron job set for 7:30am reads his email and calendar, adds a weather report and sends the result over Telegram. At first it kept failing, because macOS was asleep when the job triggered. He says it now runs reliably, yet it also broke several times during testing.
Steam library cleanup. Hermes sorted more than 400 games by genre and left his own categories intact: favorites, co-op, party games and games to play with his wife. It read the Steam client already installed on the Mac. For that, he registered a Steam web API key and gave it to the agent. The sorting took a few minutes, and he revoked the key right afterward because he no longer needed it.
Sensitive data analysis. Hermes crunched financial records and produced a spreadsheet comparing laptop specs. He kept both jobs on his own machine because of the sensitive financial data and an embargoed laptop launch. He says that was what made the tool worth using.
Benchmark automation, unfinished. He wants a script that runs the laptop benchmarks he now performs manually: about a dozen tests, three runs each. Hermes has produced vibe-coded Python scripts, but he describes the project as a work in progress.
Cost context
He mentions "a $12,000 computer" while describing the daily briefing, without saying which device he means. He also recalls an Apple demo, at a briefing he attended last month, of four Mac Studios wired into a cluster. He puts that at "nearly $50,000 worth of compute."
His verdict
Di Benedetto concludes that local AI is worth using for tasks he would keep away from the cloud, chiefly for privacy reasons. On dependability he is blunt: even powerful local models do not behave like an assistant that can do anything, and his own briefing broke more than once. He treats Hermes as a tool rather than a companion, and says he will keep testing and reporting as he works through the other machines.
