Hermes and Qwen on a Framework 13 eGPU¶
I’ve started using Hermes on my laptop with a local model instead of Openrouter. My goal is to do everything offline using open source models, since they’re more affordable and private (being local-first), and I’m afraid of becoming dependent on software for my programming skills.
My laptop is a Framework 13 AMD on the lower end of specs. When I purchased it, having issues with being OEM’d was fresh on my mind, so I splurged and went for 60 gigabytes of RAM! I have a leftover GPU from ages back playing Cyberpunk 2077 on a NUC that would eventually fulfil the role of a home server of sorts, a GeForce RTX 3060. It sits within my Razer Core X eGPU enclosure. Thanks to my Thunderbolt 4 cable lying around, these things work together nicely, especially thanks to my overadundance of laptop RAM (compared to before at least), albeit slower than they might otherwise have been on a single system.
To refresh everyone: Hermes is a LLM harness, an “agent” that facilitates the LLM talking to itself in a loop with some instructions on how to prompt me for a response, and with some tools to run some code. I run Hermes using Docker, so I give it root permissions in the container, and outside the container, it runs with a custom user that’s a part of the src group. The src group has access to my btrfs filesystem for source code for this purpose. This is on a simple Debian setup.
The results are… okay. The model I ran at first is qwen3.6-35b-a3b, a 32 billion parameter model that has some programming in its training corpus. The models are run using LM Studio, a simple downloader and runner of the models, which also provides a RPC surface that’s like OpenAI’s chat completion API for Hermes to chat with.
The first major task I gave it was to add some missing features to a production application overnight for 9lives, which it did until it hit a problem with the context size. Reading source code and having a conversation with me provides the model with context, a short-lived memory for the life of the conversation I’m having with the model. This model does not support a very large context window.
I ran it overnight and it worked okay at first without my intervention, sweating the laptop and the eGPU, but by the morning, the agent had given up. The model had become unresponsive! My laptop had taken its execution like a champ and hadn’t hung or anything, but it was a shame. There goes my first attempt at fully offline programming with the model to save me some time.
The second big task I gave it it did with relative ease over the span of a few hours. I needed it to summarise chats I had had with a team member in my organisation based on logs we had collected previously. In one conversation, it created a simple program that downloaded our chats to CSV. In another, it performed the summarisation. It was actually fine this time. I couldn’t, however, verify easily to what extent everything had gone correctly due to the sheer amount of messages and the private nature of the chat my team member and I had had. I passed the information off to my CEO who presumably ran his own model on a beefier rig to verify everything.
While I was running the second task, I was downloading another model, Qwen3-Coder-Next. I asked it to do the first task for me. It added redundant types, didn’t achieve the goal I wanted and to reuse what we already had. It tried to add new types for everything, and Hermes ultimately ran out of the iterations the model was allowed to run with.
Maybe the problem I gave it was too complicated. The code is a distributed system where the secret handling takes place in a trusted setting, and messages to and from the system take place over serial. The way the programming works is by sharing a datastructure that gets returned as-is, with some fields fleshed out, reserialised. There is a lot of type reuse across several files. I noticed a couple of errors that, if we’d let slip through, would have broken the app. Simple errors that happened because the agent misunderstood the business context of what we’re doing together. Ultimately, I had to roll everything it wrote back and try again with a different approach.
I also asked Qwen to create my fourth app, to also produce a plan for an app to help me with my planning for weightlifting, with the requirement that the app is built with Clojure and Datomic. I also asked it to not use browser side scripting anywhere, but it seemingly forgot that instruction. I prefer server hosted webapps without scripting (most of our internal apps at Fluidity Labs are CGI scripts in the end), my phone is pretty old, and the only upgrade I’m considering at this moment is a Jolla Phone. SailfishOS, which runs on the Jolla phone, has a crap web browser, so I don’t trust it with anything other than very simple websites. The result is as follows. Would you trust Qwen with this programming?
—
# Fitness Tracking App - Implementation Plan
## Project Overview
A progressive web app (PWA) for tracking fitness activities and daily tasks, built with Clojure, server-side rendered, using Datomic as the database. Runs on iPhone via PWA installation.
—
## Tech Stack
Layer |
Technology |
|---|---|
Language |
Clojure 1.12+ (clojurescript for PWA features) |
Web Framework |
Luminus or Immutant + Utica (lightweight, server-side focused) |
Database |
Datomic Cloud (transactional, immutable, full-text search via datalog) |
Frontend |
HTML + CSS + minimal JS (PWA manifest + service worker) |
Build Tool |
deps.edn + shadow-cljs (for optional ClojureScript enhancements) |
Hosting |
AWS (Datomic), any static site host (Vercel/Netlify/S3) + Lambda for backend |
—
## Core Features
### 1. User Dashboard (Daily View) - Display today’s scheduled activities - Track completed vs pending tasks - Quick add buttons for common workouts - Visual progress indicators (completed percentage)
### 2. Activity Tracking - Record workout type: cardio, strength, mobility, etc. - Log duration, intensity (1-10 scale), and notes
—
All in all: will try again in the future. I like being able to have my overnight colleague do things for me without getting in my way. I need to remember to ask Qwen to not use cloud services and half the crap it suggested here, what’s the point? Lambda for a backend for my simple app? We use metal for everything nowadays. I’ll up the iteration count though.