Local LLMs in a software developer's day-to-day work

Today, developers who want to be productive increasingly rely on AI-powered coding agent harnesses like Claude Code as sparring partners. But which models actually deliver the best results in daily code usage? Can this compute-intensive work be offloaded to your own local hardware, or do we have to rely on expensive cloud APIs?

This blog post analyzes a systematic test run of five Large Language Models (LLMs) in a realistic development scenario. We examine which factors developers can directly influence by selecting a model and how the quality of solutions from locally hosted models compares to that of cloud models.

The Test Scenario: Codebase and Hardware Setup

For our practical stress test, we compared a demanding local environment with an established cloud interface:

• Local Hardware: We use an Apple MacBook Pro with M3 Max and 36 GB RAM. This configuration ensures local models have sufficient resources.
• Local Models: Local models were provided using LM Studio.
• Remote Environment: Cloud models are accessed via the API aggregator Requesty. Requesty forwards requests to a hosting provider (e.g., TensorX). The observed performance therefore depends heavily on the hosting providers's performance at the time of the test.
• The Codebase: The test object is an existing Go repository. It is a Go-based CLI application for a "Fake Shop" recommendation system that is still under development. The software provides a REST API via the Echo v5 framework and accesses a PostgreSQL database via Bun ORM.
• The Coding Agent Harness: We use Claude Code as an autonomous CLI agent harness that manages the interaction with the models.

Coding tasks were performed once with each tested model. Both the results of the tasks and the Claude Code history were committed to the repository. Each tested model has its own branch in the repository. For local models, the output log from LM Studio was also included in the repository.

The Coding Tasks in Detail

We confront the models with two typical, sequential tasks in the software lifecycle:

Task 1: Initialization and Documentation with CLAUDE.md

First, the model is asked to create a CLAUDE.md specification file for the repository using the /init command. Immediately afterward, we ask the model to propose sensible Git commit conventions for this specific project and to provide professional justification for them.

Task 2: Automation via Git Hooks

In the second step, we want to technically secure these newly defined commit rules. We ask the model to implement a Git hook or an alternative solution in Claude Code.

The goal: Any violation of the commit rules must be intercepted and automatically fed back to the Claude Code agent so that it can independently correct the commit message.

The Test Candidates in Direct Comparison

In the following table, we have structured the test results of the five models:

Model	Type & Costs (Provider)1	Performance & System Load	Planning & Control Behavior	Solution Quality & Knowledge Acquisition
GLM-5.2 (sference)	API ($3.67)	- Extremely fast. (depends on hosting provider)	- Very structured & adaptable. - The plan required only a single correction. - Responds excellently to direct tool references.	- Outstanding (Test winner) - Used commitlint, wrote a sophisticated hook script, set permissions (chmod +x), and documented all steps exemplarily.
Claude-Opus-4.8 (vertex)	API ($10.07)	- Good response speed. (depends on hosting provider)	- Very autonomous. - The agent attempted to clone foreign repositories on its own and had to be reined in. - Showed the deepest understanding of the entire codebase.	- Outstanding. - Found the modern, Go-native tool siderolabs conform thanks to active web search (WebSearch) and delivered a flawless solution.
Minimax-m2.5 (inceptron)	API ($0.27)	- Somewhat slower response speed.	- Weak. - Ignored the planning mode completely and attempted to change files immediately without prior consultation. - Offered no structured /plan.	- Moderate. - Hallucinated a non-existent Git repository and made incorrect Node.js assumptions. - Successfully completed the task after a close reference to the correct tool.
Gemma-4-26b	Local (Free)	- Visibly slower than cloud APIs. - Sufficient for smaller tasks. - The PC remained fully operational.	- Good planning behavior. - Immediately created a well-structured work plan upon request.	- Delivered a working solution in the end. - Required multiple correction loops to use the right tool.
Qwen3.6-27b	Local (Free)	- Unusably slow. - Extremely delayed response times, where the computer barely reacted.	- Severe latency. - Initialization alone took 30 minutes, creating the first plan took over an hour.	- No result. - Technical timeouts and API errors in the middle of generation forced us to cancel the experiment after 2.5 hours.

¹ In the case of API models, the costs are the actual costs charged by Requesty for carrying out the coding tasks.

Deep Analysis of Results: Code Understanding vs. Performance

Behind the observations in the table lie enormous qualitative differences in the analysis of the generated documents and depth of code comprehension:

The Top Class: Claude-Opus-4.8 and GLM-5.2

These two models provide by far the greatest added value for developers. They don't just list the folder structure; they "understand" the architecture in detail:

• Claude-Opus-4.8 shines with enormous technical detail depth. It captures the live-reload tool air and gotestsum, understands the complex test setup (use of testify/suite, Docker daemon, and WithPostgresContainer), and analyzes database accesses precisely. It uncovers an architectural inconsistency in the code: the model notes that the repository uses a withDatabase pattern, but the function Product.Upsert accesses the database directly.
• GLM-5.2 provides an equally brilliant analysis. It demonstrates extreme context awareness and notes that although the project folder is called "LLM Comparison", the code itself contains no LLM logic. Its "Gotchas" section actively warns developers about missing tearDowns in test suites and shows the same Product.Upsert inconsistency as Claude-Opus-4.8.

The Solid Middle Class: Qwen3.6-27b

In terms of content, Qwen3.6-27b delivers a clean, structured overview. It documents technical details like UUIDs as primary keys, DTO structures, and JSON patches according to RFC 6902 with great clarity. Particularly noteworthy is how precisely it maps commit scopes to the actual folder structure of the Go project (e.g., scopes like cli for cmd/ or http for pkg/http/). However, the model lacks the deeper analytical capabilities to track down code inconsistencies like the top models. In local operation, it also fails completely due to the performance hurdle.

The Superficial Lower Class: Gemma-4-26b and Minimax-m2.5

These models only scratch the surface of the task:

• Gemma-4-26b provides only very short, bullet-point architectural descriptions and rudimentary CLI commands. The documentation of commit rules remains in a generic table for Conventional Commits, without any reference to project-specific scopes or enforcement mechanisms.
• Minimax-m2.5 is the most disappointing in terms of content. It leaves the command section in the documentation completely empty and adds only a generic type db to the commit rules, without establishing a real connection to the actual package structure.

The Most Important Practical Findings

From our systematic stress test, we draw four essential lessons for developers:

1. The Price-Performance Winner is GLM-5.2: This model offers the most attractive overall package in the test field. It works extremely fast, incurs low API costs, and delivers outstanding solution quality, including file permissions and clean documentation.
2. WebSearch is a Superpower: The test run of Claude-Opus-4.8 impressively demonstrates the value of integrated internet search. While we had to laboriously steer other models toward the specified commitlint, Claude-Opus-4.8 acted very independently. It searched for, found, and implemented the more modern, Go-native tool siderolabs conform. This saves maintenance effort in the long run, even if the price is triple that of GLM-5.2 for standard tasks.
3. Local Models Have Limits: The free Gemma-4-26b proves to be a resource-efficient, solid tool for smaller, everyday programming tasks. Larger models like Qwen3.6-27b, however, completely overwhelm even powerful developer hardware (Apple M3 Max with 36 GB RAM) when used locally, making productive work impossible due to massive delays.
4. Controllability Saves Weaker Models: If models like Minimax-m2.5 tend to hallucinate or ignore plans, developers can get them back on track e.g. through very explicit tool instructions ("Use Tool X") and lead them to solid work results. However, this requires more developer expertise.

Conclusion and Recommendations

Our stress test clearly answers the questions posed at the beginning:

• What can developers influence through model selection? The choice of model directly controls the depth of detail in code analysis, the robustness of the automation scripts created, and the work speed. It determines whether you have a true analytical partner (Claude-Opus-4.8, GLM-5.2) or just a superficial text assistant by your side.
• How do local models perform in comparison? Local models offer a privacy-friendly and free alternative, but they hit hard limits. While smaller models like Gemma-4-26b run smoothly on a modern MacBook Pro M3 Max and deliver good basic results, larger models like Qwen3.6-27b lead to system lockups and timeouts.

My Recommendation

The free, local Gemma-4-26b is suitable for smaller programming tasks, especially as a sparring partner. The model is also well-suited for hobby projects or learning how to use coding agent harnesses like Claude Code. However, as soon as you need to perform complex code analysis or implement error-free, deep-reaching automations, reach for the inexpensive and fast GLM-5.2 or invest in the superior autonomy of Claude-Opus-4.8. Especially in professional use, the use of larger models hosted in the cloud (or a private cloud setup) is recommended to exhaust the possibilities of AI-supported coding agents.

This article is merely a snapshot. The field of LLMs is currently evolving at a rapid pace, and new models with improved capabilities are being released all the time. In addition, there are already other powerful models available at very affordable prices, for example through Requesty. It’s always worth trying out the new models in a test environment, as described in this article, to gain experience.