“I use Cursor.” “I use Claude.” “We’ve introduced Copilot.”
You hear statements like these all the time, and they sound as if they refer to a single tool. Look closer, though, and that’s not quite true. When someone says “Cursor,” they have usually made a combination of five decisions, sometimes without realizing it.
Problems arise when one of those decisions changes: a new model appears, the data protection officer asks questions, or the bill gets too high. That’s often when it becomes clear that nobody really knew what the setup consisted of in the first place.
This article makes those five layers visible and shows how you can, in many cases, change them largely independently. When you use LLMs, you’re choosing more than a model or a product. You’re also deciding where you work, how context is processed, where the model runs, and who controls access.
Contents
- The five layers of an LLM stack
- Why alternatives at each layer matter
- How to switch models, harnesses, and providers
- Comparing LLM stacks: the stack-swap moment
- What this means for you
- Conclusion: understanding your LLM stack
The five layers of an LLM stack
An LLM setup can be understood as five layers. Depending on the product and configuration, some layers can be changed independently; others are coupled or predetermined. Here they are, ordered by their proximity to the user.
- 🖥️ Environment – where do I type? For example, a terminal, VS Code, IntelliJ, Cursor, or Claude Desktop.
- 🪢 Harness – what orchestrates prompts, tools, and context? For example, Claude Code, OpenCode, Cline, Aider, or Copilot Agent.
- 🧠 Model – who does the thinking? For example, Opus 5, Sonnet 5, GPT-5.5, Gemini 2.5 Pro, or DeepSeek V4.
- 🏭 Runtime – where does the model run? For example, Vertex AI, AWS Bedrock, Azure, or locally with Ollama.
- 🔑 Access – who gives me the key and pays the bill? For example, Requesty, Portkey, OpenRouter, or a Claude subscription.
For me, these five layers form the CHARM model, a mnemonic for thinking about your own stack:
- C for Console: 🖥️ environment
- H for Harness: 🪢 harness
- A for Access: 🔑 access
- R for Runtime: 🏭 runtime
- M for Model: 🧠 model
The order of the acronym is a mnemonic, not a hierarchy. Switching one layer doesn’t necessarily mean starting your entire setup from scratch. Often, you can reassess individual layers and change them to suit the task at hand.
🖥️ Layer 1: The environment – where you type
The environment doesn’t have to be a full-fledged development environment. A terminal for Claude Code or OpenCode is just as much an environment as a desktop app such as Claude Desktop or a traditional editor with plugins, such as VS Code or IntelliJ.
The environment shapes your keyboard workflow and how you write prompts. It also determines whether you have side panels or work entirely in a terminal. It’s the most visible layer and has the strongest influence on the user experience.
If you use a terminal-based harness such as Claude Code, you can work in IntelliJ today and continue in VS Code tomorrow: same agent, same model, same access. The resulting artifacts – whether code, documentation, or presentations – remain the same too.
An integrated AI IDE such as Cursor is different. Its environment and harness are coupled: changing the environment also means changing the harness. That brings us to the second layer.
🪢 Layer 2: The harness – context, tools, and orchestration
The harness is a largely invisible layer that has a major influence on how the LLM behaves and, consequently, on its results. It’s the framework around the model: it determines which files the model sees, how prompts are assembled, which tools are called, and when a task stops. This is where an important part of the difference between products lies. So when someone says “Claude Code works much better for me than the alternatives,” they may be talking about the harness rather than the model itself.
The same model can produce noticeably different results with a different harness, and the path to the result can differ too. The harness influences how prominently tests feature and how much the model checks and improves its own work. A significant part of that happens through the prompt sent to the model behind the scenes. With LM Studio, you can log this and see what actually reaches the LLM. The harness is therefore a powerful lever that often receives too little attention.
Some harness rules are portable, at least in terms of their content. Files such as .cursorrules, AGENTS.md, and CLAUDE.md can contain working agreements and project conventions; opencode.json, by contrast, configures OpenCode. When moving to another harness, you need to review and adapt the content and settings because formats, capabilities, and conventions differ. Working agreements can travel with you, but almost never without a change in behavior.
🧠 Layer 3: The model – reasoning, context window, and price
This is the LLM itself. The choice of model affects reasoning capability, speed, context window, price, and, noticeably, the quality of the output. This layer is often confused with the harness. “Claude is good at coding” sometimes refers to Claude Code as a harness and sometimes to the Opus model.
Cloud model prices can differ considerably. The model and provider matter, but so do input and output tokens, cache usage, and sometimes the selected region. Local inference usually has no per-token API fee, but hardware, electricity, and operation still cost money.
The price per token isn’t the price per task, though. A model that solves a task in fewer steps and with fewer follow-up questions can be cheaper despite a higher token price. Conversely, a less expensive model may be sufficient if the task is well scoped and the result needs little rework. Where possible, compare the completed task, including review and rework.
Benchmarks such as SWE-bench Verified can offer an initial point of reference. Their results depend on factors including the model version, harness, and test conditions. Figures from different publications are therefore not automatically comparable. The best way to find out how well a stack works for your tasks is to run your own reproducible tests.
🏭 Layer 4: Runtime and provider – where the model runs
Some models are available through different providers, with different characteristics. Claude models, for example, are available directly from Anthropic and through AWS Bedrock, Google Vertex AI, and Microsoft Foundry. The available model versions, regions, and infrastructure depend on the specific offering.
The question “Can we use this with customer data?” concerns the processing location: in the EU, in the US, or on your own hardware? But it also concerns the service providers involved, contracts, retention periods, and other data flows. A region in a model ID or a regional endpoint is a configuration clue; it does not replace an assessment of the actual processing and contractual commitments.
Ollama, vLLM, and LM Studio can run models locally on your computer. With genuinely local inference, the model prompt does not need to be sent to an external inference provider. That doesn’t automatically mean the entire setup is offline: web search, plugins, MCP servers, telemetry, model downloads, and other features can generate network traffic. Your configuration also determines whether a model is actually running locally.
This doesn’t automatically resolve all data protection questions. If you process personal data, you still need a legal basis, a deletion concept, and an understanding of what logs and caches store on disk. Check the data flows and settings of the entire stack instead of assuming that “local” means no network traffic or GDPR compliance.
The speed and quality of local models depend heavily on the model, hardware, and task. For demanding coding tasks, the gap compared with large cloud models can be noticeable; for smaller tasks and routine work, local models can be a good fit. Your own comparison will show whether they are sufficient for your use case.
🔑 Layer 5: Access – gateways, costs, and billing
Who gives you the key and pays the bill? This layer affects costs, legal considerations, and which models are available in the first place.
A gateway such as Requesty, Portkey, or OpenRouter sits between the harness and the provider. Depending on the service and configuration, it can handle authentication, routing, caching, logging, and billing. The difference between “I use the Anthropic API directly” and “I use a gateway” is therefore more than a technical one.
Gateways differ in latency, available models, and routing logic. From a legal and financial perspective, it matters which data the gateway and provider receive, which regions actually apply to the gateway and inference, and how requests are billed. What you can configure depends on the service, its configuration, and the contracts in place.
Why alternatives at each layer matter
There are alternatives at every layer. A targeted change can affect cost, data protection, performance, or model availability. Whether only one layer changes depends on the product and its interfaces.
Two dependencies are particularly important: integrated AI IDEs such as Cursor closely couple the environment and harness, so changing the environment can also change the harness. Runtime and access are often connected because a gateway may support only certain providers and regions. There are also limits between model and harness, for example around tool calling or supported API features. The model helps make these dependencies visible – it doesn’t promise that every combination will work.
Three reasons why this matters:
1. Cost
Token prices differ between models and providers. The same task can cost very different amounts depending on the model, amount of context, and number of requests. The harness influences how much context is transmitted and how many steps are needed; the model influences, among other things, the length of its responses. Review and rework time belong in the comparison too. Research approaches such as FrugalGPT and RouteLLM also investigate when routing between models can save costs; the benefit depends on the use case.
2. Data protection and compliance
“Can we use this with customer data?” is not a question for the model alone. It concerns the provider, runtime, access, and other tools in the stack. What matters are the actual data flows, the service providers involved, the agreements in place, and the requirements of the project.
3. Lock-in
If you don’t distinguish the individual layers, every change can look like a complete product switch. In reality, the effort varies: switching models can be simple if the harness and access support the new model. Changing providers can also involve contracts and data flows. A new harness noticeably changes the way you work and takes time to learn. The environment primarily shapes the workflow, while access influences cost control and available providers, among other things.
How to switch models, harnesses, and providers
Some switches can be tried with a few settings; others depend on permissions, interfaces, and policies. Let’s look at each layer. The configuration examples below illustrate the principle and need to be checked against current versions and project requirements before production use.
Switching layer 1: Environment and user interface
The most direct option is a terminal. VS Code and IntelliJ include terminals; other applications can provide one through extensions. If the same CLI harness is available and configured in the same way in both environments, you can start it in either one – for example, with opencode. Check the working directory, permissions, and available tools as well.
Switching layer 2: The harness
In your chosen layer 1 tool, start the harness you want to use, such as claude or opencode. The system behaves differently depending on the harness. You can carry over the substance of your rules and working agreements, but you need to check and, where necessary, adapt file formats, configuration, and actual behavior for the new harness.
Switching layer 3: The model
Models are configured in the harness and switched using its mechanisms. In an active Claude Code session, you can do this with a command:
/model
The available choices depend on the harness, access, and configuration. With third-party gateways, you may need to select a model through configuration or at startup rather than through the model picker. It can be worthwhile to start with a less expensive model and move to a more capable one only if the results are unsatisfactory. Whether and how context is retained when switching depends on the harness and the session.
Switching layers 4 and 5: Changing the gateway
Changing the gateway can change access and the runtime. Claude Code speaks the Anthropic API protocol; compatible gateways can receive and forward requests. The ANTHROPIC_BASE_URL environment variable initially determines the endpoint being addressed. The provider, model, and region actually used also depend on routing and model configuration.
Here’s an example using Requesty’s EU gateway. You need your own access that is approved for the intended use. First, use an appropriate secret manager to make the key available to your current shell as REQUESTY_API_KEY; don’t write it into a configuration file or repository.
The model ID is an example from the Requesty documentation. Check beforehand whether it is available through your access and approved for your use case. The EU gateway endpoint and the chosen inference region are separate settings. For data protection and compliance, you must also check the actual data flows and contractual commitments. Environment variables do not provide complete protection against access by other processes; use the safeguards appropriate for your system.
Portkey also provides a Claude Code guide, in which gateway access and provider routing are configured separately. The harness can remain the same when you switch gateways. Whether exactly the same model version and functionality are available depends on the new access.
Switching layer 4 to local: Access without a cloud gateway
You can also run a suitable model on your own computer. For local inference, an external API connection is no longer needed; the question of access remains part of the model nevertheless. There is usually no per-token API fee, but there are costs for hardware, electricity, and operation. You must separately ensure that the entire stack does not make unwanted network connections.
Here’s an example using Ollama and Claude Code: install and start both tools according to their instructions. The following command explicitly selects a local model. Ollama downloads it if necessary, which initially requires a network connection.
The model needs sufficient RAM and storage; the Ollama model page lists the download size. Allow additional memory for context and runtime overhead. The Ollama guide to Claude Code recommends a context window of at least 64k for larger repositories. Check whether your hardware supports that in your chosen configuration.
How well and how quickly the model works depends on your hardware, task, and context. Test it with representative, approved tasks and compare the result, execution time, and rework. If you switch regularly, you can manage different configurations as profiles; don’t copy credentials along with them without protection.
Comparing LLM stacks: The stack-swap moment
Changing a layer is not an end in itself. You switch models because costs are too high or quality is insufficient. You change the runtime because certain data must not be sent to a region or provider. You try another harness because the current agent selects context poorly or requires too many manual corrections. In each case, you want to know what the change means for your work.
Don’t switch the whole stack at once. Otherwise, you may see that something changed but not why. A small, controlled experiment is more informative. Pick a specific task from your daily work and define what a good result looks like. For a programming task, that might mean passing tests, meeting acceptance criteria, and limiting manual rework. Also record execution time, the number of interventions, and costs. Set the criteria before comparing; otherwise, you may end up favoring whichever outcome you happen to like better.
To understand the model’s influence, keep the environment, harness, access, and, where possible, runtime unchanged. Switch only the model. Keep the task, context, rules, and available tools the same. Then compare not just the finished code but also correction cycles, execution time, costs, and reliability. A lower token price can be offset by more rework or longer execution time. Conversely, a more expensive model can be cheaper if it completes the task faster and with fewer errors. Nils Geistmann describes a concrete example of such a model comparison in “Local LLMs in a software developer’s day-to-day work”.
Investigate the influence of the harness in a similar way: use the same model, the same access, and, where possible, the same runtime.
Compare how each harness handles context, selects files, uses tools, revises its work, and how much oversight you need to provide. A harness switch is not a pure model comparison because the working style changes too: one system asks more questions, while another performs more steps autonomously. That is part of the result and shouldn’t be hidden behind an attempt to make the interfaces look as similar as possible.
The same principle applies to runtime and access: change only the relevant layer where possible. If the same model version is available through different providers or access methods, you can compare cost, latency, availability, limits, and data flows. A local comparison is possible only with models that can be deployed locally; if the model is different, more than one layer has changed. Record the exact model version, region, and configuration. Providers may use different defaults, filters, or versions even when the model name is the same. For data protection questions, a practical test with sample data is not enough. You also need technical documentation, retention and deletion periods, details of the service providers involved, and contractual terms.
The result is not a universal ranking of the layers, but a decision for your use case: this model delivers sufficient quality on our tasks at lower cost. This runtime meets our data processing requirements. This harness saves enough manual work to justify the learning curve. Comparisons like these turn a stack swap into a reasoned change rather than just a product switch.
What this means for you
The five layers are the same for every role. The questions they raise are different.
In development, the first question is where a poor result originates. If the model lacks important context, reads the wrong files, or skips tests, the cause may lie in the harness rather than the model. If the task is clearly described and the context is right, it may be worth switching models. In Claude Code, you can try /model in the active session for supported models; depending on your access, a configuration change may be needed instead.
In many organizations, however, not every layer is freely selectable. The harness, access, and runtime are often fixed, and sometimes the model is too. In practice, that may leave mainly the environment. The CHARM model still makes these constraints visible: it helps you distinguish between a technical decision, a company policy, and a familiar interface. It also shows you where to ask questions when a policy blocks you.
For project managers, product owners, and decision-makers, other questions come to the fore. High costs may come from the model, access, or the volume of tokens transmitted by the harness. For customer data, what matters most is where processing takes place, which other service providers are involved, and what data they store. And when changing providers, it’s worth looking beyond the model: the harness and environment also create switching costs.
The work doesn’t have to involve code. A folder containing meeting notes, proposal texts, or Markdown notes can also be a working context for a harness. You don’t need an IDE for that. It doesn’t change the layer model; it simply shows that the first layer doesn’t have to be a development environment.
Conclusion: Understanding your LLM stack
“I use Claude Code” is a perfectly adequate answer in everyday conversation. But as soon as I want to change something in my setup, it becomes too vague. Then I need to look more closely: which model is running? Through what access? In which region? What does the harness handle, and which environment am I using?
For me, the CHARM model isn’t another product description. It’s a way to organize these questions. If costs are too high, I look at the model, token consumption, and access. When customer data is involved, runtime and access matter. If the results aren’t right, I first check whether the model receives the right context and whether the harness handles files, tools, and tests sensibly.
Not all layers are equally easy to switch. A model can often be changed with a line of configuration. A different access method brings questions about cost, contracts, and data processing. A new harness takes time to learn, and I need to transfer my rules to it. That takes effort, but it’s different from starting over completely. These differences become visible when I look at the five layers separately and experiment with changes deliberately.
That, for me, is the practical value of the model: I don’t have to claim that all combinations are equally good. I can find out which combination fits my task, my data, and the way I work. When one of those conditions changes, I can make a more deliberate decision about what I actually need to change.
Sources
- Nils Geistmann: Local LLMs in a software developer’s day-to-day work
- Ollama: Claude Code and Qwen3-Coder 30B
- Requesty: Claude Code and the EU endpoint and Portkey: Agent CLI
- SWE-bench Verified
- FrugalGPT: Stanford, arxiv.org/abs/2305.05176
- RouteLLM: Berkeley/LMSYS, lmsys.org/blog/2024-07-01-routellm
Note: Model availability, pricing, and technical interfaces change. Check the linked documentation and configuration examples again before using them.
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Blog author
Ralf Brauchler
People Lead · Product Owner · AI Enablement
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.