Dev Blog

Open Models for Real Work: Costs and Trade-offs

Written by Funnel Dev | Nov 20, 2026, 10:14:38 AM

By: Karim Pichara, Developer / AI Engineer

It is no secret that token costs are high, especially when it comes to proprietary frontier models.

Since GLM 5 was announced I started exploring open weight alternatives for my personal projects due to their appeal in terms of cost and capability. At the time I took a look at the pricing of “coding plans” for GLM, Kimi, and Qwen, and decided to stay with GLM, mainly after reading comments on Reddit, praising the model. I just wanted “good enough”, not “the best”.

Nowadays, I use GLM 5.3 as my daily driver in my personal projects with a couple of hiccups here and there, but overall I'm satisfied with the model (anecdotal side comment: for me, the biggest jump in quality was from 5.1 to 5.2). With that in mind, I always wanted to try GLM at Funnel, but due to the compliance requirements such as Zero Data Retention, SOC2, among others, I couldn't do it… until August this year!

During the summer we started trialling open weight models on one of the EU sovereign AI providers. After some initial testing (nothing fancy, just using it as the daily coding assistant) some questions arose:

  • Is this good enough for my everyday coding?
  • Is it fast enough?
  • Is it cheaper?

Then in October 2026, our primary cloud provider released GLM 5.3 as well. This was my chance for a larger experiment.

The experiment was simple, really. These were the steps I followed:

  1. Grab around 10 real issues from one of our repos
  2. Try to solve each issue 3 times in a clean, isolated copy of the code.
    1. Each time use a different model: Opus 5.5, GLM 5.3 (both on our primary cloud provider - let’s call it “Provider A”) and GLM 5.3 on the EU sovereign platform - let’s call it “Provider B” - run in parallel, of course.
    2. Same effort level, at least for the two versions of GLM
    3. Same prompt
  3. Run hidden deterministic tests the runs never saw, created beforehand for each task
  4. Each completed task was examined by three judges, one of them being GPT-6.1 Sol and ranked on a scale from 1 to 5 on things like code quality, requirement fit, tests, etc.

Small note about this: this experiment uses a small sample of issues; however I believe it is directionally good enough and gives a hint of the full picture.

Results

This table shows the results across all 10 tasks:

# One important aspect of the GLM-5.3 on the Provider B is that it uses NVFP4 compiled weights - this is NVIDIA’s 4-bit floating point E2M1 data format. We unfortunately have no information on the data format of the same model on Provider A, but suspect it to be a full 16-bit floating point checkpoint.

* We are a bit uncertain about why this one is so much slower. Our guess is because of the amount of output tokens, probably related to thinking, even though we put max effort on both versions of GLM.

From the table, you can draw your own conclusions, but these are my takeaways:

  • GLM 5.3 is an amazing model, extremely capable
  • In Provider B’s case, it’s very cheap at around a third of a cost of Opus 5.5 (note: in all cases we make significant use of input cached tokens)
  • Good enough if you know what you are doing (**)
  • Probably not ideal for one-shotting a big task, but boy it tries hard!

(**) Applying standard good agentic coding patterns such as clean supporting Markdown files, planning before implementing, etc.

Are you willing to wait twice as long for a third of the price? For my personal projects, I already made that choice. At work, it depends on the task, and that's a better answer than "we can't use it".