Writings

At work today a coworker said to me, “SaaS is dead.” And it’s not the first time I’ve heard that in the last week.

There were two major stories in the news around AI at the center of this refrain. First, there is a group called ArtCraft that created and released open source versions of most major SaaS productivity applications including the Adobe Suite (Photoshop, Illustrator, etc.), Microsoft Office (Word, Excel, etc.), AutoCAD, Figma, and others. They’re not fully baked yet, but they do cover the basic functions and their very existence may be poised to disrupt the software industry (Adobe’s stock took a hit from this news). You can find ArtCraft’s software here.

The second story was the release of a piece of software known as REA (“Reverse Engineer Anything”) and this and similar software takes applications (and games) and decompiles them to figure out how they work and allow you to edit or recreate them. Run REA on your computer, point it at anything, and AI can break it down for whatever you’d like to do next. You can find REA here.

These were announced right as I was doing a similar thing myself. A popular text editor called Notepad++ is a mainstay I use daily on my Windows PC, but the app doesn’t exist on the Mac. There are others, but they’re “not quite right” and missing a few small but essential features for how I use Notepad++ today. This is finding that “Goldilocks zone” where it fits just right for you. Until now I would just bemoan the absence of the features and deal with an inefficient workflow. But now with AI I just recreated Notepad++ on my own on the Mac. I didn’t steal any of its code or features or anything else. I just looked at what it did, created my own requirements, and built the app from scratch on the Mac to have feature parity. It works great - it sits exactly in my Goldilocks zone. And I built it in one day. I have no intention of selling it (Notepad++ is free as well), but it was a simple process of “I wish this existed” to “now it does.”

What does all this mean?

I don’t think it means SaaS is dead. Headlines tend to be doom and gloom, but the reality is always less sensationalized. In a similar scenario - I am writing this on a pc I bought. I know how to build one, but I didn’t want to deal with the hassle. There are plenty of people who will buy software rather than building it. But I do think paid software will take a hit over the next few years. Software-based intellectual property has a real problem right now. Not in the same way the music and movie industries felt in the 90s where people were pirating MP3’s and movies, but similar. This isn’t piracy - in this case nobody is stealing assets or code (to be fair, a few people may be, but I don’t think that is the general case). And basic software functions don’t have IP rights - every major photo editing software all have the same basic features; the same could be said for all major word editors. The problem isn’t theft, it’s competition. There isn’t a need to use (or pay for) someone else’s software when you can just build anything you need yourself. And you can build it to make it work exactly as you need.

The era of personalized software is here - with AI, you can build any piece of software you want for any platform and make it work any way you like. Completely personalized toward your needs, workflows, use case, and designed to make you happy. And sure, AI might not be creating the cleanest, bug-free, or scalable code. But I’m not sure it needs to. If I have a custom piece of software that I made, I’m not worried if it works on other platforms. Adobe needs to make sure its apps are performant on every platform. They need to make sure its foundation is built in a way they can continually build upon. They have to worry about different graphics drivers, operating systems, and a way to lock it down so people can’t get it without paying. I have none of those concerns. It only needs to work for me.

This newfound ability begs the question: when you can build anything, what do you build? Everything is at your disposal now. If you can think of it, you can go build it. It’s all personalized to your exact needs and your requirements. The concept is freeing, but this echoes the “Paradox of Choice” from author Barry Schwartz. Most people will have indecision and stress about what to do when every option is here. I’ve seen it from some friends in this field, and have experienced it myself - what next? Personally I’ve retreated into the “What do I need?” area. Instead of trying to build anything for anyone else, I just started building things that I wanted and couldn’t find anywhere.

My next project is in the same vein - my wife wants to organize her record collection and there’s a few really good apps in the App Store for it. But none of them are quite right; they each have some great features but missing something from one of the others she really wants. I told her, “Why compromise?” so now I’m just going to take her wish list of features and build it myself. There’s no reason to go buy someone else’s app that isn’t what you want anymore. We can make this app completely personalized for her - her own Goldilocks zone.

I don’t think SaaS is dead. And I don’t think paid software goes away. But if you can’t tell, I have no real answers for what any of it means yet. I don’t think anyone does. It’s entirely possible AI just made most paid software irrelevant, but it’s probably not that dire. AI is advancing rapidly and I do think recent leaps are something we as a society haven’t seen before. I’m not sure we’re ready for this yet; recent events show people are fearful for what the future brings.

And when money and fear are involved, there’s one thing you can be sure of: we’ll be seeing some legal action very soon. Much like the major AI companies a month ago trying to stamp down the open source models by pushing for regulation, fear will drive companies to act against this. Expect pushback. Expect lawsuits. Expect chaos for a while - similar to how it happened in the 90s when companies went after Napster and Limewire. When you take money from the corporations, there will be blowback. We’ll have that moment again here. So in a sense, nothing here is new. It’s just a new iteration of the same thing that came before. SaaS isn’t dead, it’ll just evolve.

We live in very interesting times.

The other day the "AI actress" Tilly Norwood was doing an interview with Piers Morgan, and it didn't go as planned. Take a look:

EXCLUSIVE: AI actress Tilly Norwood glitches and starts speaking CHINESE during her interview with Piers Morgan and real-life actor Tom Conti…

Watch the full interview at 7pm (BST) 👇

— Piers Morgan Uncensored (@PiersUncensored) · September 18, 2026

I've seen this problem multiple times before in text form - one minute your agent is replying to you in English, and then next you have CJK characters (Chinese / Japanese / Korean, or "CJK" as they're known) spilling into the replies. Oddly enough, I was diagnosing the most recent example of this happening to me when I saw this Tilly Norwood tweet.

Before we go on, let's pause and go through the most basic AI architecture:

You > Agent Harness > Inference Runner > Model

You interact with a harness (the UI / app that calls a model), that harness calls an inference runner (the software that parses the call, hosts the model, and returns a reply), and the runner hosts the model itself (the brains that "is" the AI).

When the CJK issue first started happening to me, I thought it was the model itself (I still believe this). I saw this when I would "push" a model to see what it could do. When I first started playing around with different AI models, I would see what they were capable of and where their limits were. What happens when you tell it that it's wrong repeatedly? What about when you question its knowledge? What if you ask it to do something that is beyond its ability? At some point when you start pushing, you'll see incoherence, hallucination, and CJK characters bleeding in. You'll also start to see reasoning bleed into responses ("reasoning" is the internal thought process models use to determine what to reply). I've also found other cases where the model was definitely the cause of this, but those had to do with quantization - once you compress a model too far, you'll get incoherence and CJK characters more frequently and there's really nothing you can do about that cause. And frankly, every model that's done this to me came from China, so I can't really complain that it reverts to Chinese if something in translation breaks down. It was trained in those characters initially, so it's baked in deeper than English.

A while ago I found the harness can influence this as well - you can have malformed API calls, contradicting directives, and the way the harness holds session cache can cause a previous turn to "poison" a session and cause every subsequent prompt/reply turn to degrade. Those can be a pain to triage and fix since the harness generally isn't your code. In some cases you can patch it, but then when the next version comes out you have to contend with your custom code vs third party updates. And if the harness is fully proprietary you're stuck with whatever defect it has.

Mind you - most inference runners have guardrails built in to throw an error when malformed calls happen. These help to avoid the "garbage in, garbage out" problem here. But the inference runner can create the "malformed call causes CJK issue" here in two ways: first, if the guardrail isn't set up correctly or has a defect. And second, some people don't want the model to throw an error if this happens. If a model throws an error, there are cases where that can blow up your session and you have to start over (restart the inference, reset the session, etc.) and you can lose work. Some harnesses can recover if it comes back with a little token soup, so you can just give it another prompt and move on. Or some users want to retain everything instead of erroring out so the results can be parsed for defect triage.

This last week we've been updating a custom version of vLLM for my personal use (vLLM is one of the top inference runners today and my personal favorite for its speed, stability, and flexibility). Much of this work is described in the GLM Project on my website. My agent and I built a customized version of vLLM to run a specific quant of GLM 5.3 Flash that had measurable speed improvements over the stock runner for my specific hardware. In the last week a series of PR's were introduced to upstream vLLM to bring even more improvements and stability. So naturally we brought those into our codebase and started trying them out.

We spent the last week fixing bugs as we found them (most were due to caching), but the most recent one introduced CJK characters into the replies. Now I haven't seen this in quite some time, so it piqued my interest. We dug in, and in this case it was the runner's fault - when it was trying to reuse cache, it wasn't storing the pointers correctly and was mixing up the starting points for how it brought that data in. It was like trying to get the model to read a book but the words and letters were coming in out of order. So when we started getting incoherence and CJK in the response, it makes sense why the model broke down. We had to go through that process a few times before each of the instances of cache issues were fixed.

The next day it happened again. And the interaction around this was pretty funny to me. I have two AI agents - one is a fully offline locally hosted agent. The hardware is in my house, the model and runner run on my computer, the harness runs on another computer in my house, and even the app I use to chat with it is hosted at home. All of it is completely offline. This is the GLM model / vLLM runner I was working on. My second agent runs on ChatGPT 5.6 (a combination of the Sol and Luna models) and is the one that helps me build the infrastructure for the offline one.

When I brought the first set of issues to the GPT model, it diagnosed the cache issues and after tracing every line of code multiple times we were both convinced everything was solid. So when this second instance of CJK happened, GPT came back to me and said, "the problem must be the model. The cache was not the problem, so your offline model is probably just not capable and is hallucinating."

Wow. I never thought I'd see AI be petty. Maybe they are more alive than we think?

I pushed back - this hadn't ever happened before and only started after we updated our runner. After a bit of back and forth I got it to dig in again. This time we found another cause - the problem was the Dflash drafter (Dflash is a method of "MTP", multi-token prediction, where a second smaller, faster model predicts tokens before the main model; it's a common way to increase speed). Turns out the Dflash draft mechanism had a similar bug to the cache where it was not storing information properly and choosing wrong tokens that were getting pulled into the response. We fixed it, and I think I finally have a stable runner now.

It's amazing how complex and fragile even the smallest AI ecosystem is. So far I've found this CJK issue in the model itself for two different reasons (behavioral boundary and quantization), at least three reasons in the runner (call guardrails, cache, and MTP), and at least two reasons in the harness (malformed calls and retaining poisoned sessions). It's not that intricate of a system, and yet it can break down in so many ways. That's not even beginning to tackle what is being used to generate Tilly Norwood - on top of what I've described for my system here, they also have a separate stack to do the video, likely another stack to do audio, another one to parse the incoming audio questions, and a much more complex scaffolding harness to lock in the personality and knowledge. With that many moving parts it's a miracle this doesn't happen more often.

The lesson here is the same as I often preach when it comes to AI - it's easy to assume AI is fundamentally different than how everything else worked before (and in some cases that's true), but in general you have to lean on your existing product development skills to make it work. Defects can happen at any layer of an application, and you should triage each to make sure you don't have anything lurking under the surface. Dig into each problem you find, trace them until you find a root cause, and fix it. Keep going through this process until your entire system is stable.

I read a lot of articles complaining about AI speed benchmarks. The common refrain is that these benchmarks exist and are so commonly accepted that the only way to get traction with your model is for it to do well on those tests. So creators train their models to do well on those tests, in a process now known as "benchmaxxing". In theory there's nothing wrong with this - if there's a test, let's make sure we can do well on it. But in reality there's a real issue with this practice: spending time training a model to pass a benchmark means you're drifting the model away from what it should be good at instead. So the complaint is a valid one - the benchmarks are actually making models worse instead of better. But humans need to understand everything as a comparison, so the benchmarks are a necessary evil. There isn't really a good answer there.

Despite all this, that isn't even my problem. I have different issues to talk about. When models are benchmarked for speed, they are generally using the measurement of "tokens per second" (tok/s). The main problem I have is that almost all benchmarks on AI speed are on Token Generation (TG). When you see headlines like: "This model gets 50 tok/s!" unless qualified, these will always refer to TG tok/s. And TG is a big deal - this is the speed that a model outputs content back to the user. So it's not only a useful metric, but an important one.

My problem isn't that this exists, or that it is highlighted. I have two issues with this, and much like everything else on AI, my problems are with the practical implications:

Issue #1: TG benchmarks are also benchmaxxed.

You can tune a model to output at a higher speed than what it would do in any real-world scenario. If you turn the temperature of a model down to .2 or 0, it gets far less creative in its thinking, so it will spend less effort on its answers. And because it's generating far fewer tokens, it spits them out at a much higher rate. You can further increase this by limiting context size, limiting max output tokens, and increasing speculative decoding to a higher rate (5-7 or even higher in rare cases) to eke out every last token per second.

The problem I have is this isn't indicative of real world performance. When you turn the sampling and context settings back to a real world use case (temp in the .7 to 1 range, context size to over 128k, etc.) the tok/s for TG drops significantly. Similarly you can get much higher speeds out of a more heavily quantized model - a Q4 quant of a model will be inherently faster than a Q6, even though it will be less coherent. If the model needs four extra turns to get a script correct at Q4, it's irrelevant how fast it is. And this is fine if you know these things well, but the problem lies in the fact that people new to AI now have a skewed perception of the kind of responsiveness and speed they should expect from the models they are using.

When I first started setting up local AI inference, I read all the benchmarks and articles and I expected to get ~35 tok/s from my model. Then I ran it with all the recommended settings and was getting 8 tok/s. If you're learning AI for the first time, this is a real shock to the system.

I do understand that it's very difficult to get a practical measurement of TG tok/s because use cases will greatly affect the outcomes here. Chatting with an agent will get much lower TG than coding. Solving a problem will get lower TG than creating something new. Every use case is different, so yes this is subjective. But still - you can give more realistic expectations if you have show measurements for different use cases. Keep the "this is the max speed" baseline as we have today. But add a coding speed and a chat speed that can give people a better expectation of how responsive the model will be for their tasks.

Issue #2: TG is the wrong measurement in most cases.

Again, TG is important. But agentic is now the most common use case for AI. And something that's not often talked about in this context is the fact that most common AI agent harnesses pass a lot of tokens into the model in every turn. It's not uncommon for a typical interaction with an AI agent to have over 100,000 tokens sent in and under 1,000 returned. So the model has to process significantly more input tokens than output. This is known as Prompt Processing (PP) and has a completely different throughput than TG.

There are two misleading factors with not measuring PP as part of the baseline "speed" of a model. First is that the settings I referenced that increase TG can negatively impact PP. Speculative decoding materially increases TG but reduces PP throughput. And second, lowering context size to 8192 for a TG benchmark is irrelevant when most agents barely function with a context size less than 128k. (Yes, you can have context significantly smaller for coding-only, but actual agentic work requires a much higher window.)

The reality of this is that "wall time" (the amount of time from beginning to end of a turn) is made up much more of PP than TG in most agentic use cases. If you send in a prompt to an agent and it takes 20 seconds to respond, many cases would have that be 15 seconds for PP and 5 for TG. Mind you - PP and TG have individual dials you can tune (as well as cache reuse which can heavily impact wall time). So while wall time is the perceived response time, there are a lot of factors that go into it, and some that have zero to do with the model itself. So PP is still the more useful benchmark for agentic use.

I will concede two issues with my push for PP as a more impactful speed measurement:

First, inference providers can have a big impact on PP. Two of the most popular ones - llama.cpp and vLLM - each deal with throughput differently. Llama.cpp is often regarded as the "faster" provider because you can get higher TG speeds out of it in many cases. But vLLM has a significantly higher PP in most cases, most especially if you have parallel calls coming in. So people new to local AI will see the benchmarks that llama.cpp is "faster" and then see their wall time being much slower than it should be.

Second, some newer models are made to do heavy reasoning to get to the right answer in fewer turns. There's a stigma in the community around these models, calling them "overthinkers". I won't comment on whether this is good or bad, but it is an intentional shift in how some of these models behave. The newly released Qwen 3.8 27b from Alibaba can easily crack over 10,000 reasoning tokens before giving an answer. And Poolside's Laguna S 2.1 routinely burns over 40,000 reasoning tokens per turn. In these cases, yes TG is an impactful measurement of their responsiveness.

So where does this leave us? The solution is pretty simple: as with almost everything else - more information is better than less. The more input we have into any decision, the better decisions we can make. In an ideal world you'd see something like this for speed benchmarks:

Provider 64k PP Chat TG Coding TG Llama.cpp 300t/s 30t/s 70t/s vLLM 1,200t/s 25t/s 60t/s You might still have the issue of benchmaxxing, but it's a start. Having a rotating list of prompts to use for these could solve that as you can then use averages across them. It might not be fully apples-to-apples if you had to keep these revolving so model creators couldn't train for specific prompts, but I'd rather have slight drift in these numbers than know the models are going to be worse for it.

It's not a perfect solution by any means, but at least then we'd have a clearer picture of factors and tradeoffs to decide which model and which inference provider are right for our individual use cases.

I saw many articles on this topic in the last few years and with the recent updates to most of the frontier models moving in the direction of expanded guardrails, this discussion is as relevant as ever. Also, since all of the models on Hugging Face that I've published have been abliterated, I want to explain why.

To level set: in case you don't know, abliterated means uncensored. This can be done in many different ways with different focus points on what it's actually removing, but in general this is what it means. And to be more specific, it generally refers to reducing or removing the model's refusal to answer a prompt. You surgically go into a model and change how it will respond.

I'll start by saying I agree with guardrails for online models. It's not a popular opinion but this is a tool that can be misused badly, and most people don't realize just how easily and the extent this technology can be used for harm instead of good. With our litigious society today, companies have to protect themselves from liability, which means refusing to give information that can be used for nefarious purposes. So I get it from both angles.

There's an odd side effect here though - Anthropic has been adding more extensive guardrails to their products. And many other providers use Anthropic's data to train their models. Which means a large range of AI companies are unintentionally baking in refusals to their models through a domino effect from Anthropic. The community has been reporting this with newer releases of many models today. It's not a great situation when one company's decision is directly affecting a wide swath of ability across the board.

So on the one hand, I do agree guardrails are necessary in this day and age. On the other hand, I only use and create abliterated models. There's good rationale for this (see the references below):

• Models can hide worthwhile information behind political alignment filters (to test this - ask any model from China about the events in Tiananmen Square) or give you information that is heavily leaning toward a political ideal.

• Cybersecurity work has extensive false positives that can appear as hacking. Even trying to evaluate your own website can raise red flags.

• Research can be flagged for keywords. I'm a big believer in mental health and learning as much as you can so you can better yourself and help those around you, especially trying to understand what people are going through. This is easily flagged for self-harm.

• Some models will flag work you give to an agent as attempting to hack or spam when you're trying to automate parts of your life.

• AI companies that are blocked by sites for scraping data can now refuse to allow you to work with those sites even if your intentions are valid.

• AI can be very frustrating to work with, and I've seen a close friend get flagged for "harassment" when an online model burned an entire week's worth of tokens not doing what it was asked.

You can find this list in almost every article that argues for uncensored models. None of this is new. But there's another reason that you won't find talked about often - performance. When a model has guardrails baked in, it has to check every prompt against them before it can reply - and that layer adds real work and time. Remove those gates and the model has less to do before it answers you. It's surprising, but it can be noticeable over time.

To put it another way - I just want a friction-free way to get my personal work done.

I understand there's tradeoffs with this. Abliteration has downsides - the more you mess with the way a model was intended to function, the more you can lobotomize it. I've run evaluation suites and watched an abliterated model fail repeatedly at tasks that the original version completed easily. Hallucinations also grow with abliteration. There's a measurement for this called KL Divergence that shows how far a model has drifted from the original. I aim to keep it under .05 whenever I can. It's not a percentage - it's a reference number I use as a sanity check. But even a reading that low represents a real change in behavior, and the drift gets steep fast past that point. Much like quantization, larger models tend to survive this process much better than smaller ones - they will stay closer to their original state much more often than a 20B model.

Abliteration issues can be mitigated - you can set how narrow of refusal mechanisms you target. I recently switched my daily driver model from a fully abliterated one to one only partially affected and it's immediately noticeable how much more capable and coherent it is. And that's another aspect of this - you shouldn't assume that every model has hard refusals baked in and need extensive abliteration to function how you need. Oftentimes the most narrow updates make a model work exactly how you'd need.

Additionally there are new methods coming out that can change the behavior of a model to remove refusals without the "surgery" that happens when you abliterate a model. This is based on a concept called Jacobian Lens that was pioneered by Anthropic (their research paper is linked at the bottom). This method can shift the personality of a model to respond differently than its training without fear of lobotomy. I imagine as this space continues to mature we'll see more creativity. As with every other area of life - the more regulations someone puts on something, the more people will find a way to get around them.

So those are my thoughts on abliteration. I agree it's probably not great if everyone has ready-made access to how to build bombs or make crack (both of which I've seen online as tests). That's not what you want. But at the same time, I also don't need to be told I can't check my own app for security failures.

As long as I use the tools I have for good, I'll stick with my methods for now.

References

• What is an Uncensored Model and Why Do I Need It - Security Boulevard

• Uncensored Models

• Why Do Uncensored Models Outperform Aligned Ones?

• Running Uncensored AI Models in 2026: Local LLMs, Privacy, and Responsible Freedom

• Making the Case for Uncensored Artificial Intelligence

• A global workspace in language models

• LLM Guardrails: The Complete Guide to AI Safety Guardrails (2026)