Everything You Wanted To Know About Building A Cheap AI Setup (That Still Works) But Were Afraid To Ask

Authored by Christopher Penn via 'Almost Timely News' substack,

I’ve been putting together a new course for Trust Insights, AI for Writing (and Writers), which will be coming out August 25th, and during the process of creating that course, I realized I’ve never walked through setting up a discount agentic AI system.

For a while now, many of us who work in “corporate” have had access to what feels like unlimited amounts of AI (though that free ride is coming to an end at enterprise shops thanks to absurdly large bills). For the average person who doesn’t have the backing of a corporation, how could we tap into the power of agentic systems without tapping out our wallets?

This tracks back to a question from Leslie in the Analytics for Marketers Slack group:

“have you done a video or post with recommendations for a cheap second computer set up someone can use to build their separate AI machine?“

So this week, let’s get scrappy.

Warnings, Disclaimers, Disclosures

If you work at a company, please check with IT before installing software willy-nilly. As a former IT professional, we thank you.

Second, in this issue I’ll be talking about Chinese-made AI models like Qwen and DeepSeek. There is a HUGE difference between a Chinese model hosted by a company in the People’s Republic of China (PRC), and a Chinese-made model hosted by a company outside the PRC.

It is manifestly unsafe to use any AI models hosted in the PRC for any kind of confidential information, period. Totally fine to use those models for things that are completely public, like research requests.

However, it is completely safe to use Chinese-made AI models on your own hardware, or in an infrastructure that’s trustworthy. There are many, many great inference providers that have things like Zero Data Retention (ZDR) hosting which are even safer to use than Big Tech models because the providers warrant that they don’t keep your data at all. (Most Big Tech providers have a “we hold onto your data for 30 days to prevent abuse” clause in their Terms of Service)

There is a lot of fear mongering about open weights models (including the Chinese ones) happening right now because Big Tech providers’ profits are on the line if these models continue to offer parity of capabilities at a fraction of the cost. Don’t believe the hype; open weights models are so safe that companies like HuggingFace have used them to do their own cybersecurity defenses.

Third, where I have commercial relationships, I’ll disclose them. I have an Amazon affiliate link in this week’s article, and obviously I work for my company, Trust Insights. I do not have a financial interest in the other companies mentioned as of the time of writing. (but I’m happy to, call me)

Part 1: The Model

We’ll begin our tour with what model to use. In general, you want models that are a blend of cost efficient with effective performance. One of my favorite sites for this is Artificial Analysis; there’s a nice little grid on their models page that shows cost versus intelligence. What we’re looking for in this grid is models that are at the top left corner - the vertical (y) axis is brains, where higher is better, and the horizontal (x) axis is cost, where to the left is lower.

The current winner here is DeepSeek V4 Flash 0731, which has a crazy low cost and lots of brains, on par with OpenAI’s newest model, GPT-5.6 Luna. On the far right is Claude Opus 5, which has about 15% more brains but 40x the cost. The worst quadrant to be in is the lower right - high cost, low brains.

For the purposes of building the cheapest agentic system, DeepSeek is going to be your daily driver model.

Now, it’s worth pointing out, as I have in my book Almost Timeless: 48 Foundation Principles of Generative AI that a daily driver may not be the smartest model, but in terms of day to day work, a model like DeepSeek v4 Flash is going to get the job done most of the time. Here’s a bit of retrospective, by the way, putting DeepSeek on the same scale as all of OpenAI’s major models over the years to see where it stacks up in terms of brains.

DeepSeek V4 Flash outcompetes all but the most recent releases of OpenAI’s models over the years; it is almost 50% smarter than last summer’s flagship model, GPT-5, and is only 17% less capable than OpenAI’s current flagship model.

Here’s the same chart from the Claude world:

DeepSeek V4 Flash is about on parity with Claude Opus 4.7 - which came out in April 2026, a scant FOUR MONTHS AGO.

Now, times change, tech changes, and what you should be doing is checking in on these kinds of charts over time to see which model is closest to that top left corner. This month, it’s DeepSeek V4 Flash. Next month, it might be completely different. It’s our responsibility to keep up with the charts and use the cheapest, best brains we can get a hold of.

Now that we know what model we’re aiming to use, we have to figure out where to use it.

Part 2: The Host

Just like web hosting companies host websites, model hosting companies - called inference providers - host models. There are so, so many of these out there, and they compete mainly on two things, speed and price. Some providers charge a premium but generate results (tokens per second) at ridiculous speeds, like Cerebras. Others offer a wide variety of models at slower speeds but are cheap (like DeepInfra).

Choosing a provider is straightforward: because you’ve decided the model already, it’s a question of pricing and availability. In Google AI Mode, modify this prompt to suit your needs:

Help me create a list of inference providers that match these criteria. First and most important, they MUST offer this exact model: {model}. This is immutable and non-negotiable; a provider that does not offer this model does not make the list. Second, identify the pricing for this model in three categories: cost per input tokens, cost per output tokens, cost per cached tokens. If a provider does not offer pricing for this model for cached tokens, disqualify them. Third, the provider MUST be a zero data retention (ZDR) inference provider. If ZDR pricing is additional, compute it as part of the token costs. ZDR capability is mandatory; if a provider does not offer ZDR, disqualify them. Fourth, if available, obtain the tokens per second for this model for each provider. Return your results as an ordered list in ascending order by lowest cost per tokens for each provider for our chosen model.

You might want to add specific additional needs like SOC 2 compliance, GDPR compliance, etc. To the prompt based on your needs, so customize it any way you like.

With any model you choose, be sure you capture the full model name; DeepSeek v4 Flash for example has an antecedent that lacks the 0731 suffix. Our exact model that we want is DeepSeek v4 Flash 0731.

When I stick this into Google, DeepInfra, Fireworks AI, and OpenCode Zen show up as good options. Which one you choose will be based on how much you prioritize speed versus cost. If you told me to pick one and only one, I’ve been a customer of DeepInfra for a couple of years now and have no complaints at all, BUT there is also a case to be made for OpenCode Zen. More on that in a minute.

What I like, for Leslie’s specific use case, is that pay as you go providers mean that you’re not paying if you’re not using the AI AND there’s a strong economic incentive for you to use AI as little as possible and be as efficient in its use as possible. One of the problems with token plans (which include things like Claude Pro/Max, etc.) is that if you haven’t hit your weekly/monthly limits, you’re wasting money - and thus you’re more likely to use AI frivolously. With pay as you go, you only pay for what you use.

Part 3: The Hardware

For a second computer setup that’s cheap, we have to think about how computers work. The graphical user interface we’re all used to on Windows or Mac or even desktop Linux, as well as on our phones and tablets, is an INCREDIBLY inefficient waste of compute resources. It takes enormous amounts of memory and processing power to draw and maintain the interface compared to pure text-based interfaces.

Now, the crusty old GenX computer nerd in me says that everyone should know how to use a text-based interface (often called a command line or command prompt or terminal), but that’s not realistic. However, there is one user that LOVES text-based interfaces.

Yup, you guessed it. That’s AI. AI is far, far more efficient at typing than it is using a mouse, so for our second computer as Leslie asked about, we’re going to aim for a super cheap box that can run a terminal, store things, and run an AI harness. At a price point of roughly USD 500, you won’t be running AI models directly on this hardware (hence why we searched for hosting providers in Part 2), but you’ll have a decent machine with good storage.

On the shopping venue of your choice, look for what are called gaming mini-PCs. These typically will have AMD Ryzen chips, a small hard drive (typically 256 GB or 512 GB SSD), and between 16 GB and 32 GB of RAM. The cardinal rule of computing is that you want as much RAM/memory as money can buy; a slightly slower processor with more RAM is better than a fast processor with not enough RAM. RAM is like highway lanes; a Bugatti stuck in traffic on a single lane dirt road will not go as fast as a Prius on an 8 lane highway with no traffic jam.

Google AI Mode is handy here as well. Modify this prompt to suit your needs, especially if there are some companies you would prefer not to do business with:

Help me create a list of vendors that offer gaming mini PCs for sale that ship to my location: {locale}. My criteria are that the machine cost less than {amount} inclusive of shipping and that it have a minimum of 16 GB of RAM and 500 GB of disk space, ideally with an SSD. 32 GB of RAM is preferred, and more disk space is always preferred. The system MUST be able to run Ubuntu Linux 26.06 in headless mode. The CPU can be any make or model that offers a base clock speed of 3 GHz; AMD, Intel, or other vendors are fine as long as they are fully supported on Ubuntu. Ethernet and Wifi are mandatory. USB-C, Bluetooth, and HDMI are preferred. eGPU support is preferred. The following vendors and resellers are preferred: {preferred list}. The following vendors and resellers are forbidden: {banned list} The use case for this device will be to run OpenCode and Ubuntu Linux, plus apps like PostgreSQL, Docker, SQLite, Apache2, and similar server-like systems for a single person. Arrange your results in ascending order by price as an ordered list.

I spotted some random Chinese computer on Amazon that was selling for USD 499 with 32 GB of RAM and a 1 TB SSD (affiliate link!) that has a name that resembles what happens if you stuff Scrabble tiles in your mouth and then spit them out on the table. It’s got a 2023 chip in it, the AMD Ryzen 7730U, which is plenty fast for what we’re doing.

Why these specs? You want the extra RAM and disk space so you can run multiple local applications together on the same box. For example, if you want to run Hermes Agent, chances are you’ll want it in something like a Docker container, and the RAM/disk space will REALLY help with that. Likewise, if you want to roll your own local apps, you’ll probably be running a server that has a database, maybe even a local webserver, and more memory and disk space will really help there.

And technically, though it’s a miserable experience, a mini PC like this CAN do local inference with small models. Using something like llama.cpp, koboldcpp, or Unsloth Studio, you COULD theoretically serve up a small model like Gemma 4 E4B and get a modest little AI server running that can do basic things like summarization.

Once you get your hardware, you’ll want to erase the operating system on it and install a version of Linux. Linux is the best choice for this kind of computer because it can run using very few system resources. I’d suggest Ubuntu Linux in LTS flavor - LTS stands for Long Term Support, and the Ubuntu Foundation warrants that a version of Ubuntu will be supported for 10 years from the launch date. The current version is 26.04 LTS, Resolute Raccoon.

One thing that’s nice about Ubuntu Linux is that you can start it up with a GUI for configuration and all the comforts of modern computing, and then once everything is all square, you can shut down the GUI to free up resources, running it in text-based interface alone.

Part 4: The Harness

AI models by themselves are smart but are prone to things like hallucinations. Model hosts can add some guardrails and some tuning, but the real magic happens when you put a good model in a great harness. Think of the model like the engine of a car. An engine isn’t much good by itself, and a car without an engine is a lawn ornament, doubly so if it’s on cinderblocks.

But a good engine in a great car? That’s where, quite literally, the rubber meets the road. And as with cars, sometimes you can do with a good engine but not a great one if the rest of the car is nice. Unless you’re driving NASCAR or Formula 1, a Prius doesn’t need a V8 engine.

Harnesses are the interfaces to AI. When you use ChatGPT on the web, OpenAI has put a harness around its models, and what you experience as ChatGPT is their tuned harness. The same is true for Gemini, for Claude, etc. - and different harnesses do different things. Claude Code, for example, has hundreds of words of specific language built into its system prompts to behave a certain way, plus a slew of deterministic rules that silently reject bad outputs from the Claude models before you ever see them as a user.

For our purposes, because we’re trying to build a scrappy AI setup, we want a harness that’s free and supports multiple providers. For that, there’s no better harness than OpenCode. OpenCode is Free Open Source Software, and it comes in 2 flavors, a command line version and a GUI version. What’s really nice about it is that the GUI version is built on top of the command line version, so you don’t have to do anything differently and you can configure it in GUI mode, then use it later in command line mode with no changes at all.

OpenCode also supports all the modern skills and plugins that other AI systems use, so if you see a cool skill that someone’s showing off in Claude Code, for example, you can install that same skill in OpenCode.

OpenCode also offers 2 paid coding plans to go with it, OpenCode Zen which is pay as you go, and OpenCode Go, which is a subscription plan. There are some free models in OpenCode Zen but those DO collect your data and are not zero data retention. Remember in the world of AI that if you’re not paying, you and your data are the product being sold to someone else.

If you want the convenience of not having to manage your AI hosting separate from your AI harness, OpenCode Zen is a great choice. If you want the lowest possible costs, then use OpenCode with DeepInfra.

Part 5: Wrapping Up, the Process in a Nutshell

So in a nutshell, here is the order of operations to get up and running - and you can put this list (or this entire newsletter, really) into your favorite chat-based AI tool to walk you through this process.

1. Choose a model.

2. Choose a hosting provider.

3. Choose a hardware box of some kind.

4. Install Linux on that box.

5. Install OpenCode on that box.

6. Connect OpenCode to your hosting provider.

With those six big steps, you’re ready to go! I would suggest also installing and configuring tmux on your new box, because that gives you the ability to remote control it very easily, and if you want to get FANCY, install Tailscale on it as well, to create a VPN of your computers together so you can access your new AI box from anywhere in the world. That’s probably best for another time, though your AI of choice can walk you through that pretty easily.

The key to remember here is that this machine is a self-contained little server. Once you have OpenCode installed on it and connected to your model of choice, one of the first tasks you can do is put it to work securing itself. Using your favorite deep research tools, ask for a best practices guide to securing your little computer and optimizing it for AI usage, especially with the most popular command line tools that will help AI save tokens and usage. Because you’re using a provider like DeepInfra, every request you make costs you a tiny bit of money, and the more you can reduce that, the more you save.

If you deploy this system as written, you’ll spend about USD 500 on the hardware, and probably USD 5 to 10 per month in AI inference costs for moderate usage, which I define as a major coding project per week, something where your AI is working with sustained effort for about 2 hours a day. Models like DeepSeek V4 and similarly capable models are incredibly capable and dirt cheap to use.

It’s worth pointing out that if you have a working computer today, you could skip the hardware purchase and install OpenCode on your computer as is, eliminating the hardware cost. For folks on limited incomes, this is a great way to get access to high end AI without high end subscriptions, like job seekers without stable income.

Enjoy running your own little AI operations center!