You already have software. Maybe it is a customer portal your last developer built, maybe it is an internal tool for dispatch or quoting, maybe it is a web app that is the business. It works. And now everyone, including your own staff, is asking when it will “have AI.”
The advice you get is either a vendor pitch for a whole new platform or an engineering blog post that assumes you know what an embedding is. Neither tells you the thing you actually need to know: which feature to add first, how it bolts on to what you have, and how to do it without putting the thing that runs your company at risk.
This article is that. By the end you will know how to add AI to existing software in a way that is safe, cheap to try, and easy to undo, and you will know which four features to start with because they pay off almost every time.
What this actually is
Adding AI to existing software almost always means one thing in practice: your application sends some text to a large language model (an LLM, the kind of system behind Claude, GPT-class models, and Gemini) over the internet, gets text back, and does something with the answer. If you want the plain version of what one of those models is and why it sometimes invents things, read What Is an LLM? A Plain Explanation for Business Owners first.
The business analogy is a very fast, well-read temp who sits outside your building. You slide a folder under the door with a note saying “summarize this” or “sort these into three piles,” and the folder comes back a few seconds later. The temp has never seen your systems, remembers nothing between folders, and will confidently guess if the note is vague. So: give the temp what it needs, write a clear note, and check the work before it touches anything that matters.
The key point for an owner: you are not rebuilding your software. You are adding one more thing it can call, the way it already calls a payment processor or an email service. The model is a service at the end of a wire. Your app stays your app.
The four features that pay off first
Four things a model does reliably enough, on the first attempt, that we recommend them as starting points for almost any business application. They share a property: a wrong answer is visible and cheap. Nobody loses money because a summary was slightly off. That is exactly what you want from a first feature.
1. Summaries of things your staff already read
Every business system has long text that people skim: support tickets, job notes, call transcripts, the fourteen-message email thread attached to an order. The first feature to add is a short, consistent summary at the top of each of those records.
The implementation is nearly trivial. When a record is saved, the app sends the text to the model with a fixed instruction like “summarize this in three sentences for a dispatcher who has thirty seconds: what is being asked, what has been done, what is still open.” The answer is stored next to the record and shown above it. The full text stays.
What goes wrong: summaries that are too long, that vary in format from record to record, or that state something as fact when the thread was ambiguous. All three are fixed by writing a tighter instruction (the “prompt,” which is just the note in the folder) and by giving the model two or three examples of a good summary. We cover that in Prompt Engineering for Business Applications That Work. Done well, nobody notices the summary is AI, which is the right outcome.
2. Search that understands meaning, not just words
Your existing search probably matches exact words. Someone types “leaking tap” and misses every record that says “dripping faucet.” Meaning-based search fixes that. The model turns each record into a list of numbers that represents what it is about (an “embedding,” which you can think of as the record’s position on a very detailed map of topics), and a search finds the records whose position is closest to the question.
This runs in a database you probably already have. Postgres, the most common database under business web apps, has an extension called pgvector that stores these lists of numbers and finds the nearest ones. Your developer adds a column, runs every existing record through the embedding model once, and updates it when records change. The same mechanism is how you later give the model the right records to answer from (a technique called RAG).
3. Drafting the reply, never sending it
When a support ticket, a review, an RFQ, or a customer email arrives, the model writes a first reply using the record, your tone rules, and any facts the app can hand it. The draft appears in the reply box. A person reads it, edits it, and clicks send. The model never sends anything.
The discipline that makes it safe is the word “never.” The model must not be able to send, and the app should make it slightly harder to send an unedited draft than an edited one (a “reviewed” checkbox is enough).
What goes wrong: drafts that promise things (a refund, a delivery date, a discount) the model was never told were allowed. Give it the list of things it may offer and tell it to write “I will check on that” for anything else.
4. Classification and tagging
Every intake queue has someone whose job is partly sorting: which department, what urgency, complaint or question. A model does this well and does it the same way at 2 a.m. as at 2 p.m.
Ask the model to pick from a fixed list you define (“one of: billing, technical, sales, other”) and return only the label, as a structured answer the app can trust without parsing prose (LLM Structured Output and Tool Calling, Explained Plainly). Add a “confidence” field and route anything the model marks as unsure to a person.
The trap: letting the model invent categories. If the list is not fixed, you get “billing-ish” and “technical/billing” and your reports fall apart. Fix the list, require exactly one, and reject anything else in code. Classification also benefits most from a small test set, a few hundred old records a human already labeled, so you can measure how often the model agrees.
How it fits into what you already have
5. Put the model behind one door in your code
Every call to the model goes through a single place in your application. The rest of the app never talks to OpenAI or Anthropic directly. It asks your own “AI service” for a summary or a label, and that service handles the provider, the prompt, the retry, and the logging.
Why: providers change prices and models every few months. If the model name is written in forty places, switching is a project. In one place, it is an afternoon. It is also what lets you run a cheap model first and a better one on escalation.
6. Log every request and every answer
The AI service records what was sent, what came back, which model, how long it took, and roughly what it cost, for every call, in your own database, for at least ninety days. When a customer says “your system told me X,” you can find the exact call. When the bill jumps, you can see why.
In the systems we build, this log is the first thing we add, before the first feature. It is also the raw material for improving prompts: the wrong answers you find in the log become the examples you add to the instruction. Without it, every complaint is a mystery.
7. Make every AI feature switchable off
Each feature gets a setting you control that turns it off instantly without a deploy. Provider outages happen. Bills spike. A prompt change makes summaries weird on a Friday afternoon. With the switch off, the record shows no summary, search falls back to word matching, the reply box is empty. Nothing breaks. The app looks like it did before AI, which is the app that already worked.
8. Keep the model away from anything irreversible, for now
The four features above share one property: the model advises and a person or a rule acts. Do not, in the first six months, let the model send messages, change prices, issue refunds, delete records, or write to your accounting system on its own. Those features need the guardrails and approval steps in AI Hallucination Guardrails for Business Applications, and you will build them better once a few months of logs tell you where the model is reliable.
Picture a business like this one
The business below is a composite of the kind of company that writes to us, not a client. The numbers describe the shape of the problem, not a case study.
Picture a business like this one: a regional HVAC service company with eleven technicians, two dispatchers, and an internal web app a contractor built four years ago. The app holds jobs, customer records, and the notes technicians type on their phones. It works, it is not pretty, and nobody wants to replace it.
What was wrong: the dispatchers spend the first hour of every day reading yesterday’s technician notes to decide which jobs need a follow-up visit and which need a parts order. Notes are long and full of shorthand. Search only works on the exact word the technician used. Customer emails asking “what happened with my furnace” take fifteen minutes each.
What gets built, in order:
- A single AI service inside the existing app, with logging and an on/off switch per feature, plus a hidden admin page showing yesterday’s calls and their cost.
- A three-line summary on every job, generated when the technician saves notes, in a fixed format: what was found, what was done, what is still needed.
- A “needs follow-up” and “needs parts” classification on each job, from a fixed list, shown as a filter on the dispatch board. Anything the model marks uncertain shows a grey flag instead of a colored one.
- Meaning-based search over notes using pgvector in the app’s existing Postgres database, alongside the old word search.
- A draft reply to customer status emails, built from the job summary and the company’s tone rules, shown in the reply box for a dispatcher to edit and send.
What changes: the morning read-through drops from an hour to about fifteen minutes. Customer status replies take a couple of minutes. Nothing about how technicians work changes, which is why they do not resist it. The app is the same app. It just reads its own notes now.
What it costs to run
The model is billed by usage, in units called tokens (roughly three quarters of a word each). For the features above, a business this size sends a few thousand short requests a month. In our experience that lands somewhere between a few dollars and a few tens of dollars a month with a mid-tier model, and less with the cheaper tiers most of these features can use. Check the provider’s current pricing page; prices have been falling for years.
Embeddings for search are a one-time cost to process existing records (usually a few dollars for tens of thousands of records) and then a fraction of a cent per new record. pgvector is free; it runs inside the Postgres you already pay for. Expect no change to the server bill, since the heavy work happens at the provider.
The real running cost is people: someone reads the log for twenty minutes a week for the first two months, improves the prompt, and decides when to turn the next feature on. Budget that time. It is where the quality comes from.
The mistakes we see most
- Starting with a chatbot. A chat window is the hardest feature to get right and the easiest to get embarrassed by. The four features above deliver more value with less risk.
- Calling the provider from everywhere. Model names and prompts scattered through the code base. Six months later nobody can change the model without a week of work.
- No log. The first complaint arrives and nobody can find out what the model was shown or what it said.
- Letting the model act. A draft becomes an auto-send “to save time.” Then it promises a refund that was never authorized.
- Vague instructions. “Summarize this” produces a different shape every time. A precise format and two examples fix it in an hour.
- Skipping the test set. A classification feature ships with no measurement of how often it is right, and quietly corrupts the reports.
When to bring in help
If your software is a platform with an AI add-on (a CRM with built-in summaries, a help desk with suggested replies), turn the add-on on and try it. You do not need a developer for that, and it will teach you what you like.
If your software is custom, you need whoever maintains it, or a developer comfortable in its language, to do this properly: the single AI service, the log, the switches, the prompts in version control. A capable developer can add the first two features in a week or two on a well-kept code base. Choosing the provider is a small decision that is easy to get wrong for the wrong reasons, so read Choosing an LLM Provider for Your Business in 2026 before anyone signs up for anything.
Levelbrook builds this for businesses: we work inside the application you already have, add the features in the order above, and leave you with logs, switches, and prompts you own. Fixed price from a written scope, everything runs in accounts you own, and the form below is how a conversation starts.