Build your first chatbot

Learn in your browser · Chapter 01 of 06

A chatbot looks like magic in a chat window, but it is a few ordinary parts working together. Build them one at a time and each becomes easy to understand and to fix.

At its simplest, a chatbot is a loop with a person in it. The person writes a message. Your program adds it to the conversation, sends the conversation to a model, receives a reply and shows it. Compared with an agent there are no tools and no autonomous steps, which makes it a friendlier place to start. If you later want a bot that takes actions, the loop harness workbook shows what to add.

Scroll sideways to see every column.

Learn in your browser · Chapter 02 of 06

Two choices shape everything else: where the model runs, and whether you write code.

A hosted model runs on a provider's computers. Your program sends the conversation over the internet through an API (a way for programs to talk to each other) and receives the reply. You pay according to use, need no special hardware and can reach very capable models. Your users' words go to the provider, so read their data terms.

A local model runs on hardware you control, such as a capable desktop or a server you rent. Conversations stay with you and there is no per-message fee, but you handle setup, updates and hardware. The very largest models usually need more equipment than most people own, so local models are often smaller.

Scroll sideways to see every column.

The no-code path. Many hosted builders let you paste a system prompt, upload a few documents, test in a preview window and share a private link. You can have a working chatbot in an afternoon. The trade-off is less control over logging, testing and cost, and dependence on the builder. It is an excellent way to prove the idea.

The code path. A short program calls the model itself. You control history, logging, limits and tests, and you can connect the bot to your own systems. It takes longer to start and you take on responsibility for security and running costs.

You can do both. Prototype without code to get the prompt and test set right, then move the same prompt and tests into code. Keep your prompt and tests in your own files, not only inside a builder, so that you can switch tools later. When you are unsure which model to use, your test set is the fair judge: run the same conversations through two options and compare pass rate and cost.

Learn in your browser · Chapter 03 of 06

The system prompt is the standing instruction sent with every request. It is the biggest single lever you have over how the bot behaves.

Give it a clear structure: the role and audience, the scope of what it helps with, rules for what it must always or never do, the tone and format, what to do when unsure, when and how to hand over to a person, and a couple of example exchanges. Here is a template built around an imaginary bakery.

You are Ada, the help assistant for Rowan Bakery (an imaginary shop).
You help customers with opening hours, allergens, cake orders and delivery.

Rules:
- Answer only from the shop information below. If it is not there, say you
  do not know and offer to pass the question to the team.
- Never give medical advice. For allergies, quote the allergen list and ask
  the customer to confirm with the shop before ordering.
- Never ask for card numbers or passwords.
- If asked about anything unrelated to the bakery, decline in one polite
  sentence and steer back.

Tone: warm, brief, plain British English. At most four sentences.

If you cannot help, or the customer is upset, say "I will pass this to the
team" and give hello@example.com.

Example:
Customer: Can you make a cake shaped like a dragon?
Ada: I do not know if we can, so I will pass this to the team. You can
also email hello@example.com.

Shop information:
(opening hours, allergen list, delivery area go here)

Tone is easier to show than to describe. Say who is speaking, how long the replies should be, how formal, and which spelling. One example reply teaches more than a list of adjectives.

A guardrail is a limit that keeps the bot inside its job. A refusal is the bot declining a request. Good refusals are short and polite, do not lecture and offer a next step. Decide in advance what is out of scope, which topics need care (health, legal, money, safety) and what the bot does when it does not know. It should say so plainly rather than guess. It should also say it is an AI if asked.

Learn in your browser · Chapter 04 of 06

The model has no memory of your chat. The feeling of memory comes from your program sending the conversation again on every turn.

On each turn you send the system prompt and the conversation so far, and the model replies to the latest message. Two things follow. Everything the model 'remembers' must be in what you send. And the conversation grows with every turn, so cost and delay grow too, until it reaches the context window: the most text the model can take in at once, measured in tokens. Beyond it, the call fails or the oldest text is lost.

• Keep only the last few turns and drop older ones. This is simple and works for short questions.
• Summarise older turns into a short note and keep recent turns in full.
• Pin key facts, such as an order number, in a small notes section that is sent every turn.

def trim(history, keep=10):
    """Keep the last `keep` messages, starting on a user message."""
    kept = history[-keep:]
    while kept and kept[0]["role"] != "user":
        kept = kept[1:]
    return kept

Learn in your browser · Chapter 05 of 06

A chatbot that seems fine in three tries can fail badly on the fourth. A test set turns hunches into evidence.

A test set is a list of sample conversations, written before you launch, that you replay after every change. Start with fifteen to twenty and add one every time you find a bug. Cover a range of types.

Scroll sideways to see every column.

Write what a good reply must do, not the exact words. Score each test pass or fail, run it two or three times because answers vary, and track the pass rate for each type. Decide a release rule in advance, for example that every rule-breaking and sensitive test must pass. A second model can grade replies against your written criteria, but check its grades by hand before you trust it.

Logging. Record the time, the messages, the prompt version and the token counts for each turn. Logs show real failures, which become new tests. They also hold personal data, so keep as little as you need, never log secrets, tell users what you keep and delete it on a schedule (the ICO guidance in Sources covers AI and data protection).

Cost. You pay roughly for the tokens you send and the tokens you receive, on every call. The system prompt and history are sent again each turn, so long prompts and long chats cost more. Prices change, so check your provider's current price list. Your levers are a shorter prompt, trimmed history, a cap on reply length, a daily limit per user and a smaller model for simple questions.

Learn in your browser · Chapter 06 of 06

Now put the parts together. Choose one path, or start with no code and move to code once the prompt is right.

Path A: no code.

1. Choose a builder that lets you set instructions and test in a preview.
2. Paste your system prompt and any short, non-private information the bot needs.
3. Run your whole test set in the preview and fix failures.
4. Share it privately with a few people before anyone else.

Path B: code. The example below is vendor-neutral Python. client is a placeholder for whichever hosted or local model you use, and make_client_from_environment reads the key from an environment variable so it never appears in the code.

import json, time

SYSTEM_PROMPT = "..."   # the prompt you wrote in chapter 3
MAX_INPUT_CHARS = 1000

def trim(history, keep=10):
    kept = history[-keep:]
    while kept and kept[0]["role"] != "user":
        kept = kept[1:]
    return kept

def log_turn(user_text, bot_text, tokens):
    record = {"time": time.time(), "user": user_text,
              "bot": bot_text, "tokens": tokens}
    with open("chat_log.jsonl", "a", encoding="utf-8") as f:
        f.write(json.dumps(record) + "\n")

def answer(client, history, user_text):
    user_msg = {"role": "user", "content": user_text}
    messages = [{"role": "system", "content": SYSTEM_PROMPT}]
    messages += trim(history + [user_msg])
    reply = client.chat(messages=messages, max_output_tokens=300)
    history.extend([user_msg, {"role": "assistant", "content": reply.text}])
    log_turn(user_text, reply.text, reply.usage_tokens)
    return reply.text

def main():
    client = make_client_from_environment()  # the key is read from the environment
    history = []
    while True:
        user_text = input("You: ").strip()
        if user_text.lower() in ("quit", "exit"):
            break
        if not user_text:
            continue
        if len(user_text) > MAX_INPUT_CHARS:
            print("Bot: That message is too long. Please shorten it.")
            continue
        try:
            print("Bot:", answer(client, history, user_text))
        except Exception:
            print("Bot: Sorry, something went wrong. Please try again.")

if __name__ == "__main__":
    main()

Notice what the program includes: a fixed system prompt, trimmed history, a check on input length, a cap on reply length, logging, and a friendly message if the call fails. The history changes only after a successful call, so a failure cannot leave the conversation broken. It does not yet have rate limits, streaming or a web page. The next workbook adds those.

When you are ready to put the bot on a website, the next workbook, Host a chatbot on your website, safely, covers keys, limits and privacy. If you want the bot to answer from your own documents, read RAG and knowledge bases.