Blog

Notes from production

Practical field notes on shipping LLM systems that survive real users — cost, reliability, retrieval, and agents. No hype, just what works. Filter by who each piece is written for.

Filter by audience
Jul 27, 2025 6 min read

QLoRA: fine-tuning a big model on one GPU — and the knobs that matter

QLoRA fine-tunes a large model by freezing a 4-bit-quantized base and training a tiny adapter. It put fine-tuning within reach of a single GPU. Here's how it works and which hyperparameters to tune.

Engineering
Read
Jul 26, 2025 6 min read

How an LLM is trained, end to end: from raw data to a fine-tuned model

Behind a chat model is a long pipeline: curate data, pre-process it, pretrain a base model, scale to a frontier model, then fine-tune. Here's the journey, stage by stage.

EngineeringProduct
Read
Jul 20, 2025 7 min read

Making RAG better: a menu of techniques that actually move the needle

Basic RAG is easy; good RAG is a stack of techniques. From chunking and reranking to query rewriting, Graph RAG, and agentic retrieval — here's the toolkit, organised by where it helps.

EngineeringProduct
Read
Jul 19, 2025 6 min read

Vector databases: where your embeddings live

Once you're embedding text, you need somewhere to store and search millions of vectors fast. Here's what a vector database does, the popular options, and when a Postgres extension is enough.

Engineering
Read
Jul 13, 2025 5 min read

LangSmith: seeing what your LLM app is actually doing

LLM apps fail in ways ordinary logs don't capture. LangSmith gives you tracing, evals, and monitoring across the whole chain — and it works whether or not you build on LangChain.

EngineeringProduct
Read
Jul 12, 2025 6 min read

LangGraph: when your agent needs a real state machine

LangGraph models an agent as a graph of nodes and edges with shared state — giving you explicit, durable control over the multi-step and cyclic flows a linear chain can't express.

Engineering
Read
Jul 6, 2025 6 min read

Encoders and vector embeddings: turning meaning into numbers

Generative models get the attention, but encoder models quietly power search, RAG, and recommendations by turning text into vectors that capture meaning. Here's how embeddings work.

EngineeringProduct
Read
Jul 5, 2025 6 min read

Measuring a GenAI solution: technical metrics and business metrics

A model can ace every technical metric and still fail the business — or the reverse. You need both layers, and you need them connected. Here's the two-part scorecard.

Founders & BusinessProductEngineering
Read
Jun 29, 2025 6 min read

Three kinds of AI product: GPT wrappers, specialists, and agents

Not all AI apps are built — or defended — the same way. A look at the three archetypes: thin wrappers, domain specialists, and agentic products, and where the moat actually is.

Founders & BusinessProductEngineering
Read
Jun 28, 2025 6 min read

Reading LLM benchmarks: MMLU, GPQA, AIME, LiveCodeBench, MuSR, HLE

Every model launch quotes a wall of benchmark scores. Here's what the big ones actually measure, why some are saturating, and how to read them without being sold to.

EngineeringProductFounders & Business
Read
Jun 22, 2025 5 min read

The Chinchilla scaling law: the 20-tokens-per-parameter rule of thumb

DeepMind's Chinchilla showed most big models were undertrained — and gave a rule of thumb for balancing model size against data. Here's the idea and why it reshaped how models are built.

EngineeringProduct
Read
Jun 21, 2025 5 min read

Three levels of the Hugging Face API: pipelines, tokenizers, and models

The transformers library lets you work at whatever altitude you need — a one-line pipeline, or hands-on with the tokenizer and the model. Knowing the three levels tells you when to drop down.

Engineering
Read
Jun 15, 2025 5 min read

Google Colab + Hugging Face Datasets: a free lab for AI experiments

Colab gives you a free GPU-backed notebook in the browser; Hugging Face Datasets gives you thousands of ready datasets in one line. Together they're the fastest way to start experimenting.

Engineering
Read
Jun 14, 2025 5 min read

Text to speech with OpenAI's audio API

One call turns text into natural-sounding speech. Here's how OpenAI's text-to-speech API works — the models, voices, formats, and streaming — and where it fits in a product.

EngineeringProduct
Read
Jun 8, 2025 5 min read

Image generation with DALL·E 3: prompt in, picture out

DALL·E 3 turns a plain-language description into an image, and its real leap was prompt-following. Here's how it works, how to call it, and what to know before you build on it.

EngineeringProduct
Read
Jun 7, 2025 6 min read

Generative AI vs Agentic AI: what's the difference, and which does your business need?

Generative AI produces content when you ask; agentic AI pursues a goal and takes actions on its own. The distinction matters because they solve different problems — and carry different risks.

Founders & BusinessProductEngineering
Read
Jun 1, 2025 6 min read

RAG fundamentals: giving a model the knowledge it wasn't trained on

Retrieval-Augmented Generation lets a model answer from your data — docs, tickets, a knowledge base — without retraining it. Here's the whole pipeline, from indexing to a grounded answer.

EngineeringProduct
Read
May 31, 2025 7 min read

The anatomy of an agent framework: loop, tools, MCP, and structured output

Strip away the branding and every agent framework assembles the same few parts: an agent loop, tool calling, a way to connect tools (increasingly MCP), and structured outputs. Here's how they fit.

Engineering
Read
May 25, 2025 6 min read

The agent framework landscape: lightweight SDKs vs orchestration frameworks

OpenAI Agents SDK, Google ADK, AWS Strands, CrewAI, LangGraph — the space looks crowded, but it splits into two camps. Knowing which camp a tool belongs to tells you what it's good at.

EngineeringProduct
Read
May 24, 2025 6 min read

Where your agent actually runs: Lambda, Bedrock AgentCore, Anthropic Managed Agents, Vertex Agent Engine

Writing an agent is one thing; hosting it so it runs reliably, holds state, and scales is another. A tour of the runtimes — from general-purpose serverless to purpose-built agent platforms.

EngineeringFounders & Business
Read
May 18, 2025 6 min read

The OpenAI Responses API: should you use it, or stick with Chat Completions?

OpenAI's Responses API is its new default for building on the platform — stateful, with built-in tools and agentic features. Here's what it changes, and when Chat Completions is still the right call.

EngineeringProduct
Read
May 17, 2025 5 min read

CrewAI Studio: assembling a team of agents, visually

CrewAI models multi-agent systems as a crew — agents with roles, given tasks, working together. Studio puts that on a low-code interface so you can build and run a crew without wiring it in Python.

ProductFounders & BusinessEngineering
Read
May 11, 2025 5 min read

ElevenLabs: building voice agents you can actually talk to

ElevenLabs built its name on lifelike text-to-speech; its conversational AI platform assembles the whole voice-agent stack — speech-in, LLM, speech-out, turn-taking — into one builder.

ProductFounders & BusinessEngineering
Read
May 10, 2025 5 min read

OpenAI's Agent Builder: agents on a visual canvas

OpenAI's Agent Builder brings agent creation onto a drag-and-drop canvas — nodes, tools, and guardrails wired together visually. Here's what it is and who it's really for.

EngineeringProductFounders & Business
Read
May 4, 2025 5 min read

Gradio: a working UI for your model in a dozen lines of Python

Gradio turns a Python function into a shareable web app with inputs, outputs, and a link — no front-end code. It's how AI demos and internal tools get built in an afternoon.

EngineeringProduct
Read
May 3, 2025 6 min read

Prompt caching: stop paying full price for the same prefix

If every call resends the same long system prompt and context, you're paying to process identical tokens over and over. Prompt caching fixes that — here's how OpenAI, Anthropic, and Gemini each do it.

EngineeringFounders & Business
Read
Apr 27, 2025 5 min read

LiteLLM: one OpenAI-shaped interface for a hundred providers

LiteLLM lets you call OpenAI, Anthropic, Bedrock, Gemini, and local models through one consistent format — as a library or a proxy with keys, budgets, and logging. Here's where it fits.

Engineering
Read
Apr 26, 2025 6 min read

LangChain: the framework everyone starts with (for better and worse)

LangChain is the best-known toolkit for building LLM apps — chains, agents, memory, and integrations for everything. Here's what it gives you, and an honest take on when to use it.

EngineeringProduct
Read
Apr 20, 2025 5 min read

The reasoning-effort knob: paying for thinking only when it's worth it

Reasoning models let you dial how hard they think — low, medium, or high. Used well it's a precise lever over the quality-versus-cost trade. Used carelessly it's a silent bill.

EngineeringProductFounders & Business
Read
Apr 19, 2025 5 min read

Streaming LLM responses: why the tokens arrive one at a time

That typewriter effect isn't a gimmick — it's the model streaming tokens as it generates them. Here's how streaming works, why it transforms perceived speed, and where it gets tricky.

EngineeringProduct
Read
Apr 13, 2025 6 min read

Structured outputs: getting reliable JSON out of a text model

If your code has to parse the model's answer, 'please return JSON' isn't enough. Structured outputs and schema enforcement turn a probabilistic text generator into a dependable API.

Engineering
Read
Apr 12, 2025 5 min read

Zero-, one-, and few-shot prompting: teaching by example, in the prompt

One of the most reliable ways to steer a model is to show it examples. Here's the difference between zero-, one-, and few-shot prompting, and when each earns its tokens.

EngineeringProduct
Read
Apr 6, 2025 5 min read

The context window: the model's short-term memory, and its hard limit

Everything a model 'knows' in a request has to fit in its context window. Understanding that budget — what fills it, what it costs, where it fails — is core to building with LLMs.

EngineeringProduct
Read
Apr 5, 2025 5 min read

Tokens and tokenisers: the units your model actually reads (and bills)

Models don't see words or characters — they see tokens. Understanding the tokeniser explains your context limits, your bill, and why some languages cost more than others.

EngineeringProduct
Read
Mar 30, 2025 5 min read

Inference-time scaling: buying better answers with more thinking

The newest way to get a smarter answer isn't a bigger model — it's letting the model think longer at inference. Reasoning models made test-time compute a scaling axis of its own.

EngineeringProductFounders & Business
Read
Mar 29, 2025 5 min read

Training-time scaling: why bigger, longer, and more data kept working

For years the recipe for a better model was blunt: more parameters, more data, more compute. Scaling laws made that predictable — and eventually expensive. Here's the idea and its limits.

EngineeringProductFounders & Business
Read
Mar 23, 2025 5 min read

Model parameters: what they are, and why the count drives your bill

"7B", "70B", "quantized to 4-bit" — parameter counts and precision decide a model's memory, speed, and cost. Here's what a parameter really is, in operational terms.

EngineeringProduct
Read
Mar 22, 2025 7 min read

Self-attention and RLHF: how models focus, and how they're aligned

Two ideas do a lot of the heavy lifting in modern LLMs: self-attention, which lets a model decide what to focus on, and RLHF, which aligns it to what humans actually want.

Engineering
Read
Mar 16, 2025 7 min read

The Transformer: the architecture behind every modern LLM

The 2017 paper "Attention Is All You Need" replaced sequential processing with attention — and made today's LLMs possible. Here's the architecture, in plain terms.

Engineering
Read
Mar 15, 2025 6 min read

Base, Instruct, Reasoning, Hybrid: the LLM model types, explained

Not all LLMs are the same kind of thing. Knowing the difference between a base model, an instruct model, a reasoning model, and a hybrid tells you which one to reach for.

EngineeringProductFounders & Business
Read
Mar 9, 2025 6 min read

LLMs, neural networks, and parameters: what "7B" actually means

Under the branding, a language model is a big neural network, and its parameters are just numbers — weights and biases — learned from data. Here's the foundation in plain terms.

EngineeringProduct
Read
Mar 8, 2025 5 min read

Hugging Face: the GitHub of machine learning

Models, datasets, demos, and the libraries that tie them together — Hugging Face is where the open-source AI world lives. Here's a tour and why it matters.

EngineeringProduct
Read
Mar 2, 2025 5 min read

Amazon Bedrock: managed GenAI for teams already on AWS

Bedrock gives you many foundation models behind one API, without running any infrastructure — and keeps your data inside your AWS boundary. Here's what it is and who it fits.

EngineeringFounders & Business
Read
Mar 1, 2025 6 min read

Distillation: how DeepSeek packed a big model's smarts into small ones

Knowledge distillation trains a small "student" model to imitate a large "teacher". DeepSeek used it to hand its reasoning ability to models a fraction of the size — here's how it works.

EngineeringProductFounders & Business
Read
Feb 23, 2025 6 min read

Scraping modern websites with Playwright

Half the web is rendered by JavaScript, and a plain HTTP request sees none of it. Playwright drives a real browser — here's how to use it to scrape, and how to do it responsibly.

Engineering
Read
Feb 22, 2025 6 min read

The Chat Completions API: the interface that became a standard

OpenAI's Chat Completions API is the messages-based interface most of the industry now speaks. Here's how it works — roles, parameters, streaming, and tool calls.

Engineering
Read
Feb 16, 2025 6 min read

Prompt foundations: the anatomy of a prompt that works

A prompt isn't a magic incantation — it's an interface. Here are the fundamentals: the roles, the structure, and the handful of techniques that do most of the work.

EngineeringProduct
Read
Feb 15, 2025 5 min read

Jupyter Notebooks: the AI engineer's scratchpad

Before the code becomes a service, it lives in a notebook. Here's what Jupyter is, why it's the default workspace for AI work, and the traps that bite teams who forget it's a scratchpad.

Engineering
Read
Feb 9, 2025 5 min read

OpenRouter: one API for every model

OpenRouter is a single, OpenAI-compatible gateway to hundreds of models across providers — with fallback, routing, and unified billing. Here's when to route through it.

EngineeringProductFounders & Business
Read
Feb 8, 2025 5 min read

Groq: when latency is the feature

Groq runs open models on custom hardware at speeds that change what real-time AI can do. Here's what it is, and where that speed is worth designing around.

EngineeringProductFounders & Business
Read
Feb 2, 2025 5 min read

Ollama: run open models on your own machine — and when that's the right call

Ollama makes running open-weight LLMs locally as easy as one command. Here's what it is, when local inference is genuinely the right choice, and when it isn't.

EngineeringFounders & Business
Read
Feb 1, 2025 5 min read

Stop anthropomorphizing your agents (especially behind the scenes)

Giving your assistant a friendly name in the UI is fine. Designing your architecture as if the model thinks, remembers, or intends anything is a bug waiting to page you.

EngineeringProduct
Read
Jan 26, 2025 5 min read

Before you build the AI thing: what problem are we actually solving?

A snarky reminder from your friendly neighbourhood AI engineer: not everything needs a large language model. Some problems need a database index, a form, or someone to just make a decision.

Founders & BusinessProductEngineering
Read
Jan 25, 2025 6 min read

Guardrails: the line between an agent demo and one you can deploy

An agent with real tools and real users needs more than a clever prompt. Input and output guardrails are what keep it safe, on-topic, and out of trouble in production.

EngineeringProduct
Read
Jan 19, 2025 6 min read

Observability and evals: measure against the production goal, not the vibe

A dashboard full of green metrics means nothing if none of them predict the outcome you actually ship for. Tie every trace and eval back to the end goal — or you're measuring theatre.

EngineeringProduct
Read
Jan 18, 2025 Live dashboard

Independent analysis of AI: live model benchmarks

Model quality, speed, and price move every week. Rather than quote a snapshot that's stale by tomorrow, here's the live, vendor-neutral dashboard from Artificial Analysis — embedded in full.

EngineeringProductFounders & Business
Read
Jan 12, 2025 7 min read

n8n explained: what it is, and which businesses it's right for

n8n has become the default way teams glue their tools together and add AI to everyday operations. Here's a plain-English intro — and an honest look at who it fits, and who should look elsewhere.

Founders & BusinessProductEngineering
Read
Jan 11, 2025 8 min read

Workflows vs AI agents: which one does your problem actually need?

"Agent" is the word everyone reaches for — and often the wrong tool. A clear-eyed comparison of workflows vs agentic frameworks: the pros, the cons, the risks nobody demos, and how to choose.

EngineeringProductFounders & Business
Read
Jan 5, 2025 7 min read

How we cut LLM spend 38% without losing quality

Model routing, prompt caching, and per-tenant budgets sound obvious. The hard part is proving quality held. Here's the playbook — and the evals that made it safe to ship.

EngineeringFounders & Business
Read
Jan 4, 2025 8 min read

Building RAG that doesn't hallucinate in front of customers

A demo that answers your three test questions is easy. A retrieval system that stays grounded across thousands of real queries is a different discipline. Here's what actually matters.

EngineeringProduct
Read
Dec 29, 2024 6 min read

Designing AI agents that actually finish the job

Most agent demos loop, stall, or wander off. Shipping agents that complete real workflows is about state, tools, validation, and knowing when to ask a human.

EngineeringProduct
Read
Dec 28, 2024 5 min read

Evaluator–Optimizer: let the model critique its own work

Pair a generator with an evaluator in a feedback loop — generate, critique, refine — until the output clears a bar you define. Iteration as an AI workflow.

Engineering
Read
Dec 22, 2024 5 min read

Orchestrator–Workers: decompose the task at runtime

When you can't know the subtasks in advance, let an orchestrator LLM break the work down on the fly, delegate to workers, and synthesise their output.

Engineering
Read
Dec 21, 2024 4 min read

Parallelization: run LLM calls side by side, then combine

Fan a task out into independent LLM calls that run at once — either sectioning a job into parts, or voting for reliability — then aggregate the results.

Engineering
Read
Dec 15, 2024 4 min read

Routing: send each input to the model that should handle it

Classify the incoming request first, then dispatch it to a specialised prompt or model. Separation of concerns for AI workflows — and a quiet way to cut cost.

Engineering
Read
Dec 14, 2024 4 min read

Prompt Chaining: the workflow pattern to reach for first

The simplest AI workflow pattern — decompose a task into a fixed sequence of LLM calls, each handling one job, with gates to catch failures before they compound.

Engineering
Read