we build pebble, an ai chat widget that drops into any site. the chat itself is the easy bit. the real engineering is in everything that has to not happen: the widget must not read the host page, must not expose a key, and must never let one client's data surface for another. here are the notes.
isolation from the host page
the widget runs inside an isolated frame, not directly in the host's document. that boundary cuts both ways. the host site can't reach in and scrape the conversation or the widget's internals, and the widget can't reach out and read the host's page, forms, or cookies. the embed stays a sealed box that talks to its own backend and nothing else. for the host, that's the whole point: adding a chat widget shouldn't widen their attack surface.
strict separation between tenants
one widget serves many clients. the cardinal rule is that client a's knowledge base can never bleed into client b's answers. that's enforced in more than one place, not trusted to a single check: the data layer isolates each tenant's rows, and the application enforces the same boundary again on the way out. defence in depth, because a single guard is a single point of failure.
proving isolation, not assuming it
claiming tenants are separated is easy. proving it is the work. one reliable technique is the canary: seed a unique, meaningless fact into a single tenant's corpus, then continuously assert that this fact only ever appears in that tenant's responses and never anyone else's. if the canary ever shows up where it shouldn't, isolation has broken, and you find out immediately in testing rather than in production with real data.
the cost guard sits before the model
a note that belongs here too. the threat to running cost is request volume, not message length. and only a hard stop placed before the model call is truly free. once a request reaches the model, even a refusal costs tokens. so abuse and runaway traffic get stopped at the edge, ahead of the expensive part, not politely declined by the model after the bill has already been run up.
the goal
a chat widget done right is boring to the host site and invisible to every tenant but its own. no leaks, no crossover, no surprise spend. unremarkable, which is exactly what you want from something sitting on a client's page in front of their customers.