In-app AI

In-App AI Agent Architecture

Add an AI assistant that actually does things inside your existing Flutter app, using the app’s own actions, without a rewrite.

Clay illustration of an in-app AI agent
In-App AI Agent Architecture featured graphic

How it works

A left-to-right map of the work. Ports stay on fixed sides so the edges do not cross more than they have to. Pan or pinch if you want a closer look.

The usual problem

  • “AI in the app” often means a gimmicky chatbot that cannot act on live data.
  • A vendor-locked agent can force a rewrite, or put an LLM next to payments and identity.
  • Product teams need guided workflows, not another chat window bolted on the side.

How I approach it

  • The agent calls the app’s own APIs and screens. It does not replace them.
  • The orchestration layer is provider-agnostic, so the model can be swapped without touching app code.
  • Context is scoped to the current screen and task, not a dump of the whole product.
  • Sensitive steps stay in the real UI. The agent never handles login, PIN, payments, or NID verification.

Process

  1. 01

    Discovery

    Which journeys the agent may run, and which data it may read.

  2. 02

    Action catalogue

    Existing APIs and screens mapped to tools the agent can call.

  3. 03

    Guardrails

    Three security tiers written down before any model call ships.

  4. 04

    Build

    Orchestration, tool layer, voice input, and streaming replies.

  5. 05

    Prove

    Critical flows tested end to end, including the blocked sensitive path.

  6. 06

    Handover

    Docs for adding a new tool without opening the orchestration core.

Stack and tools

FlutterGetXSTT / TTSStreaming repliesTool callingExisting REST APIsSwappable model provider

Proof from real work

Kadipay

Sazim mobile lead and architecture owner

In-app agent for repair financing: live status, pre-approval, vehicle registration, KYC guidance, and repair requests through screens already in the app.

Open the case study

What you get

  • An orchestration layer the client owns
  • A documented tool catalogue
  • The three-tier security policy
  • Voice and streaming UI components
  • A guide for adding new actions

How risk stays down

  • No core rewrite. The agent sits on top of the existing app.
  • Provider-agnostic, so there is no model lock-in.
  • Login, PIN, payments, and identity stay in dedicated screens.
  • The same layer can move to another Flutter app.

Questions agencies ask