Model usage assessment

Who it is for

Product leads, engineering managers, and founders of AI-assisted apps who cannot currently say, with confidence, which screens, jobs, and user paths generate model calls — and in what volume.

What you leave with

A written usage picture for a named app: a call inventory by route and purpose, a short narrative of what the week actually looked like, and a list of questions the current logs still cannot answer.

Scope

One production app (or a clearly bounded staging twin that mirrors production traffic). We look at model-backed user paths, background jobs that invoke a model, retries, fallbacks, and any silent calls that fire without a visible control.

Included

• Intake call to name the app, the model providers in use, and the screens believed to be model-backed
• A call inventory: each distinct invocation site, what it is for, and whether a person is waiting on the result
• Reconstruction of one recent measurement window (typically seven to fourteen days of logs or traces)
• Notes on retries, duplicate fires, and fallbacks that inflate apparent usage
• A written report in plain language, plus a ninety-minute walkthrough

Not included

• Rewriting prompts or changing model vendors
• Building or installing a usage product inside your stack
• Load testing, red-teaming, or security assessment
• Legal advice on training data or output ownership

Who does the work

Aisha Rahman leads the assessment from our Petaling Jaya office. Wei Liang Tan joins when the logs need sequencing work. Priya Menon runs the walkthrough if the audience is mixed (product, ops, and engineering).

Sequence

1. Name the app. We agree which app, which environments, and which measurement window count. You name a counterpart who can grant log access or sit with us while we read.
2. Take a call inventory. We list every place a model is invoked — composer, summariser, classifier, background rewrite — and mark whether a person is waiting.
3. Read the window. We reconstruct the chosen days from traces, request logs, provider invoices, or a supervised export. Gaps are written down, not guessed.
4. Write the picture. The report describes what actually fired, what sat idle, and which questions remain unanswerable until logging changes.
5. Walk through it. A ninety-minute session in person at SS15 or on a call. We read the report together; we do not present slides in place of the document.

Duration

Most assessments take ten to fifteen working days from confirmed access to the walkthrough. Thin logs add time; we say so at intake rather than filling gaps with estimates.

Where it happens

Work is led from 1 10 Jln Ss15/4B Ss15 Petaling Jaya. On-site days in the Klang Valley are available. Remote reading of logs is fine when access is supervised and the counterpart can answer route questions the same day.

What to prepare

Before day one we need: a named app, a list of model-backed screens as you currently understand them, access to logs or a sitting session with someone who has it, and (if you have them) provider invoices for the same window.

Limits

We cannot assess an app we cannot observe. If logs omit request identity, we will spend part of the window reconstructing sequences from timestamps — and we will say how much of the picture is inferred. We do not sign up to a second app under the same fee.

Fee basis

From RM 8,400 for a single app and a two-week window. The fee moves with the number of invocation sites, whether traces already exist, and whether we must sit with you to read logs. A written estimate follows the intake call.

Next step