Gemini Nano in Chrome: The Prompt API and On-Device General-Purpose LLM

Chrome ships two separate on-device AI models. Gemma 197M handles the task-specific APIs — Summarizer, Language Detector, Translator — each optimised for a narrow output space where world knowledge is unnecessary. Gemini Nano takes the other half of the problem: free-form generation, multi-turn conversation, structured JSON output, and multimodal inference. This article is about that second model and the Prompt API that exposes it — stable in Chrome 148, no server required, no API key.

The two models are complementary. If you know the output type in advance — "summarise this text", "detect the language", "translate to English" — Gemma 197M gives you a zero-configuration path. If you need something Gemma 197M cannot do — custom classification schemas, multi-step reasoning, persona-based writing assistance, or image analysis — you need Gemini Nano via the Prompt API. The decision framework is covered at the end of this article. The rest walks through the API in depth, with working code you can test in Chrome right now.

Two Models, One Browser

Chrome's on-device AI architecture separates inference into two tiers. Gemma 197M is the task specialist: a small, fast model with 197 million active parameters, calibrated precisely for bounded output tasks. Gemini Nano is the generalist: a larger model designed for open-ended language tasks, capable of handling prompts that require broad world knowledge, multi-step reasoning, or structured output conforming to an arbitrary JSON schema (Google Chrome for Developers, 2026e).

The model download is shared across all APIs that use Gemini Nano. A user who has already triggered the download for the Prompt API does not re-download the model when Writer or Rewriter is first used. Chrome manages the model lifecycle transparently — download, caching, and garbage collection are handled by the browser (Google Chrome for Developers, 2026a).

Hardware Requirements and CPU Support

The Prompt API requires Chrome 148 or later running on Windows 10 or 11, macOS 13 (Ventura) or later, Linux, or ChromeOS on Chromebook Plus devices. Storage requirements are significant: at least 22 GB of free disk space on the volume containing the Chrome profile. GPU inference requires more than 4 GB of VRAM. CPU inference — supported from Chrome 140 — requires 16 GB of RAM and at least four CPU cores (Google Chrome for Developers, 2025a).

The Prompt API

The Prompt API exposes Gemini Nano through a LanguageModel global. A session is created with LanguageModel.create() and then prompted with either session.prompt() for a complete response or session.promptStreaming() for a streaming iterator. Sessions hold conversation history automatically — each call to prompt() appends to the context window — making multi-turn interactions straightforward without manual history management.

Availability Check

Before creating a session, check whether the model is ready. LanguageModel.availability() accepts the same options you intend to pass to create() and returns one of four states: 'available' (model cached, use immediately), 'downloadable' (model not yet cached, download needed), 'downloading' (download in progress), or 'unavailable' (hardware does not meet requirements). Pass this check to the user rather than silently failing.

const opts = {
  expectedInputs:  [{ type: 'text', languages: ['en'] }],
  expectedOutputs: [{ type: 'text', languages: ['en'] }],
};

const state = await LanguageModel.availability(opts);

if (state === 'unavailable') {
  throw new Error('Gemini Nano not available on this device');
}
if (state === 'downloadable') {
  // Inform the user: first inference will trigger a ~2 GB download
  showDownloadNotice();
}

LanguageModel.create()

Session creation accepts several optional parameters. initialPrompts is the most important: an array of {role, content} objects that set the conversation's starting state. A system prompt — with role: 'system' — constrains the model's behaviour for the session lifetime. User and assistant turns in initialPrompts allow you to inject conversation history or few-shot examples before the first live user prompt.

async function createSession({
  systemPrompt,
  onProgress,
  signal,
} = {}) {
  if (!('LanguageModel' in self)) throw new Error('Prompt API not supported');

  const opts = {
    expectedInputs:  [{ type: 'text', languages: ['en'] }],
    expectedOutputs: [{ type: 'text', languages: ['en'] }],
  };

  const state = await LanguageModel.availability(opts);
  if (state === 'unavailable') throw new Error('Gemini Nano not available on this device');

  return LanguageModel.create({
    ...opts,
    initialPrompts: systemPrompt
      ? [{ role: 'system', content: systemPrompt }]
      : undefined,
    signal,
    monitor(m) {
      m.addEventListener('downloadprogress', (e) => {
        const pct = e.total ? Math.round((e.loaded / e.total) * 100) : 0;
        onProgress?.(pct);
      });
    },
  });
}

prompt() vs promptStreaming()

session.prompt() returns a Promise that resolves to the complete response string. session.promptStreaming() returns an async iterator that yields incremental delta chunks — each chunk is the new text since the previous chunk, not the full accumulated response. Concatenate with result += chunk. The defensive guard in the code block below is the recommended pattern: it correctly handles delta semantics and is safe against any future API change.

const session = await createSession({
  systemPrompt: 'You are a concise technical assistant. Respond in 3 sentences or fewer.',
});

let result = '';
const stream = session.promptStreaming('What is a context window in an LLM?');

// Defensive guard: handles both cumulative and delta chunk semantics
for await (const chunk of stream) {
  if (chunk.startsWith(result)) result = chunk;
  else                           result += chunk;
  updateUI(result);
}

session.destroy();

Structured JSON Output with responseConstraint

The responseConstraint option on session.prompt() forces the model to emit JSON conforming to a JSON Schema object. Chrome enforces schema compliance at the inference layer — the model cannot produce output that violates the schema. This eliminates the need for post-processing regex or retry loops when you need structured data. It is the correct approach for classification, extraction, scoring, and any task where downstream code expects a specific shape (Google Chrome for Developers, 2026b).

The schema must use English string values for enum and const fields when the session's expectedOutputs language is set to English. The model's free-form reasoning is not part of the JSON output unless you include a reasoning or explanation field in your schema explicitly — and this is often worth doing, because it gives you an audit trail and can improve accuracy.

async function classify(text, schema, systemPrompt) {
  const session = await createSession({ systemPrompt });

  try {
    const raw = await session.prompt(
      `Classify the following text:\n\n"${text}"`,
      { responseConstraint: schema }
    );
    return JSON.parse(raw);
  } finally {
    session.destroy();
  }
}

// Sentiment schema
const sentimentSchema = {
  type: 'object',
  properties: {
    sentiment:    { type: 'string', enum: ['positive', 'negative', 'neutral'] },
    confidence:   { type: 'number' },
    brief_reason: { type: 'string' },
  },
  required: ['sentiment', 'confidence', 'brief_reason'],
};

const result = await classify(
  'The new on-device AI APIs are surprisingly fast and privacy-friendly.',
  sentimentSchema,
  'You are a sentiment classifier. Return only valid JSON.'
);
// → { sentiment: "positive", confidence: 0.93, brief_reason: "..." }

Session Management

A session maintains its own context window — the rolling buffer of tokens that Gemini Nano can attend to. Every prompt and every response consumes tokens from this budget. When the buffer fills, the model starts forgetting earlier turns. Chrome exposes two properties for monitoring this: session.contextUsage (tokens consumed so far) and session.contextWindow (the session's total token budget). Dividing these gives you a fill fraction you can surface in UI or use to trigger context management logic.

const session = await createSession({
  systemPrompt: 'You are a technical documentation assistant.',
});

// Monitor context fill after each turn
function contextFill() {
  return session.contextUsage / session.contextWindow;
}

// Graceful handling: fork before overflow
session.addEventListener('contextoverflow', async () => {
  const fork = await session.clone();
  session.destroy();
  // Continue using fork — it holds the most recent context
  useSession(fork);
});

// Manual clone: fork conversation at a specific checkpoint
const checkpoint = await session.clone();
// One branch continues the main conversation
// Other branch explores a different direction
await session.prompt('Explain the architecture in detail');
await checkpoint.prompt('Give me a one-line summary instead');

session.clone() is particularly useful in agentic patterns where you need to explore multiple response paths from the same conversation state without re-running the entire history. The clone shares the model but maintains independent context from the moment of forking. Both sessions must be destroyed independently.

For production pipelines that run long sessions — document analysis, iterative refinement — monitor contextFill() after each turn and proactively fork before the contextoverflow event fires. Waiting for the event means context has already been truncated; proactive forking preserves the last N turns.

Multimodal Inputs

From Chrome 148, the Prompt API supports multimodal inputs: images, audio, and video alongside text. Declare the expected input types in expectedInputs at session creation, then pass DOM elements or binary data directly in the prompt content array (Google Chrome for Developers, 2026a).

const session = await LanguageModel.create({
  expectedInputs: [
    { type: 'text',  languages: ['en'] },
    { type: 'image' },
  ],
  expectedOutputs: [{ type: 'text', languages: ['en'] }],
});

const imgEl = document.querySelector('img#screenshot');

// Wrap in a {role, content} object; use value (not content) for each part
const stream = session.promptStreaming([{
  role: 'user',
  content: [
    { type: 'image', value: imgEl },
    { type: 'text',  value: 'Describe any UI issues visible in this screenshot.' },
  ],
}]);

let result = '';
for await (const chunk of stream) {
  if (chunk.startsWith(result)) result = chunk;
  else                           result += chunk;
  updateUI(result);
}

session.destroy();

Supported visual input types include HTMLImageElement, HTMLVideoElement, HTMLCanvasElement, ImageBitmap, VideoFrame, Blob, and ImageData. Audio accepts AudioBuffer, ArrayBufferView, ArrayBuffer, and Blob. Multimodal input is not available in Web Workers — the Prompt API requires a top-level window context (Google Chrome for Developers, 2026b).

↳ Live demo — Image analysis (multimodal)

Drop an image here or click to select

Describe image in detail Identify any text (OCR) Detect UI / layout issues Identify main objects Analyse → Clear

Writer API (Origin Trial)

The Writer API generates new content from a task description. It uses Gemini Nano and shares the same model download. The API accepts a sharedContext parameter for batch content generation — you create a single writer instance describing the document or persona, then issue multiple write tasks within that context. This produces coherent output across sections without re-stating context in every call (Google Chrome for Developers, 2026c).

async function writeContent(task, {
  tone   = 'neutral',
  format = 'plain-text',
  length = 'medium',
  sharedContext,
  outputLanguage = 'en',
} = {}) {
  if (!('Writer' in self)) throw new Error('Writer API not available');

  const writer = await Writer.create({
    tone, format, length, sharedContext, outputLanguage,
  });

  let output = '';
  for await (const chunk of writer.writeStreaming(task)) {
    if (chunk.startsWith(output)) output = chunk;
    else                          output += chunk;
  }

  writer.destroy();
  return output;
}

// Generate three coherent sections of a product page in one batch
const context = 'B2B SaaS product page for an AI-powered data pipeline monitoring tool';

const [headline, intro, cta] = await Promise.all([
  writeContent('Write a compelling headline',   { tone: 'formal', length: 'short',  sharedContext: context }),
  writeContent('Write a two-sentence introduction', { tone: 'formal', length: 'medium', sharedContext: context }),
  writeContent('Write a call-to-action label',      { tone: 'formal', length: 'short',  sharedContext: context }),
]);

Rewriter API (Origin Trial)

The Rewriter transforms existing text — adjusting tone, length, or format while preserving meaning. It is the editing complement to the Writer API and is particularly suited to scenarios where you have content of unknown quality entering a system: user-submitted support messages, form fields, or auto-generated descriptions that need to be normalised before storage or display (Google Chrome for Developers, 2026d).

async function rewrite(text, {
  tone   = 'as-is',
  length = 'same',
  format = 'plain-text',
  context,
  outputLanguage = 'en',
} = {}) {
  if (!('Rewriter' in self)) throw new Error('Rewriter API not available');

  const rewriter = await Rewriter.create({ tone, length, format, outputLanguage });

  let output = '';
  for await (const chunk of rewriter.rewriteStreaming(text, { context })) {
    if (chunk.startsWith(output)) output = chunk;
    else                          output += chunk;
  }

  rewriter.destroy();
  return output;
}

// Formalise a user-submitted support ticket
const formal = await rewrite(
  "hey so basically the dashboard just stopped working, no idea why lol",
  {
    tone:    'more-formal',
    length:  'same',
    context: 'User support message submitted via web form',
  }
);
// → "The dashboard has become unresponsive. The root cause is currently unknown."

Gemini Nano vs Gemma 197M

The two models share the same hardware requirements and the same browser distribution mechanism, but they solve different problems. The decision point is whether you can define the output type at build time. If you can — "summarise this", "detect this language", "translate this" — Gemma 197M's task APIs are faster, more efficient, and require no prompt engineering. If the output type is dynamic, user-specified, or requires reasoning you cannot pre-encode — classification with a custom schema, structured extraction, multi-turn Q&A — Gemini Nano via the Prompt API is the right tool.

In practice, many production pipelines use both. A document processing flow might use the Language Detector to identify source language, the Translator to normalise to English, the Summarizer to condense, and then the Prompt API with a responseConstraint to extract structured metadata from the summary — four on-device calls, no server, no tokens billed.

Production Checklist