---
title: Transformers Chat
description: Run a small chat model on-device with transformers.js
---

Give an AI node a small chat model that runs inside your browser, even on a computer without WebGPU.

## What it does

Transformers Chat is a model you plug into the **Model** slot of an AI node, such as the [Tools Agent](/nodes/builtin/ai/aiagents/toolsagentnode/) or [Basic LLM Chain](/nodes/builtin/ai/aiagents/basicllmchainnode/). It does not run on its own and has no output. When the AI node runs, a small instruct model (for example SmolLM2) answers on your device. It uses WebGPU when available and falls back to the processor otherwise. No API key, no cloud call.

* **Temperature** is 0.5 and **Max new tokens** is 256 by default, so replies are short unless you raise it.
* The model works with about 2,000 tokens of context, so a connected [Chat Memory](/nodes/builtin/ai/aidependencies/chatmemories/chatmemory/) sends only the latest part of the conversation.
* It loads on first use (downloading it if needed), then stays ready for the session.

### Using it with tools

It works with the [Tools Agent](/nodes/builtin/ai/aiagents/toolsagentnode/). These models have no built-in tool feature, so the tools are described in the prompt and the model writes its tool calls in its reply. The common formats are all understood: `<tool_call>…</tool_call>`, `[TOOL_CALLS] […]`, `<function=name>…</function>`, a fenced JSON block, or a reply that is only the JSON call. The agent runs the tool, sends back the result, and the model answers or calls another tool. Hidden reasoning (`<think>…</think>`) is removed from the reply.

## When to use it

* You want local chat but your device has no WebGPU, so [Web LLM](/nodes/builtin/ai/aidependencies/llm/wbellm/) can't run.
* Quick, low-stakes replies with the smallest possible model.
* You already use on-device models for other tasks and want the same engine.

For better answers on a device with WebGPU, use [Web LLM](/nodes/builtin/ai/aidependencies/llm/wbellm/).

## Inputs and settings

The **Model** picker lists only chat models for this engine. Install one from the [Local Models](/app/local-models/) page, or download it from the picker. When you publish or share the workflow, **Model** is cleared, so whoever installs it picks their own.

## Outputs

This node has no output port. It plugs into a slot on another node instead of passing items along.

* **Model** slot of [Tools Agent](/nodes/builtin/ai/aiagents/toolsagentnode/), [Q\&A Agent](/nodes/builtin/ai/aiagents/qanode/), [RAG Agent](/nodes/builtin/ai/aiagents/ragnode/), [Basic LLM Chain](/nodes/builtin/ai/aiagents/basicllmchainnode/), [Indexer](/nodes/builtin/ai/aiagents/indexernode/), [Text Classifier](/nodes/builtin/ai/aiagents/textclassifiernode/), [Sentiment Analysis](/nodes/builtin/ai/aiagents/sentimentanalysisnode/), [Summarization](/nodes/builtin/ai/aiagents/summarizationchainnode/), [Information Extractor](/nodes/builtin/ai/aiagents/informationextractornode/), [Guardrails](/nodes/builtin/ai/aiagents/guardrails/), [Auto-fixing Output Parser](/nodes/builtin/ai/aidependencies/outputparser/autofixingoutputparser/)

The AI node you connect it to produces the output. Transformers Chat only decides which model answers.

## Dependencies and credentials

None. This node needs no connection and has no slots for other nodes.

## Example workflow

1. A [Context Menu](/nodes/builtin/trigger/contextmenu/) trigger on selected text.
2. [Basic LLM Chain](/nodes/builtin/ai/aiagents/basicllmchainnode/) with **Prompt** `Give this text a short title: {{ $input.selectionText }}`.
3. Transformers Chat in its **Model** slot, with an installed SmolLM2 instruct model and **Max new tokens** `40`.
4. [Clipboard Write](/nodes/extension/clipboardwrite/) with **Text** `{{ $input.response }}`.

## Common issues

* **"Model parameter is required and cannot be empty".** Pick a model in **Model**.
* **"… isn't a transformers.js chat model. Pick one from the picker, or use the Web LLM node."** The model belongs to another engine (often after importing a workflow). Pick one from this node's list, or use [Web LLM](/nodes/builtin/ai/aidependencies/llm/wbellm/).
* **Replies are cut short.** Raise **Max new tokens**. This also fixes tool calls that get cut off.
* **The agent never calls a tool.** The model answered in plain text. Use an instruct model trained for tool use, give tools clear descriptions, and keep the system message short. A call to a tool that isn't connected, or with broken JSON, stays as plain text.

## Related nodes

* [Web LLM](/nodes/builtin/ai/aidependencies/llm/wbellm/): larger in-browser models, needs WebGPU.
* [Generate Text](/nodes/builtin/ai/localai/generate-text/): on-device text generation as a standalone node.
* [Tools Agent](/nodes/builtin/ai/aiagents/toolsagentnode/): an agent that can use tools and memory.