nodes / builtin / ai / aidependencies / llm
Local Q&A Model
Answer questions on-device by quoting your Knowledge
Let the Q&A Agent answer on your device by quoting the right passage from your knowledge base, instead of asking a chat model.
What it does
Section titled “What it does”Local Q&A Model is a model you plug into the Model slot of the Q&A Agent. With it connected, the agent answers in your browser, free and private, and only with text that is really in your documents. When the agent runs:
- It finds the most relevant passages in its connected Local Knowledge (Knowledge documents (k) on the agent, 4 by default).
- The model looks for the answer in each passage (each cut to Max knowledge context length, 4000 characters by default).
- The best-scoring answer wins, with the document it came from.
Because it can only quote, the Q&A Agent must have a Knowledge connected when you use this model.
When to use it
Section titled “When to use it”- Look up facts in your own documents without sending them or the question to a server.
- You want short answers quoted word for word, not generated text.
For open questions that need a written, combined answer, connect a chat model such as Web LLM instead.
Inputs and settings
Section titled “Inputs and settings”| Setting | Type | What it does |
|---|---|---|
| Model | Model picker | An on-device extractive Q&A model (e.g. DistilBERT SQuAD). Install one from Local AI, or download it from the picker. |
The Model picker lists on-device question-answering models (for example DistilBERT SQuAD). Install one from the Local Models page, or download it from the picker. When you publish or share the workflow, Model is cleared, so whoever installs it picks their own.
Outputs
Section titled “Outputs”This node has no output port. It plugs into a slot on another node instead of passing items along.
- Model slot of Q&A Agent
The Q&A Agent produces the output: answer (the quoted text), score (how sure the model is; higher is better) and source (the document it came from).
Dependencies and credentials
Section titled “Dependencies and credentials”None. This node needs no connection and has no slots for other nodes.
Example workflow
Section titled “Example workflow”- Fill a knowledge base with your product manuals on the knowledge bases page.
- A Context Menu trigger on selected text (the question).
- Q&A Agent with Prompt
{{ $input.selectionText }}, with Local Q&A Model in its Model slot and Local Knowledge (your manuals) in its Knowledge slot. - Browser Notification with Message
{{ $input.answer }} ({{ $input.source }}).
Common issues
Section titled “Common issues”- “The Local Q&A Model answers by quoting a passage, so it needs a context: connect a Knowledge to the Q&A Agent.” Connect a Local Knowledge to the agent.
- “Choose an on-device Q&A model in the Local Q&A Model node.” Model is empty. Pick a model.
- “Model “…” isn’t available. Install it from Local AI.“ Install it from Local Models, or download it from the picker.
- The answer is empty. None of the passages found contained an answer (the run log shows a warning). Raise Knowledge documents (k), rephrase the question, or check that the knowledge base holds the information.
Related nodes
Section titled “Related nodes”- Q&A Agent: the node that accepts it.
- Local Knowledge: the knowledge base it quotes from.
- Answer Question: on-device Q&A over a passage you provide.