Recursive Character Text Splitter

Tell the Indexer how to cut long text into chunks, trying a list of separators in order.

What it does

Recursive Character Text Splitter is not a step on its own: it has no output port and never runs by itself. You plug it into the Text Splitter slot of the Indexer, which uses it to cut its Input into chunks before saving them to a knowledge base.

It splits the text at the first separator in Separators that appears in it. Any piece still longer than Chunk Size characters is split again with the next separator, and so on. Small pieces are then joined back up to about Chunk Size characters, with Chunk Overlap characters repeated between neighbouring chunks so a sentence cut at the border is not lost.

When Separators is empty, the only separator is a line break. A piece of text with no line break in it, longer than Chunk Size, then stays one oversized chunk.

When to use it

• Index articles, notes or page text where line breaks mark paragraphs. This is the usual choice for the Indexer.
• Index Markdown, HTML or source code: pick its Language so chunks break at headings, tags or functions.
• For one fixed separator and no retries, use Character Text Splitter.

Inputs and settings

Setting	Type	What it does
Chunk Size	Number	The size of each chunk. Default: 1000
Chunk Overlap	Number	The overlap between chunks. Default: 200
Separators	Tags	The separators to use.
Language	Choice	The language of the text. 16 options.

• Chunk Size and Chunk Overlap count characters. Keep the overlap smaller than the chunk size. A value of 0 in either field is replaced by its default (1000 and 200).
• Separators are tried in the order you add them. Add a full stop . after the line break to split long paragraphs into sentences.
• Language has separators built in for markdown, html, latex, rst and programming languages such as python or js. When a language is set, Separators is ignored.

Outputs

This node has no output port. It plugs into a slot on another node instead of passing items along.

• Text Splitter slot of Indexer

The chunks are counted in the Indexer’s output (chunks) and stored in the knowledge base connected to it.

Dependencies and credentials

None. This node needs no connection and has no slots for other nodes.

Example workflow

1. Schedule runs every Monday.
2. URL Document Loader with URL https://example.com/docs/getting-started and Output field name text.
3. Indexer with Input {{ $input.text }} and Source link https://example.com/docs/getting-started. Connect Local Knowledge to its Knowledge slot.
4. Recursive Character Text Splitter connected to the Indexer’s Text Splitter slot, with Chunk Size 800 and Chunk Overlap 100.

Common issues

• “Cannot have chunkOverlap >= chunkSize”. Chunk Overlap is as large as Chunk Size or larger. Lower the overlap.
• The Indexer reports chunks: 1 for a long text. The text has no line breaks. Add . to Separators, or set a Language.
• You cannot turn overlap off. 0 falls back to 200. Use 1 for almost no overlap.
• “Text splitter not found” on the Indexer. This node is not set up correctly. Reopen it and check its values.

Related nodes

• Indexer: the node this splitter plugs into.
• Character Text Splitter: split on a single separator.
• Local Knowledge: the knowledge base the chunks go to.
• URL Document Loader: get page text to index.