Fixed-Size Text Chunking with Overlap

Fixed-Size Text Chunking with Overlap

Splitting long documents into smaller, overlapping chunks is a foundational step in building Retrieval-Augmented Generation (RAG) applications.

Tasks

Implement the function fixed_size_chunking(text, chunk_size, overlap_size) to:

1. Slice the input string text into substrings of length chunk_size characters.
2. Slide the window forward by a step size equal to chunk_size - overlap_size.
3. Ensure that if overlap_size is greater than or equal to chunk_size, a ValueError is raised.
4. Ensure that if chunk_size is less than or equal to 0, a ValueError is raised.
5. Handle empty strings by returning an empty list [].

Example

fixed_size_chunking("abcdefgh", 4, 2)
# Returns: ["abcd", "cdef", "efgh"]