expose include_orig_elements param in api

[tylorbayer]

TL;DR

Adds include_orig_elements as a new form parameter to control whether original elements are included in chunk metadata.

What changed?

A new include_orig_elements boolean parameter (defaulting to True) has been added to the API. When True, the elements used to form each chunk are attached to that chunk's .metadata.orig_elements as a gzipped+base64 blob. When set to False, these blobs are omitted from the response. The parameter is wired through GeneralFormParams, pipeline_api, and all relevant chunking call sites.

How to test?

Submit a document partition request with include_orig_elements=false in the form body and verify that the response chunks do not contain orig_elements in their metadata. Submit the same request with include_orig_elements=true (or omit the parameter entirely) and confirm that orig_elements is present in the chunk metadata as expected.

Why make this change?

For large documents — particularly those with large tables — the orig_elements blob gets duplicated into every chunk, which can dramatically inflate the response payload size. Giving callers the option to opt out of this behavior allows them to receive significantly smaller responses when the original element data is not needed.

[tylorbayer]

• expose include_orig_elements param in api #1 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.