Skip to content

Fix C++ TsFile writer flush offsets for multi-GB files.#839

Merged
jt2594838 merged 6 commits into
apache:developfrom
hongzhi-gao:fix/cpp-4gb-tsfile
Jun 16, 2026
Merged

Fix C++ TsFile writer flush offsets for multi-GB files.#839
jt2594838 merged 6 commits into
apache:developfrom
hongzhi-gao:fix/cpp-4gb-tsfile

Conversation

@hongzhi-gao

@hongzhi-gao hongzhi-gao commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Fix flush_stream_to_file() so chunk/metadata offsets stay correct after repeated flushes on multi-GB files.
  • Reset ByteStream consumer state after flush; use int64_t for aligned chunk read offsets.
  • Add tree-model and table-model ~4GB gtest coverage; replace write-only multi-flush test with read-back regression.

Test plan

  • TsFileWriterTest.MultiFlushWriteAndRead
  • LargeFileTreeTest.LargeFile4GB_TreeWriteAndRead
  • LargeFileTableTest.LargeFile4GB_TableWriteAndRead

Sync file_base_offset after flush, reset the write stream and consumer,
and add tree/table 4GB regression tests.
Mirror C++ tree/table ~4GB regression coverage in pytest and drop POSIX-only headers from large-file gtests.
Keep shared constants and utilities in one pytest entry point for tree/table 4GB tests.
@jt2594838 jt2594838 merged commit 9e929f0 into apache:develop Jun 16, 2026
29 checks passed
@jt2594838 jt2594838 deleted the fix/cpp-4gb-tsfile branch June 16, 2026 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants