Skip to content

BE-614: HashQL: Generate synthetic closure bodies for intrinsics used as first-class values#8895

Open
indietyp wants to merge 3 commits into
bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closurefrom
bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value
Open

BE-614: HashQL: Generate synthetic closure bodies for intrinsics used as first-class values#8895
indietyp wants to merge 3 commits into
bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closurefrom
bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value

Conversation

@indietyp

Copy link
Copy Markdown
Member

🌟 What is the purpose of this PR?

Allows intrinsics to be used in value position (passed as arguments, bound to variables) by generating synthetic MIR wrapper bodies during reification. Previously, intrinsics like <= could only appear at call sites where specialization rewrites them. Now they can flow through higher-order functions:

let apply = fn (a, b, f) = (f a b) in
(apply 2 3 <=)

🔍 What does this change?

Synthetic body generation:

  • Synthetics struct in CrossCompileState manages creation and caching of wrapper bodies
  • SyntheticBuilder generates MIR bodies with fat closure ABI (unit env as first parameter)
  • T! macro for matching qualified paths against ConstantSymbol arrays: &T![::core::cmp::gt] in slice patterns, sym::path::core::cmp::gt::CONST for the name
  • Dispatch table covers all first-classable intrinsics (comparison, boolean, arithmetic, bitwise) with local binary!/unary! macros deriving path and name from the same segments

Thin-call specialization:

  • rvalue_call_thin_specialize recognizes Call(Thin, Qualified(intrinsic), []) (the administrative thunk-force from the thunking phase) and produces the closure aggregate directly, skipping thunk body generation
  • Type unwrapping handles the thunking phase's () -> ClosureType wrapper to extract the actual signature

Intrinsic classification:

  • First-classable: comparison (gt, lt, gte, lte, eq, ne), boolean (and, or, not), arithmetic (add, sub), bitwise (and, or, not)
  • Unconstructible MIR ops (BinOp variants with ! payload): mul, div, rem, mod, pow, xor, shl, shr
  • No MIR ops: sqrt, cbrt, root
  • Not first-classable (syntactic forms): graph::head::entities, graph::body::filter, graph::tail::collect produce a specific user-facing diagnostic

Diagnostics:

  • intrinsic_not_first_class: user-facing error for graph intrinsics in value position
  • synthetic_binary_arity_mismatch / synthetic_unary_arity_mismatch: ICE for monomorphized type invariant violations

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

⚠️ Known issues

  • Bare intrinsic at module root ("<=" as the entire program) is not supported. The thunking phase does not wrap the root body when it is a variable, so it never reaches the synthetic machinery. Tracked with a run: skip test.

🐾 Next steps

  • Untyped property access (the original motivation for this work)
  • Module system support for root-level qualified paths (BE-67)

🛡 What tests cover this?

Compiletests (8 new):

  • reify/synthetic-intrinsic-value: basic synthetic body generation and thin-call specialization
  • reify/synthetic-intrinsic-reused: same intrinsic used twice shares one body (caching)
  • reify/synthetic-intrinsic-multiple: different intrinsics get separate bodies
  • reify/synthetic-graph-not-first-class: graph intrinsic in value position produces error
  • reify/synthetic-intrinsic-bare: bare intrinsic at root (run: skip)
  • interpret/synthetic-intrinsic-value: end-to-end correctness (evaluates to true)
  • post_inline/synthetic-intrinsic-hof: full optimization cascade with constants (collapses to return true)
  • post_inline/synthetic-intrinsic-hof-dynamic: HOF with dynamic inputs collapses to bare x <= y

❓ How to test this?

cargo run --package hashql-compiletest -- run --filter "test(synthetic)"

📹 Demo

The post_inline/synthetic-intrinsic-hof-dynamic test shows the full cascade. A higher-order function receiving <= as a value with dynamic inputs:

Initial:  construct closure for <=, pass to apply, call through fat pointer
Final:    %0 = input LOAD x
          %1 = input LOAD y
          %2 = %0 <= %1
          return %2

All indirection eliminated. Indistinguishable from writing ["<=", x, y] directly.

Copilot AI review requested due to automatic review settings June 22, 2026 16:43
@vercel

vercel Bot commented Jun 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment Jul 3, 2026 4:36pm
2 Skipped Deployments
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Jul 3, 2026 4:36pm
petrinaut Skipped Skipped Jul 3, 2026 4:36pm

@vercel vercel Bot temporarily deployed to Preview – petrinaut June 22, 2026 16:43 Inactive
@indietyp indietyp changed the title BE-614: HashQL MIR: Generate synthetic closure bodies for intrinsics used as first-class values BE-614: HashQL: Generate synthetic closure bodies for intrinsics used as first-class values Jun 22, 2026
@cursor

cursor Bot commented Jun 22, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches the MIR reification and calling-convention path for qualified names; behavior changes for intrinsics passed as values, though scoped to compiler internals with broad compiletest coverage.

Overview
Enables intrinsics in value position (e.g. passing <= into a higher-order function) by generating cached synthetic MIR bodies during reification instead of treating qualified paths as unsupported modules.

Reification: A new Synthetics cache on CrossCompileState maps known ::core::… paths to fat-closure bodies (Source::Synthetic) plus optional thunk wrappers. Qualified variables in operand resolve through this path to FnPtr thunks; thin calls with no args are specialized in rvalue_call_thin_specialize to closure aggregates without an extra thunk round-trip. Graph pipeline intrinsics (head::entities, etc.) get intrinsic_not_first_class instead of a generic external-module error.

Supporting tweaks: ConstantSymbolSymbol conversion, lhs/rhs symbols for synthetic locals, scratch.reset() after MIR reify in graph compile, and compiletests covering reuse, multiple intrinsics, HOF + inlining, and graph errors.

Reviewed by Cursor Bugbot for commit 0694d8d. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown
Member Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 48906c1. Configure here.

Comment thread libs/@local/hashql/mir/src/reify/synthetic.rs
@codecov

codecov Bot commented Jun 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 3.62%. Comparing base (489dd1a) to head (0694d8d).

Additional details and impacted files
@@                                         Coverage Diff                                          @@
##           bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure   #8895      +/-   ##
====================================================================================================
+ Coverage                                                               2.34%   3.62%   +1.27%     
====================================================================================================
  Files                                                                    156     478     +322     
  Lines                                                                   3962   14463   +10501     
  Branches                                                                 918    3020    +2102     
====================================================================================================
+ Hits                                                                      93     524     +431     
- Misses                                                                  3868   13894   +10026     
- Partials                                                                   1      45      +44     
Flag Coverage Δ
apps.hash-ai-worker-ts 1.39% <ø> (?)
apps.hash-api 6.39% <ø> (?)
local.hash-backend-utils 2.81% <ø> (ø)
local.hash-graph-sdk 10.00% <ø> (ø)
local.hash-isomorphic-utils 0.18% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq

codspeed-hq Bot commented Jun 22, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 27.89%

❌ 1 regressed benchmark
✅ 55 untouched benchmarks
⏩ 24 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
pattern_match_constant 150.8 ns 209.2 ns -27.89%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value (0694d8d) with bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure (489dd1a)

Open in CodSpeed

Footnotes

  1. 24 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 4ad5098 to 74017c2 Compare July 3, 2026 11:39
@indietyp indietyp force-pushed the bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value branch from 7a2aad8 to 686b850 Compare July 3, 2026 11:39
@vercel vercel Bot temporarily deployed to Preview – petrinaut July 3, 2026 11:39 Inactive

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated no new comments.

@indietyp indietyp force-pushed the bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value branch from 686b850 to 538afac Compare July 3, 2026 14:27
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 74017c2 to 410fa1b Compare July 3, 2026 14:27
@vercel vercel Bot temporarily deployed to Preview – petrinaut July 3, 2026 14:27 Inactive
Copilot AI review requested due to automatic review settings July 3, 2026 14:28
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 410fa1b to ac8ed9e Compare July 3, 2026 14:28
@indietyp indietyp force-pushed the bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value branch from 538afac to 88472a7 Compare July 3, 2026 14:28
@vercel vercel Bot temporarily deployed to Preview – petrinaut July 3, 2026 14:28 Inactive

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.

Comment on lines +36 to +43
/// Constructs a `&'static [ConstantSymbol]` path from `::` separated segments.
///
/// Each segment maps to the corresponding `sym::` constant.
///
/// ```ignore
/// T![::core::math::add] // expands to &[sym::core, sym::math, sym::add]
/// T![::core::cmp::eq] // expands to &[sym::core, sym::cmp, sym::eq]
/// ```
@indietyp indietyp force-pushed the bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value branch from 88472a7 to d3b03b8 Compare July 3, 2026 14:36
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from ac8ed9e to 4aacfd8 Compare July 3, 2026 14:36
@vercel vercel Bot temporarily deployed to Preview – petrinaut July 3, 2026 14:36 Inactive
Copilot AI review requested due to automatic review settings July 3, 2026 15:21
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 4aacfd8 to f2b45cb Compare July 3, 2026 15:21
@indietyp indietyp force-pushed the bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value branch from d3b03b8 to 015b737 Compare July 3, 2026 15:21
@vercel vercel Bot temporarily deployed to Preview – petrinaut July 3, 2026 15:21 Inactive

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated no new comments.

@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from f2b45cb to 489dd1a Compare July 3, 2026 16:24
@indietyp indietyp force-pushed the bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value branch from 015b737 to 0694d8d Compare July 3, 2026 16:24
@vercel vercel Bot temporarily deployed to Preview – petrinaut July 3, 2026 16:25 Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

2 participants