Wip/precomputed segments for flags by danoswaltCL · Pull Request #3198 · CarnegieLearningWeb/UpGrade

danoswaltCL · 2026-06-26T21:04:15Z

requirements largely found here: #3191

note that this is just for the flags at this moment, as its somewhat more straightforward, the main mechanics will be the same for both.

This creates a new table for precomputed segments, which will get updated on segment list changes.
getKeys() will now do very little to snag flags.

I have this going to the old flow still in the event of some error finding the precomputed segment, which hopefully would be really hard to do.

…flag include lists

…d-segments-for-flags

Copilot

Pull request overview

Implements “precomputed segment lists” for feature flags to avoid recursive segment resolution on the /featureflag read path by persisting flattened inclusion/exclusion member IDs and doing in-memory lookups.

Changes:

Added precomputed_segment storage (migration + entity + repository) and a PrecomputedSegmentService to compute/cache flattened segment member IDs.
Wired recompute/backfill triggers into segment + feature flag write paths, and added startup backfill.
Updated FeatureFlagService.getKeys to use a fast-path that pulls precomputed rows and filters via Set.has() checks (with fallback to legacy on-the-fly resolution when a row is missing).

Reviewed changes

Copilot reviewed 21 out of 22 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
packages/types/src/Experiment/enums.ts	Adds cache prefix for precomputed segment caching.
packages/backend/src/database/migrations/1779500000000-precomputedSegment.ts	Creates `precomputed_segment` table keyed by feature flag id.
packages/backend/src/api/models/PrecomputedSegment.ts	New TypeORM entity for precomputed flag segment members.
packages/backend/src/api/repositories/PrecomputedSegmentRepository.ts	Upsert + batch fetch helpers for precomputed rows.
packages/backend/src/api/services/PrecomputedSegmentService.ts	Flattens segment graphs into ID arrays; caching; backfill; recompute scheduling.
packages/backend/src/api/services/FeatureFlagService.ts	Fast-path filtering using precomputed rows; recompute/seed triggers.
packages/backend/src/api/services/SegmentService.ts	Schedules recomputes on segment mutations; recompute-after-delete logic.
packages/backend/src/api/repositories/SegmentRepository.ts	Adds parent-segment lookup for ancestor recompute walks.
packages/backend/src/api/repositories/FeatureFlagRepository.ts	Adds minimal projection query for `/featureflag` keys path.
packages/backend/src/init/seed/backfillPrecomputedSegments.ts	Startup backfill for missing precomputed flag rows.
packages/backend/src/app.ts	Wires startup backfill into boot sequence.
packages/backend/test/unit/services/*.test.ts	Adds/updates unit tests for precomputed behavior + triggers.
packages/backend/CLAUDE.md	Documents the precomputed flag segment design + triggers.
clientlibs/js/src/*/.spec.ts	Formatting-only test changes.
.gitignore	Ignores `.claude*`.
.claude/plans/precomputed-segments-experiments.md	Planning doc for future experiment parity.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…smaller improvements

Copilot

Pull request overview

Copilot reviewed 21 out of 22 changed files in this pull request and generated 7 comments.

+    for (const segment of segments) {
+      segment.individualForSegment.forEach((ind) => ids.push(ind.userId));
+      segment.groupForSegment.forEach((grp) => ids.push(grp.groupId));
+      segment.subSegments.forEach((sub) => {


+    // Flatten all group IDs from the user's group map (type is ignored per design decision)
+    const userGroupIds: string[] = experimentUser.group ? Object.values(experimentUser.group).flat() : [];
+


+      // Recompute precomputed sets for each affected flag after the transaction commits
+      const affectedFlagIds = [...new Set(listsInput.map((l) => l.id))];
+      await Promise.all(
+        affectedFlagIds.map((flagId) => this.featureFlagPrecomputedSegmentService.recomputeForFlag(flagId, logger))
+      );


+    await this.featureFlagPrecomputedSegmentService.recomputeForFlag(listInput.id, logger);
+


+    if (flagId) {
+      await this.featureFlagPrecomputedSegmentService.recomputeForFlag(flagId, logger);
+    }


Copilot · 2026-07-02T19:29:52Z

+## Precomputed Segment Lists (Feature Flags)
+
+Segment inclusion/exclusion for **feature flags** is precomputed and stored flat in the `precomputed_segment` table rather than resolved on-the-fly at assignment time.
+
+### How it works
+
+- **`PrecomputedSegment` entity** (`src/api/models/PrecomputedSegment.ts`) — one row per feature flag, columns: `featureFlagId` (PK), `inclusionIds: text[]`, `exclusionIds: text[]`. FK to `feature_flag` with `onDelete: CASCADE`.
+- **`PrecomputedSegmentService`** (`src/api/services/PrecomputedSegmentService.ts`) — owns all computation and cache logic:
+  - `recomputeForFlag(flagId)` — flattens all enabled inclusion/exclusion segments (recursive sub-segments) into flat ID arrays and upserts the row.
+  - `scheduleRecomputeForSegment(segmentId)` — fire-and-forget; finds all flags referencing a segment (and its parents) and calls `recomputeForFlag` for each.
+  - `getAffectedFlagIds(segmentId)` — public helper that returns flag IDs affected by a given segment (used before deletion).
+  - `getPrecomputedSets(flagIds[])` — cache-wrapped batch fetch, returns a `Map<flagId, PrecomputedSegment>`.
+  - `backfillMissingFlags(logger)` — called at startup; computes rows only for flags that have none yet (no-op once all flags are populated).
+  - `recomputeAllFlags(logger)` — full refresh of every flag; not called automatically, available for manual recovery.
+- **Assignment read path** — `FeatureFlagService.featureFlagLevelInclusionExclusion()` calls `getPrecomputedSets()` and does in-memory `Set.has()` checks. No recursive segment queries at assignment time.
+- **Cache** — keyed by `CACHE_PREFIX.PRECOMPUTED_SEGMENT_KEY_PREFIX + flagId`. Invalidated by `recomputeForFlag`.


Fixed in the latest commit — updated all references in CLAUDE.md from precomputed_segment/PrecomputedSegmentService/PrecomputedSegment to feature_flag_precomputed_segment/FeatureFlagPrecomputedSegmentService/FeatureFlagPrecomputedSegment.

Group IDs in feature_flag_precomputed_segment were stored bare in the same flat arrays as individual user IDs, so a group ID could collide with a user ID and flip an include/exclude decision. Matching also ignored group type, diverging from the type-aware experiment / on-the-fly resolution path. Namespace group IDs as `type:groupId` (individuals stay bare) via a shared precomputedGroupKey helper used by both the write and read paths. Add a migration that clears the table so the startup backfill rebuilds every row in the new format; missing rows fall back to on-the-fly resolution in the interim. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

danoswaltCL · 2026-07-02T19:29:46Z

@copilot please review 8193311, which addresses a couple comments about grouptype information

Copilot · 2026-07-02T19:35:26Z

@copilot please review 8193311, which addresses a couple comments about grouptype informati...

Reviewed 8193311. The implementation correctly addresses both issues:

Type-aware matching: precomputedGroupKey(type, groupId) namespaces group IDs as type:groupId on both the write path (FeatureFlagPrecomputedSegmentService) and read path (FeatureFlagService), matching the GroupForSegment composite key (groupId, type) semantics.
Group/user ID collision prevention: Groups stored as schoolId:user123 vs individuals stored bare as user123 — the individual Set.has(experimentUser.id) check can never accidentally match a group entry.
Migration: Clears stale bare-format rows so the startup backfill rebuilds them correctly; missing rows safely fall back to on-the-fly resolution.

All 49 unit tests pass, including the two new type-aware and collision tests. No issues found.

danoswaltCL added 5 commits June 25, 2026 13:55

feature flags complete with plan for extending to experiments as well

cf2683c

handle a bug with toggling enable/disable on a large list in feature-…

3925f2f

…flag include lists

Merge remote-tracking branch 'origin/release/6.5' into wip/precompute…

c5f5dc0

…d-segments-for-flags

fixes seeding blank row and closing gaps for stale data

957a127

test updates for latest changes

833846d

danoswaltCL requested a review from Copilot June 26, 2026 21:04

Copilot started reviewing on behalf of danoswaltCL June 26, 2026 21:04 View session

Copilot AI reviewed Jun 26, 2026

View reviewed changes

bcb37 reviewed Jul 1, 2026

View reviewed changes

Comment thread packages/types/src/Experiment/enums.ts Outdated

bcb37 reviewed Jul 1, 2026

View reviewed changes

Comment thread packages/backend/src/api/models/FeatureFlagPrecomputedSegment.ts

bcb37 reviewed Jul 1, 2026

View reviewed changes

Comment thread packages/backend/src/api/models/FeatureFlagPrecomputedSegment.ts

danoswaltCL added 2 commits July 1, 2026 13:24

name change of precomputed segment table for feature flags and a few …

e0bb14b

…smaller improvements

make linter happy

eeab870

danoswaltCL requested a review from Copilot July 2, 2026 17:49

Copilot started reviewing on behalf of danoswaltCL July 2, 2026 17:49 View session

Copilot AI reviewed Jul 2, 2026

View reviewed changes

danoswaltCL and others added 2 commits July 2, 2026 15:25

better error log

b4a56f9

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot started work on behalf of danoswaltCL July 2, 2026 19:28 View session

Copilot finished work on behalf of danoswaltCL July 2, 2026 19:30

Copilot started work on behalf of danoswaltCL July 2, 2026 19:30 View session

Copilot finished work on behalf of danoswaltCL July 2, 2026 19:35

fix: update CLAUDE.md to use correct FeatureFlagPrecomputedSegment names

aded2f0

danoswaltCL force-pushed the wip/precomputed-segments-for-flags branch from fcfe52f to aded2f0 Compare July 2, 2026 22:15

make linter happy

26f23d1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Wip/precomputed segments for flags#3198

Wip/precomputed segments for flags#3198
danoswaltCL wants to merge 11 commits into
release/6.5from
wip/precomputed-segments-for-flags

danoswaltCL commented Jun 26, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jul 2, 2026

Uh oh!

Uh oh!

danoswaltCL commented Jul 2, 2026

Uh oh!

Copilot AI commented Jul 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		// Flatten all group IDs from the user's group map (type is ignored per design decision)
		const userGroupIds: string[] = experimentUser.group ? Object.values(experimentUser.group).flat() : [];

		await this.featureFlagPrecomputedSegmentService.recomputeForFlag(listInput.id, logger);

Uh oh!

Conversation

danoswaltCL commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

danoswaltCL commented Jul 2, 2026

Uh oh!

Copilot AI commented Jul 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

danoswaltCL commented Jun 26, 2026 •

edited

Loading