[Doc] Memory-efficient RL training tutorial + cross-refs

[vmoens]

Summary

Ties together the three recently-merged memory-efficiency PRs into a
single story:

• compact_obs collector flag ([Performance] Add compact_obs flag to DataCollector #3742)
• NextStateReconstructor RB transform ([Feature] NextStateReconstructor RB transform #3743)
• NaN-safe value-estimator forward ([BugFix] Sanitize NaN ('next', obs) in value-estimator forward #3744)

Two parts:

1. Runnable Sphinx-gallery tutorial at
tutorials/sphinx-tutorials/memory_efficient_rl.py. Sections:

• Where the observation memory goes (concrete td.bytes() numbers)
• Why ("next", obs) is kept by default — bootstrap target at
  trajectory ends, MultiStepTransform n-step fallback
• Knob 1 — SyncDataCollector(compact_obs=True)
• Knob 2 — NextStateReconstructor with the traj_id + done contract
• Knob 2.5 — value-estimator NaN safety
  (_sanitize_next_obs_nan), GAE finite everywhere
• When not to take this path — MultiStepTransform incompatibility,
  the V(obs[t]) ≈ V(real_next_obs) approximation at truncated steps,
  and how shifted=True interacts
• Knob 3 — LazyMemmapStorage for buffers ≥ VRAM
• Knob 4 — SliceSampler + the new "scan" / "triton"
  recurrent backends for padding-free sequence training
• End-to-end pipeline snippet
• Conclusion + Further reading

Runs end-to-end on CPU (CartPole-v1, 200 frames; <2s wall) and reports
the byte-level savings concretely from td.bytes().

2. Docstring cross-references so a reader landing on any of the
three new APIs finds the other two:

• Collector(compact_obs=…) (and the multi-process collectors):
  pointers to NextStateReconstructor, the value-estimator
  sanitizer, the MultiStepTransform incompatibility note, and the
  new tutorial.
• NextStateReconstructor: .. seealso:: block covering
  compact_obs, the sanitizer, MultiStepTransform, and the
  tutorial.
• ValueEstimatorBase._sanitize_next_obs_nan: .. seealso::
  block to compact_obs, NextStateReconstructor, and the
  tutorial.

docs/source/index.rst registers the new tutorial under "Basics".

Test plan

• Tutorial runs end-to-end with concrete output (memory savings
  reported, NaN at slice boundaries confirmed to coincide with
  trajectory boundaries, GAE advantage finite everywhere, memmap
  roundtrip works).
• All cross-references resolve to existing public symbols
  (verified by reading the rendered class docstrings via
  Collector.__doc__ etc.).

🤖 Generated with Claude Code

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3745

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Run pull request jobs on OSDC runners in shadow mode

❌ 1 New Failure

As of commit 01502ba with merge base 6e46542 ():

NEW FAILURE - The following job has failed:

• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh)
  test/transforms/test_observation_transforms.py::TestNextObservationDelta::test_trans_parallel_env_check

This comment was automatically generated by Dr. CI and updates every 15 minutes.