feat(cli): unify quant/optimize/analyze/max-optim flags across build, config, perf, eval#923
Merged
Merged
Conversation
… config, perf, eval Add shared Click option decorators in utils/cli.py so the build-pipeline toggles share one definition and identical semantics across commands: - quant_option() -> --quant/--no-quant, with --quantize/--no-quantize kept as an alias (perf previously used --quantize; existing invocations keep working) - optimize_option() -> --optimize/--no-optimize - analyze_option() -> --analyze/--no-analyze - max_optim_iterations_option() -> --max-optim-iterations - build_pipeline_extra_kwargs() centralizes the (optimize, analyze, max_optim_iterations) -> (skip_optimize, hack_max_optim_iterations) mapping shared by build/perf/eval All four commands now expose the full set: - build: refactored to use the shared helpers (behavior unchanged) - perf: --quantize renamed to --quant (alias retained); optimize/analyze/ max-optim added and plumbed through BenchmarkConfig and the per-module build path - eval: quant/optimize/analyze/max-optim added to WinMLEvaluationConfig and applied in _load_model - config: flags added for CLI parity; the build-only flags warn when passed since config only emits a static config WinMLAutoModel.from_pretrained now forwards skip_optimize / hack_max_optim_iterations to build_hf_model so the HF build path honors these flags (from_onnx already forwarded them).
`winml config` only emits a static WinMLBuildConfig and never runs the build pipeline, so --optimize/--analyze/--max-optim-iterations had no effect there. Remove them (and the accompanying "ignored" warning) so config exposes only the flags that change its output: --quant/--no-quant and --compile/--no-compile. The other three remain on build/perf/eval, which do run the pipeline.
DingmaomaoBJTU
approved these changes
Jun 23, 2026
DingmaomaoBJTU
left a comment
DingmaomaoBJTU
left a comment
Collaborator
There was a problem hiding this comment.
Overall the PR is well-structured and correct. build_pipeline_extra_kwargs is a clean centralization that eliminates the per-command duplication and makes the flag→kwarg mapping testable. The --quantize alias on perf and all the new eval wiring looks right. All 175 affected unit tests pass. Two minor nits follow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Unifies the five build-pipeline CLI flags —
--quant/--no-quant,--optimize/--no-optimize,--analyze/--no-analyze, and--max-optim-iterations— acrossbuild,config,perf, andevalso they share one definition and identical semantics, defined inutils/cli.py.Shared helpers (
utils/cli.py)quant_option()→--quant/--no-quant, with--quantize/--no-quantizekept as an alias so existingperfinvocations keep working.optimize_option()→--optimize/--no-optimizeanalyze_option()→--analyze/--no-analyzemax_optim_iterations_option()→--max-optim-iterationsbuild_pipeline_extra_kwargs()centralizes the(optimize, analyze, max_optim_iterations)→(skip_optimize, hack_max_optim_iterations)mapping shared by build/perf/eval.Per-command changes
buildperf--quantize→ canonical--quant(alias retained);optimize/analyze/max-optimadded and plumbed throughBenchmarkConfig, the single-model load path, and the per-module build path.evalquant/optimize/analyze/max-optimadded toWinMLEvaluationConfigand applied in_load_model.configconfigonly emits a static config.Plumbing fix
WinMLAutoModel.from_pretrainednow forwardsskip_optimize/hack_max_optim_iterationstobuild_hf_modelso the HF build path honors these flags (from_onnxalready forwarded them).Testing
tests/unit/utils/test_cli.py— new coverage for the four option helpers (incl. the--quantizealias) andbuild_pipeline_extra_kwargs.tests/unit/commands/test_boolean_flag_pairs.py— updated to the unified surface (canonical--quant+ alias, new flags on config/perf/eval).tests/unit/eval/test_eval.py— updated_load_modelcall assertion + new test that--no-quant/--no-optimize/--max-optim-iterationsreachfrom_pretrained.tests/unit/commands,tests/unit/eval,tests/unit/utils/test_cli.py,tests/unit/optracing,tests/regression/test_design_gaps.py— all green.