Skip to content

feat(workflows): add dry-run previews - #3933

Open
VictorMaxWang wants to merge 1 commit into
github:mainfrom
VictorMaxWang:fix/2661-workflow-dry-run
Open

feat(workflows): add dry-run previews#3933
VictorMaxWang wants to merge 1 commit into
github:mainfrom
VictorMaxWang:fix/2661-workflow-dry-run

Conversation

@VictorMaxWang

Copy link
Copy Markdown

Description

Add specify workflow run --dry-run so contributors can inspect resolved, agent-facing workflow configuration before allowing an integration CLI to run.

The dry-run mode:

  • resolves and previews built-in command, prompt, and gate steps without agent dispatch or interactive gate input
  • persists across workflow resume
  • emits ordered structured previews in human and JSON output, including previews completed before a later failure
  • keeps loop and concurrent fan-out preview ordering deterministic
  • removes stale nested previews when a resumed run takes a different branch

Dry-run is deliberately not a side-effect-free sandbox. Built-in shell and init steps, along with custom step types, retain their normal behavior; the CLI and documentation warn about this before execution.

Closes #2661

Testing

  • Tested locally with uv run specify --help
  • Ran existing tests with uv sync && uv run pytest
  • Tested with a sample project (if applicable)

Additional validation performed on Windows:

  • uv sync --extra test
  • uv pip install -e ".[test]"
  • uv pip install -e .
  • affected workflow, CLI, engine, state, project-outside-run, and agent consistency tests: 239 passed, 3 skipped
  • tests/test_workflows.py: 910 passed, 7 skipped; 20 tests could not create their symlink fixtures because this Windows session lacks symlink privilege (WinError 1314)
  • Ruff 0.15.0 and git diff --check
  • disposable Codex-integration sample project: human and JSON dry-runs completed with six ordered previews; an invocation sentinel confirmed that the integration executable was not called, state persisted dry_run: true, and no spec artifact was created

The monolithic .venv\Scripts\python -m pytest tests -q command exceeded the local runner time limit. The final commit was therefore also exercised through disjoint test-directory partitions: 6,103 passed, 178 skipped, and 130 Windows symlink/locale/MSYS parity or scope-disjoint failures. These failures were not baseline-confirmed against the parent commit, so this PR does not claim a fully green local suite.

The contributor confirmed completing line-by-line review and hands-on validation before this PR was opened.

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance (describe below)

OpenAI Codex was used to inspect repository guidance and Issue/PR history, implement the workflow dry-run behavior, documentation, and regression tests, run automated tests, execute the disposable Codex-integration sentinel E2E, rebase onto current upstream, and prepare this pull request. The contributor subsequently performed the required human review and validation.

Preview agent-facing workflow steps without dispatch, persist the mode across resume, and expose ordered partial previews in human and JSON output.

Assisted-by: OpenAI Codex (model: GPT-5, autonomous)
@VictorMaxWang
VictorMaxWang requested a review from mnriem as a code owner August 1, 2026 04:34
Copilot AI review requested due to automatic review settings August 1, 2026 04:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Add dry-run flag to preview spec output without AI invocation

2 participants