Skip to content

Markdown import: Limit the source hosts, and escape the Learn image-link rewrite - #751

Closed
obenland wants to merge 12 commits into
WordPress:trunkfrom
obenland:wporg-cli/markdown-import-source-hosts
Closed

Markdown import: Limit the source hosts, and escape the Learn image-link rewrite#751
obenland wants to merge 12 commits into
WordPress:trunkfrom
obenland:wporg-cli/markdown-import-source-hosts

Conversation

@obenland

@obenland obenland commented Jul 30, 2026

Copy link
Copy Markdown
Member

Two plugins let an editor supply a URL that the site later fetches or renders, without constraining where it points.

WP-CLI (wporg-cli)

Handbook pages on make.wordpress.org/cli are imported from Markdown in the wp-cli GitHub organization, but the "Markdown source" field accepted any http(s) URL — and the fetched body is written straight into the post as content.

A new validate_markdown_source() limits sources to github.com and raw.githubusercontent.com, requires an http or https scheme, and requires the repository owner — the first path segment on both hosts — to be wp-cli. The hosts alone would still have served every repository and gist on GitHub; the handbook comes from one organization, and all 148 entries in the published manifest are github.com/wp-cli/…. Hosts and owners are both compared case insensitively, since GitHub serves them that way. It runs in three places:

  • On save, along with the edit_post capability check action_save_post() was missing — it relied on the nonce alone, which is user-bound but is not a capability check. A rejected source leaves the stored one in place and reports it in an admin notice, since the prefilled field posts the stored value back on every save.
  • On fetch, because the 15-minute wporg_cli_markdown_import cron reuses stored values as-is, so a save-time check alone would not cover existing rows.
  • On manifest import, before the post is inserted. Otherwise a bad entry leaves behind a handbook page that every scheduled run then fails on. Skipped entries emit a WP_CLI::warning so a manifest regression stays visible.

Three smaller changes:

  • wp_safe_remote_get() replaces wp_remote_get() for both the source and the manifest, so redirect targets are validated rather than followed anywhere. The manifest URL is a hardcoded constant, so that half closes nothing that was reachable; it just leaves no unvalidated request in either importer.
  • The blob → raw URL rewrite is now case insensitive, matching how the host is validated. A mixed case blob URL passed validation but was left unrewritten, so GitHub's HTML page was fetched and parsed as Markdown into the post.
  • $_POST is guarded against a non-string value, which would otherwise reach esc_url_raw() and fatal on PHP 8.

Learn (wporg-learn)

wporg-learn carries a near-verbatim copy of the same importer, but every one of its hooks has been commented out since 2020-08-12. One function is still live:

add_filter( 'the_content', array( 'WPOrg_Learn\Markdown_Import', 'replace_image_links' ) );

It concatenated the wporg_learn_markdown_source post meta straight into an <img src=""> attribute with no escaping. That meta key is never registered, so it has no sanitize or auth callback, and it carries no underscore prefix, leaving it editable as free text through the custom fields metabox the lesson-plan post type supports. A value containing a quote breaks out of the attribute: stored XSS, injectable by anyone who can edit a lesson plan.

The value is now escaped with esc_url(). It is deliberately not checked against the host allow-list — the hosts present in existing stored data aren't known from this repository, and dropping a legitimate one would break lesson plan images.

The remaining changes mirror the WP-CLI plugin so the importer is safe if it is ever re-activated, as the file's own comment anticipates. All of that is unreachable today.

Tests

Adds a PHPUnit suite for wporg-cli, following the layout the handbook plugin uses. It runs in the existing make environment, which already serves the other make.wordpress.org plugins, rather than in one of its own — wporg-cli runs make.wordpress.org/cli, and a second environment for the same site would only have duplicated it. The environment gains the handbook plugin, which registers the post type the importer operates on, and Jetpack, which supplies the WPCom_GHF_Markdown_Parser the import hands its Markdown to, so the importer can also be exercised by hand. Neither is a test dependency: the suite's bootstrap registers a stand-in post type and stubs the parser.

cd environments && npm run make:test      # all three suites in one environment
cd environments && npm run make:test:cli  # this plugin's suite alone

make:test starts the environment once and runs every suite it holds, so npm run now describes the same thing CI does. The workflow calls these scripts instead of reimplementing them: one matrix entry per environment, and the redundant polyfill install step is gone, since each environment's afterStart already installs PHPUnit and the polyfills. That also adds the CI coverage this suite was missing — it had npm scripts but no matrix entry, so it had never run on a pull request.

66 tests cover the accepted hosts and owners and the rejected ones (loopback, private ranges, link-local, lookalike hostnames, userinfo-prefixed authorities, non-HTTP schemes, protocol-relative URLs, other owners, owner lookalikes, an owner-less URL); that blob URLs are fetched from the raw host whatever case the host was entered in; that a disallowed stored source is rejected with no outbound request; that a disallowed or absent manifest source creates no post; that a rejected save keeps the stored source; that slashes added to $_POST don't survive into the stored value; and that a user without edit_post can't change a source.

There are no tests for wporg-learn — it has no PHPUnit suite or wp-env environment today, and adding one was out of scope. That leaves the esc_url() fix, the only change here that affects live behaviour on Learn, without a regression test.

Both new PHP files pass phpcs with no errors or warnings and declare strict_types. The modified files carry pre-existing violations; this PR adds none to either (wporg-cli 15/20 and wporg-learn 1/3, unchanged before and after).

Handbook pages are imported from Markdown files in the wp-cli GitHub
organization, but the importer accepted any http(s) URL in the "Markdown
source" field.

Check the source against an allow-list of github.com and
raw.githubusercontent.com, both when it is saved and again when it is
fetched, since stored values are reused by the scheduled import. Switch the
fetch to wp_safe_remote_get(), and add the edit_post check that the save
handler was relying on the nonce for.

Adds a PHPUnit suite for the plugin, runnable via `npm run cli:test`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 30, 2026 19:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props obenland.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

obenland and others added 2 commits July 30, 2026 15:19
`create_post_from_manifest_doc()` stored whatever the manifest supplied. The
fetch-time check added in the previous commit covers the import itself, but a
manifest entry pointing elsewhere still created a handbook page that every
scheduled run then failed on. Checking before the insert skips the entry
instead, and warns through WP-CLI so a manifest regression is visible rather
than silently stopping page creation.

Also guards `$_POST` against a non-string value, which would otherwise reach
`esc_url_raw()` and fatal on PHP 8.

Tests cover both manifest paths, and seed a stored source before asserting a
rejected save clears it, so the assertion can no longer pass when
`action_save_post()` returns early.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…inks.

`replace_image_links()` concatenated the `wporg_learn_markdown_source` post
meta straight into an `<img src="">` attribute. The key is never registered,
so it has no sanitize or auth callback, and it carries no underscore prefix,
leaving it editable as free text through the custom fields metabox that the
`lesson-plan` post type supports. A value containing a quote could therefore
break out of the attribute and land script in the rendered content for every
front end visitor. Escaping it on output closes that.

The value is escaped rather than checked against the host allow-list below,
since the hosts in the existing stored data aren't known and dropping a
legitimate one would break lesson plan images.

The rest of the importer has been disabled since 2020, so the remaining
changes are unreachable today. They mirror the WP-CLI plugin so that the
importer is safe if it is ever re-activated, as the file comment anticipates:
an allow-list of the training team's GitHub hosts, checked when a source is
saved and again when it is fetched, an `edit_post` capability check that the
save handler was missing, and `wp_safe_remote_get()` in place of
`wp_remote_get()`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@obenland obenland changed the title WP-CLI: Limit Markdown imports to the GitHub source hosts Markdown import: Limit the source hosts, and escape the Learn image-link rewrite Jul 30, 2026
obenland and others added 4 commits July 30, 2026 15:45
Rewrite GitHub blob URLs to the raw host case insensitively, matching how the
host is validated. A mixed case blob URL passed validation but was left
unrewritten, so GitHub's HTML page was fetched and parsed as Markdown into the
post.

Require an http or https scheme. A protocol relative URL parses to an allowed
host but the HTTP API refuses to request it, leaving behind a source that every
scheduled import fails on.

Keep the stored source when a submitted one is rejected, and report it in an
admin notice. The field is prefilled with the stored source, so any save of a
post whose source predates the check posted it back and silently wiped it,
taking the front end edit link and the scheduled import with it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The importer accepts the GitHub host in any case and rewrites blob URLs to the
raw host case insensitively, but the front end edit link compared the host with
a case sensitive `!==`. A stored source of `https://GitHub.com/…/blob/main/…`
therefore imported correctly while its "Edit" link was left pointing at
GitHub's file view rather than the editor.

Mirror the same flag onto the wporg-learn rewrite, which is commented out but
would carry the original mismatch back if the import is ever re-enabled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…field.

`action_save_post()` only guards the plugin's own field, which the block editor
never renders. The meta key carries no underscore prefix and the handbook post
type supports custom fields, so the source was editable as free text through
that metabox, which saves through WordPress rather than through the importer.
A write over REST or from any other caller of `update_post_meta()` skipped the
check the same way.

Register the meta with the allow-list as its `sanitize_callback`, making that
the one point every write passes through. The fetch still re-checks, since
values stored before any of this predate both.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The notice can never fire. Its transient is written only by `action_save_post()`,
which returns early unless the plugin's own input field was posted, and that
field is printed on `edit_form_after_title` — a hook only the classic editor
fires. On a block editor site there is no field, so no rejected value to report.

The save keeps ignoring a disallowed source and leaving the stored one in place.
Every other route into the meta is now checked by its `sanitize_callback`, which
is equally silent, so nothing is lost by not reporting here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@obenland
obenland force-pushed the wporg-cli/markdown-import-source-hosts branch from 0e74bb2 to 37fd365 Compare July 30, 2026 22:16
obenland and others added 5 commits July 31, 2026 09:02
The manifest URL is a hardcoded constant in both plugins, so this closes nothing
that was reachable. It makes every outbound request either importer performs go
through `wp_http_validate_url()`, including each redirect hop, matching the
source fetch that already uses the safe helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The environment loaded `wporg-cli` alone, which owns neither the post type the
importer operates on nor the parser it hands the fetched Markdown to. Add the
handbook plugin, which registers the `handbook` post type, and Jetpack, which
supplies `WPCom_GHF_Markdown_Parser`, so the import can be exercised by hand
rather than only through the tests.

The PHPUnit suite depends on neither: its bootstrap registers a stand-in post
type and stubs the parser, so it still covers the plugin's own logic.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`wporg-cli` runs make.wordpress.org/cli, and the make environment already backs
two suites the same way CI runs them: one config, one container, a different
`--env-cwd` per plugin. A third environment for one more plugin on the same site
only duplicated it — its `afterStart` script installed the same PHPUnit 11 and
polyfills the make one does, with nothing of its own.

Fold the plugin in and drop `environments/cli`. The environment gains the
handbook plugin, which registers the post type the importer operates on, and
Jetpack, which supplies the `WPCom_GHF_Markdown_Parser` the import hands its
Markdown to, so the importer can be exercised by hand rather than only through
the tests. Neither is a test dependency: the suite's bootstrap registers a
stand-in post type and stubs the parser.

`make:test` now runs all three suites, with a target per plugin mirroring the CI
matrix, which gains the entry for `wporg-cli` it was missing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The workflow reimplemented what `environments/package.json` already describes:
one matrix entry per plugin, each starting its own environment and installing
the polyfills by hand. That is why three entries pointed at the make
environment, starting it three times to run one suite each.

Run the scripts instead. Each starts its environment, whose `afterStart` already
installs PHPUnit and the polyfills — the step doing it again was redundant — and
then runs every suite that environment holds, so the make entry covers its three
plugins in one job. `npm ci` replaces the global install, pinning wp-env to the
version in the lockfile, and CI runs what a developer runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The allow-list named the hosts the handbook is published from, but those hosts
serve every repository on GitHub, so any account's repository or gist still
qualified as a source. The handbook comes from one organization: all 148 entries
in the published manifest are `github.com/wp-cli/…`.

Check the first path segment against a list of owners. Both hosts name the owner
there, so the same list covers a source before and after the rewrite between
them, and owners are compared case insensitively, since GitHub serves them that
way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@obenland
obenland force-pushed the wporg-cli/markdown-import-source-hosts branch from 8e39213 to 6533a5e Compare July 31, 2026 14:52
@bazza bazza closed this in 6b6c62c Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants