Skip to content

COMMONSXML-4: Make the resolver floors ignore-all with a throw-on-unresolved toggle - #30

Open
ppkarwasz wants to merge 14 commits into
feature/reduce-shade-footprintfrom
feature/ignore-all-resolver-floors
Open

COMMONSXML-4: Make the resolver floors ignore-all with a throw-on-unresolved toggle#30
ppkarwasz wants to merge 14 commits into
feature/reduce-shade-footprintfrom
feature/ignore-all-resolver-floors

Conversation

@ppkarwasz

@ppkarwasz ppkarwasz commented Aug 1, 2026

Copy link
Copy Markdown
Member

Stacked on #28 (feature/reduce-shade-footprint); only the commits above that branch belong to this PR.

Implements COMMONSXML-4: a consistent contract for denied external fetches, with ignore-all as the default and an opt-in system property that switches to throwing.

Why

The deny-all resolver floors rejected every unresolved external reference with an exception, so a document that merely carries a DOCTYPE (without needing anything from it) failed to parse. Resolving unresolved references to empty content keeps the identical security property, nothing is fetched and nothing leaks, while letting such documents parse. The ignore outcome is best-effort: where an implementation prescribes a rejection, the block remains an acceptable outcome. Per the COMMONSXML-4 discussion, silently empty resolutions can also hide problems during debugging, so a system property lets deployments opt into rejection instead.

What

  1. Make the resolver floors ignore-all instead of deny-all. The four floors resolve unresolved lookups to empty content and are renamed FallbackDeny* to FallbackIgnore*. The deny/ignore split made several variants redundant (DtdAwareDenyResolver, DtdSubsetFloor, the old Woodstox-only FallbackIgnoreXMLResolver); removing them takes the shade closure from 33 to 30 classes. The StAX floor collapses into the HardeningXMLInputFactory constructor, with a single setXMLResolver covering both the JDK Zephyr and Woodstox.
  2. Drop XERCES_LOAD_EXTERNAL_DTD from the hardening recipes. Skipping the external DTD subset up front is redundant once the floor resolves it to empty. Actually requesting the subset exposed two gaps, fixed here: the floors now echo the requested identifiers on the empty source they return (Xerces derives the entity's base URI from the system id and fails on null), and the LSResourceResolver floor hands its content over as a character stream because the JDK's DOMEntityResolverWrapper discards empty string data.
  3. Extend the ignore-all floor to Saxon resource resolution. Saxon's ALLOWED_PROTOCOLS="" lockdown was the odd one out, rejecting instead of ignoring. It is replaced by a nature-aware ignore-all ResourceResolver on the HardenedConfiguration (EmptySource for XML/XSLT/XSD natures, an empty StreamSource for text, null for entity/DTD natures so the lookup falls through to the hardened reader's floor and caller allow-listing keeps working), made non-removable by a setResourceResolver override (plain-JAXP TransformerFactory.setURIResolver replaces the Configuration resolver wholesale), plus CollectionFn.EMPTY_COLLECTION for fn:collection, the one channel that bypasses the resolver.
  4. Resolve ignored URIs to a well-formed empty document. An empty character stream is not well-formed XML, so XSLTC rejected it for document() and both XSLTC and Xalan rejected an ignored xsl:include/xsl:import at compile time. The URIResolver floor now answers with an empty DOM document that every engine digests, and the include, import and document() tests assert the strict no-leak outcome on all implementations.
  5. State the undeclared-entity contract precisely. The dual "blocks or does not leak" assertion survives only where an entity is genuinely undeclared after hardening: all of ExternalParameterEntityTest (the JDK parser reports the reference as a well-formedness fatal although XML 1.0 section 4.1 demotes it to an unreported validity constraint when the internal subset contains a parameter-entity reference; Apache Xerces implements the demotion) and the StAX case of ExternalDtdTest (Woodstox rejects undeclared references unconditionally). Every other hardened test asserts the strict no-leak outcome, test method names state the assertion they make, and the Woodstox-only undeclared-entity-resolver override is removed from the StAX recipe as a per-implementation workaround the contract no longer needs.
  6. Add the COMMONSXML-4 toggle. Setting -Dorg.apache.commons.xml.throwOnUnresolved=true switches every floor, including Saxon's resource resolver and collection finder, from resolving an unresolved reference to empty content to rejecting it with the resolution hook's exception (SAXException, XMLStreamException, LSException, TransformerException, XPathException), carrying the pre-existing HardeningException.forbidden message with the full identifier tuple. The property is read at resolution time so it applies to existing factories, the failure message names the denied identifier and the enabling property, and references resolved by a caller-supplied resolver are unaffected. Documented in the XmlFactories Javadoc and the threat model, which now records the one system property the library reads.

Testing

Full surefire matrix green (stock JDK, Xerces, Xalan, Xalan+Xerces, Saxon, Saxon+Xerces, Woodstox; 84 reports). DenyUnresolvedTest exercises the toggle at the resolver level. ShadingFootprintTest keeps pinning the per-hardener shade closures; the StAX closure deliberately gains HardeningException, which now carries the shared toggle mechanism and denial message.

🤖 Generated with Claude Code

@ppkarwasz
ppkarwasz marked this pull request as draft August 2, 2026 08:33
@ppkarwasz ppkarwasz changed the title Make the resolver floors ignore-all instead of deny-all COMMONSXML-4: Make the resolver floors ignore-all with a throw-on-unresolved toggle Aug 2, 2026
@ppkarwasz
ppkarwasz marked this pull request as ready for review August 2, 2026 13:14
@ppkarwasz

Copy link
Copy Markdown
Member Author

How to read this PR

This is probably one of the most disruptive PRs, since it inverts the default behavior that was there from the start.

TL;DR: to accommodate as many users as possible, the (non-configurable) behavior of the library is as forgiving as possible. Users that want more drastic behavior will be allowed to do that themselves, e.g. set disallow-doctype-decl to have parsers throw (if they support the feature). However that behavior will be devoid of any security implications since the security floor is assured.

Cases where this PR can not prevent the occurrence of exceptions or parsing errors reported through ErrorHandler/ErrorListener are those where the implementation structurally cannot proceed with empty content:

  • An entity left undeclared by an emptied DTD. Emptying an external subset or a parameter entity erases the declarations it carried, so a later reference to such an entity is undeclared. XML 1.0 section 4.1 demotes that to an unreported validity constraint (Apache Xerces conforms), but the JDK's parser reports a well-formedness fatal unless the DOCTYPE has a system identifier, and Woodstox rejects every undeclared reference unconditionally. This reaches every surface parsing with the JDK parser (DOM, SAX, schema, validation, TrAX stylesheet and input parsing) and StAX on Woodstox.
  • XInclude with parse="xml". The emptied include target is not a well-formed XML document, so the include fails on every implementation; parse="text" completes with empty content.
  • XML Schema composition and location hints. An XSD whose xs:import, xs:include or xs:redefine target was emptied does not compile, and validation driven by an xsi:schemaLocation/xsi:noNamespaceSchemaLocation hint pointing at an unresolved schema fails the same way: an empty document is not a schema document.
  • Secure-processing limits (unrelated to resolution): oversized entity expansion such as Billion Laughs and similar limits still abort the parse on every implementation.

Emptied xsl:include/xsl:import and document() are deliberately absent from this list: they now compile and evaluate to no content on every engine.

@garydgregory

Copy link
Copy Markdown
Member

@ppkarwasz
Do I still get errors/exceptions if I register a listener?

@ppkarwasz

Copy link
Copy Markdown
Member Author

Every external fetch will return an empty stream or DOMSource, depending on the JAXP factory.

That operation alone will not cause any errors, but its side-effects will: a missing entity declaration, an empty imported schema, and so on.

This PR also defines a system property org.apache.commons.xml.throwOnUnresolved that throws on any external fetch, if set to true.

@ppkarwasz

Copy link
Copy Markdown
Member Author

I documented the system property in: 8edd19b

An external reference the caller's resolver does not resolve is now resolved
to empty content rather than rejected with an exception. Nothing is fetched
either way, so the security property is unchanged; the difference is that a
parse now continues with empty content instead of failing.

Rename the four floors accordingly and collapse the variants the deny/ignore
split required:

  FallbackDenyEntityResolver2    -> FallbackIgnoreEntityResolver2
  FallbackDenyXMLResolver        -> FallbackIgnoreXMLResolver
  FallbackDenyLSResourceResolver -> FallbackIgnoreLSResourceResolver
  FallbackDenyURIResolver        -> FallbackIgnoreURIResolver

`SAXParserHardener.DtdAwareDenyResolver` and `StaxHardener.DtdSubsetFloor`
existed only to exempt the external DTD subset from the deny, and the old
`FallbackIgnoreXMLResolver` only to exempt Woodstox's undeclared entities;
with ignore as the default all three are redundant. Removing them, together
with the now-dead `HardeningException.forbidden`, takes the shade closure from
33 classes to 30.

StAX collapses further: the floor moves into the `HardeningXMLInputFactory`
constructor (as `HardeningXMLReader` already did), a single `setXMLResolver`
covers both the JDK Zephyr and Woodstox (whose `setXMLResolver` fans out to
its DTD-subset and entity resolvers), and the Zephyr `ignore-external-dtd`
property is dropped because the floor already empties the subset. Only
Woodstox's undeclared-entity hook stays separate: emptying the external subset
leaves the entities it declared undeclared, and that hook is outside the
fan-out.

Tests that asserted an exception now assert the external resource does not
leak. Where the outcome differs by implementation (Saxon still rejects through
ALLOWED_PROTOCOLS, an emptied schema import fails to compile) they accept
either outcome through the new assert*BlocksOrDoesNotLeak helpers.

Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With the ignore-all floors as universal behavior, skipping the external
DTD subset up front is redundant: when the parser requests it, the floor
resolves it to empty content and the parse continues. Remove the feature
from the DOM and SAX recipes together with the setOptionalFeature
helpers it was the only user of.

Actually requesting the subset exposed two gaps in the floors, fixed
here: both now echo the requested identifiers on the empty source they
return (Xerces derives the entity's base URI from the system id and
fails on null), and the LSResourceResolver floor hands its empty content
over as a character stream, because the JDK's DOMEntityResolverWrapper
discards empty string data.

The ignore outcome stays best-effort: Saxon's ALLOWED_PROTOCOLS
restrictor sits ahead of the floor and rejects the subset lookup
outright, so the affected TrAX tests accept either outcome through the
assert*BlocksOrDoesNotLeak helpers and DoctypeOnlyTest documents the
dual outcome.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Saxon was the odd one out: its ALLOWED_PROTOCOLS="" lockdown rejected
every unresolved external reference (for example an external DTD subset)
with an exception, where the other implementations resolve it to empty
content through a floor.

Drop the ALLOWED_PROTOCOLS setting and install a nature-aware ignore-all
ResourceResolver on the HardenedConfiguration instead:

- XML, XSLT and XSD natures resolve to Saxon's EmptySource, so an
  unresolved xsl:include/xsl:import compiles as an empty module and
  doc()/document() return the empty sequence.
- Text and binary natures resolve to an empty StreamSource, so
  unparsed-text() yields the empty string.
- External-entity and DTD natures return null so the lookup falls
  through to the hardened reader's entity-resolver floor, keeping caller
  allow-listing on the reader working.

The floor backs every resolution chain ahead of Saxon's direct-fetch
fallback. A setResourceResolver override re-wraps any resolver installed
later (including through the plain-JAXP TransformerFactory.setURIResolver
route, which replaces the Configuration resolver wholesale) with the
floor as its fallback, and an empty CollectionFinder covers
fn:collection, the one channel that bypasses the resource resolver.

Saxon now passes the DOCTYPE-only and external-DTD suites like the other
stacks; tests where implementations still diverge on unresolved
xsl:import/include (XSLTC and Xalan reject the emptied module, Saxon
compiles it) accept either outcome and assert no leak. Pin the new
nested class in ShadingFootprintTest.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Shorten the floor and constructor comments and install the floor through
the setResourceResolver override instead of calling super directly.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Make AttackTestSupport.captureCharacters package-private, add an
InputSource overload and a capturingHandler factory, and replace the
eleven anonymous DefaultHandler copies in EntityResolverFloorTest and
XIncludeTest with them.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Fold CDATA capture into AttackTestSupport.captureStaxEventText, make it
package-private and drop the duplicate readStaxText.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
An empty character stream is not a well-formed XML document, so XSLTC
rejects it for document() and both XSLTC and Xalan reject an ignored
xsl:include or xsl:import at compile time. A well-formed empty document
lets every implementation proceed and evaluate to no content, so the
include, import and document() tests now assert the strict no-leak
outcome on all implementations.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
With Saxon's protocol restrictor gone, the external-DTD and
external-general-entity TrAX tests assert the strict no-leak outcome.
The parameter-entity tests stay strict through a caller-supplied error
listener that accepts only the JDK parser's "referenced, but not
declared" report, which XML 1.0 section 4.1 demotes to an unreported
validity constraint when the internal subset contains a parameter
entity reference. Drop the now-unused blocks-or-does-not-leak Templates
and Transformer helpers.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Keep the pipeline strict and hand a thrown TransformerException to a
caller-supplied ThrowingConsumer that rethrows it (the default) or
returns to accept the block. This replaces the ErrorListener overloads
and their swallowed-report tracking: with the strict listener installed
the factories throw at the first report, so the half-built Templates
path never runs.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
ExternalGeneralEntityTest declares its entity in the internal subset,
so nothing is ever undeclared and every surface asserts the strict
no-leak outcome. ExternalParameterEntityTest is the one payload with a
genuinely undeclared entity after hardening, so all its surfaces assert
the dual blocks-or-does-not-leak contract, replacing the trax
message-matching acceptance. Methods are renamed to state the assertion
they make.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Emptying the external subset leaves any entity it declared undeclared,
and Woodstox rejects such a reference unconditionally. The rejection
keeps the resource just as unfetched as the empty resolution the other
implementations produce, so the per-implementation hook is unnecessary;
a caller can still opt references in by setting the property, which
lands their resolver behind a floor like every other resolver hook.
ExternalDtdTest's StAX case accepts the block accordingly.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Setting -Dorg.apache.commons.xml.throwOnUnresolved=true switches every
resolver floor from resolving an unresolved external reference to empty
content to rejecting it with the resolution hook's exception. The
property is read at resolution time, so it also applies to existing
factories, and references resolved by a caller-supplied resolver are
unaffected. Requested by Gary in COMMONSXML-4.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Restore HardeningException.forbidden with its five-part identifier
tuple, now also naming the enabling property, in place of the
single-argument unresolvedDenied helper the toggle had introduced. Each
floor passes the same identifiers the old deny floors did.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Add a Configuration section: external references resolve to empty
streams, applications that want rejection tighten the factory
themselves (a stricter feature or a throwing resolver, with no security
weight either way), and org.apache.commons.xml.throwOnUnresolved is a
temporary debugging switch. Align the Usage paragraph with the same
contract.

Assisted-By: Claude Fable 5 <noreply@anthropic.com>
@garydgregory
garydgregory force-pushed the feature/ignore-all-resolver-floors branch from 8edd19b to d962e4c Compare August 2, 2026 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants