COMMONSXML-4: Make the resolver floors ignore-all with a throw-on-unresolved toggle - #30
COMMONSXML-4: Make the resolver floors ignore-all with a throw-on-unresolved toggle#30ppkarwasz wants to merge 14 commits into
Conversation
How to read this PRThis is probably one of the most disruptive PRs, since it inverts the default behavior that was there from the start. TL;DR: to accommodate as many users as possible, the (non-configurable) behavior of the library is as forgiving as possible. Users that want more drastic behavior will be allowed to do that themselves, e.g. set Cases where this PR can not prevent the occurrence of exceptions or parsing errors reported through
Emptied |
|
@ppkarwasz |
|
Every external fetch will return an empty stream or That operation alone will not cause any errors, but its side-effects will: a missing entity declaration, an empty imported schema, and so on. This PR also defines a system property |
|
I documented the system property in: 8edd19b |
An external reference the caller's resolver does not resolve is now resolved to empty content rather than rejected with an exception. Nothing is fetched either way, so the security property is unchanged; the difference is that a parse now continues with empty content instead of failing. Rename the four floors accordingly and collapse the variants the deny/ignore split required: FallbackDenyEntityResolver2 -> FallbackIgnoreEntityResolver2 FallbackDenyXMLResolver -> FallbackIgnoreXMLResolver FallbackDenyLSResourceResolver -> FallbackIgnoreLSResourceResolver FallbackDenyURIResolver -> FallbackIgnoreURIResolver `SAXParserHardener.DtdAwareDenyResolver` and `StaxHardener.DtdSubsetFloor` existed only to exempt the external DTD subset from the deny, and the old `FallbackIgnoreXMLResolver` only to exempt Woodstox's undeclared entities; with ignore as the default all three are redundant. Removing them, together with the now-dead `HardeningException.forbidden`, takes the shade closure from 33 classes to 30. StAX collapses further: the floor moves into the `HardeningXMLInputFactory` constructor (as `HardeningXMLReader` already did), a single `setXMLResolver` covers both the JDK Zephyr and Woodstox (whose `setXMLResolver` fans out to its DTD-subset and entity resolvers), and the Zephyr `ignore-external-dtd` property is dropped because the floor already empties the subset. Only Woodstox's undeclared-entity hook stays separate: emptying the external subset leaves the entities it declared undeclared, and that hook is outside the fan-out. Tests that asserted an exception now assert the external resource does not leak. Where the outcome differs by implementation (Saxon still rejects through ALLOWED_PROTOCOLS, an emptied schema import fails to compile) they accept either outcome through the new assert*BlocksOrDoesNotLeak helpers. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With the ignore-all floors as universal behavior, skipping the external DTD subset up front is redundant: when the parser requests it, the floor resolves it to empty content and the parse continues. Remove the feature from the DOM and SAX recipes together with the setOptionalFeature helpers it was the only user of. Actually requesting the subset exposed two gaps in the floors, fixed here: both now echo the requested identifiers on the empty source they return (Xerces derives the entity's base URI from the system id and fails on null), and the LSResourceResolver floor hands its empty content over as a character stream, because the JDK's DOMEntityResolverWrapper discards empty string data. The ignore outcome stays best-effort: Saxon's ALLOWED_PROTOCOLS restrictor sits ahead of the floor and rejects the subset lookup outright, so the affected TrAX tests accept either outcome through the assert*BlocksOrDoesNotLeak helpers and DoctypeOnlyTest documents the dual outcome. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Saxon was the odd one out: its ALLOWED_PROTOCOLS="" lockdown rejected every unresolved external reference (for example an external DTD subset) with an exception, where the other implementations resolve it to empty content through a floor. Drop the ALLOWED_PROTOCOLS setting and install a nature-aware ignore-all ResourceResolver on the HardenedConfiguration instead: - XML, XSLT and XSD natures resolve to Saxon's EmptySource, so an unresolved xsl:include/xsl:import compiles as an empty module and doc()/document() return the empty sequence. - Text and binary natures resolve to an empty StreamSource, so unparsed-text() yields the empty string. - External-entity and DTD natures return null so the lookup falls through to the hardened reader's entity-resolver floor, keeping caller allow-listing on the reader working. The floor backs every resolution chain ahead of Saxon's direct-fetch fallback. A setResourceResolver override re-wraps any resolver installed later (including through the plain-JAXP TransformerFactory.setURIResolver route, which replaces the Configuration resolver wholesale) with the floor as its fallback, and an empty CollectionFinder covers fn:collection, the one channel that bypasses the resource resolver. Saxon now passes the DOCTYPE-only and external-DTD suites like the other stacks; tests where implementations still diverge on unresolved xsl:import/include (XSLTC and Xalan reject the emptied module, Saxon compiles it) accept either outcome and assert no leak. Pin the new nested class in ShadingFootprintTest. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Shorten the floor and constructor comments and install the floor through the setResourceResolver override instead of calling super directly. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Make AttackTestSupport.captureCharacters package-private, add an InputSource overload and a capturingHandler factory, and replace the eleven anonymous DefaultHandler copies in EntityResolverFloorTest and XIncludeTest with them. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Fold CDATA capture into AttackTestSupport.captureStaxEventText, make it package-private and drop the duplicate readStaxText. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
An empty character stream is not a well-formed XML document, so XSLTC rejects it for document() and both XSLTC and Xalan reject an ignored xsl:include or xsl:import at compile time. A well-formed empty document lets every implementation proceed and evaluate to no content, so the include, import and document() tests now assert the strict no-leak outcome on all implementations. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
With Saxon's protocol restrictor gone, the external-DTD and external-general-entity TrAX tests assert the strict no-leak outcome. The parameter-entity tests stay strict through a caller-supplied error listener that accepts only the JDK parser's "referenced, but not declared" report, which XML 1.0 section 4.1 demotes to an unreported validity constraint when the internal subset contains a parameter entity reference. Drop the now-unused blocks-or-does-not-leak Templates and Transformer helpers. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Keep the pipeline strict and hand a thrown TransformerException to a caller-supplied ThrowingConsumer that rethrows it (the default) or returns to accept the block. This replaces the ErrorListener overloads and their swallowed-report tracking: with the strict listener installed the factories throw at the first report, so the half-built Templates path never runs. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
ExternalGeneralEntityTest declares its entity in the internal subset, so nothing is ever undeclared and every surface asserts the strict no-leak outcome. ExternalParameterEntityTest is the one payload with a genuinely undeclared entity after hardening, so all its surfaces assert the dual blocks-or-does-not-leak contract, replacing the trax message-matching acceptance. Methods are renamed to state the assertion they make. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Emptying the external subset leaves any entity it declared undeclared, and Woodstox rejects such a reference unconditionally. The rejection keeps the resource just as unfetched as the empty resolution the other implementations produce, so the per-implementation hook is unnecessary; a caller can still opt references in by setting the property, which lands their resolver behind a floor like every other resolver hook. ExternalDtdTest's StAX case accepts the block accordingly. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Setting -Dorg.apache.commons.xml.throwOnUnresolved=true switches every resolver floor from resolving an unresolved external reference to empty content to rejecting it with the resolution hook's exception. The property is read at resolution time, so it also applies to existing factories, and references resolved by a caller-supplied resolver are unaffected. Requested by Gary in COMMONSXML-4. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Restore HardeningException.forbidden with its five-part identifier tuple, now also naming the enabling property, in place of the single-argument unresolvedDenied helper the toggle had introduced. Each floor passes the same identifiers the old deny floors did. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
Add a Configuration section: external references resolve to empty streams, applications that want rejection tighten the factory themselves (a stricter feature or a throwing resolver, with no security weight either way), and org.apache.commons.xml.throwOnUnresolved is a temporary debugging switch. Align the Usage paragraph with the same contract. Assisted-By: Claude Fable 5 <noreply@anthropic.com>
8edd19b to
d962e4c
Compare
Stacked on #28 (
feature/reduce-shade-footprint); only the commits above that branch belong to this PR.Implements COMMONSXML-4: a consistent contract for denied external fetches, with ignore-all as the default and an opt-in system property that switches to throwing.
Why
The deny-all resolver floors rejected every unresolved external reference with an exception, so a document that merely carries a DOCTYPE (without needing anything from it) failed to parse. Resolving unresolved references to empty content keeps the identical security property, nothing is fetched and nothing leaks, while letting such documents parse. The ignore outcome is best-effort: where an implementation prescribes a rejection, the block remains an acceptable outcome. Per the COMMONSXML-4 discussion, silently empty resolutions can also hide problems during debugging, so a system property lets deployments opt into rejection instead.
What
FallbackDeny*toFallbackIgnore*. The deny/ignore split made several variants redundant (DtdAwareDenyResolver,DtdSubsetFloor, the old Woodstox-onlyFallbackIgnoreXMLResolver); removing them takes the shade closure from 33 to 30 classes. The StAX floor collapses into theHardeningXMLInputFactoryconstructor, with a singlesetXMLResolvercovering both the JDK Zephyr and Woodstox.XERCES_LOAD_EXTERNAL_DTDfrom the hardening recipes. Skipping the external DTD subset up front is redundant once the floor resolves it to empty. Actually requesting the subset exposed two gaps, fixed here: the floors now echo the requested identifiers on the empty source they return (Xerces derives the entity's base URI from the system id and fails on null), and the LSResourceResolver floor hands its content over as a character stream because the JDK'sDOMEntityResolverWrapperdiscards empty string data.ALLOWED_PROTOCOLS=""lockdown was the odd one out, rejecting instead of ignoring. It is replaced by a nature-aware ignore-allResourceResolveron theHardenedConfiguration(EmptySourcefor XML/XSLT/XSD natures, an emptyStreamSourcefor text,nullfor entity/DTD natures so the lookup falls through to the hardened reader's floor and caller allow-listing keeps working), made non-removable by asetResourceResolveroverride (plain-JAXPTransformerFactory.setURIResolverreplaces the Configuration resolver wholesale), plusCollectionFn.EMPTY_COLLECTIONforfn:collection, the one channel that bypasses the resolver.document()and both XSLTC and Xalan rejected an ignoredxsl:include/xsl:importat compile time. TheURIResolverfloor now answers with an empty DOM document that every engine digests, and the include, import anddocument()tests assert the strict no-leak outcome on all implementations.ExternalParameterEntityTest(the JDK parser reports the reference as a well-formedness fatal although XML 1.0 section 4.1 demotes it to an unreported validity constraint when the internal subset contains a parameter-entity reference; Apache Xerces implements the demotion) and the StAX case ofExternalDtdTest(Woodstox rejects undeclared references unconditionally). Every other hardened test asserts the strict no-leak outcome, test method names state the assertion they make, and the Woodstox-only undeclared-entity-resolver override is removed from the StAX recipe as a per-implementation workaround the contract no longer needs.-Dorg.apache.commons.xml.throwOnUnresolved=trueswitches every floor, including Saxon's resource resolver and collection finder, from resolving an unresolved reference to empty content to rejecting it with the resolution hook's exception (SAXException,XMLStreamException,LSException,TransformerException,XPathException), carrying the pre-existingHardeningException.forbiddenmessage with the full identifier tuple. The property is read at resolution time so it applies to existing factories, the failure message names the denied identifier and the enabling property, and references resolved by a caller-supplied resolver are unaffected. Documented in theXmlFactoriesJavadoc and the threat model, which now records the one system property the library reads.Testing
Full surefire matrix green (stock JDK, Xerces, Xalan, Xalan+Xerces, Saxon, Saxon+Xerces, Woodstox; 84 reports).
DenyUnresolvedTestexercises the toggle at the resolver level.ShadingFootprintTestkeeps pinning the per-hardener shade closures; the StAX closure deliberately gainsHardeningException, which now carries the shared toggle mechanism and denial message.🤖 Generated with Claude Code