Use in ctxt->input->entity instead of ctxt->inputNr to determine whether
we are inside a parameter entity.
Stop using ctxt->external to check whether we're in an external DTD.
This is signaled by ctxt->inSubset == 2.
Implement section "4.6 Predefined Entities" of the XML 1.0 spec and
check whether redeclarations of predefined entities match the original
definitions.
Note that some test cases declared
<!ENTITY lt "<">
But the XML spec clearly states that this is illegal:
> If the entities lt or amp are declared, they MUST be declared as
> internal entities whose replacement text is a character reference to
> the respective character (less-than sign or ampersand) being escaped;
> the double escaping is REQUIRED for these entities so that references
> to them produce a well-formed result.
Also fixes#217 but the connection is only tangential. The integer
overflow discovered by fuzzing was more related to the fact that various
parts of the parser disagreed on whether to prefer predefined entities
over their redeclarations. The whole situation is a mess and even
depends on legacy parser options. But now that redeclarations are
validated, it shouldn't make a difference.
As noted in the added comment, this is also one of the cases where
overly defensive checks can hide interesting logic bugs from fuzzers.
xmlSnprintfElementContent failed to correctly check the available
buffer space in two locations.
Fixes bug 781333 (CVE-2017-9047) and bug 781701 (CVE-2017-9048).
Thanks to Marcel Böhme and Thuan Pham for the report.
There were two bugs where parameter-entity references could lead to an
unexpected change of the input buffer in xmlParseNameComplex and
xmlDictLookup being called with an invalid pointer.
Percent sign in DTD Names
=========================
The NEXTL macro used to call xmlParserHandlePEReference. When parsing
"complex" names inside the DTD, this could result in entity expansion
which created a new input buffer. The fix is to simply remove the call
to xmlParserHandlePEReference from the NEXTL macro. This is safe because
no users of the macro require expansion of parameter entities.
- xmlParseNameComplex
- xmlParseNCNameComplex
- xmlParseNmtoken
The percent sign is not allowed in names, which are grammatical tokens.
- xmlParseEntityValue
Parameter-entity references in entity values are expanded but this
happens in a separate step in this function.
- xmlParseSystemLiteral
Parameter-entity references are ignored in the system literal.
- xmlParseAttValueComplex
- xmlParseCharDataComplex
- xmlParseCommentComplex
- xmlParsePI
- xmlParseCDSect
Parameter-entity references are ignored outside the DTD.
- xmlLoadEntityContent
This function is only called from xmlStringLenDecodeEntities and
entities are replaced in a separate step immediately after the function
call.
This bug could also be triggered with an internal subset and double
entity expansion.
This fixes bug 766956 initially reported by Wei Lei and independently by
Chromium's ClusterFuzz, Hanno Böck, and Marco Grassi. Thanks to everyone
involved.
xmlParseNameComplex with XML_PARSE_OLD10
========================================
When parsing Names inside an expanded parameter entity with the
XML_PARSE_OLD10 option, xmlParseNameComplex would call xmlGROW via the
GROW macro if the input buffer was exhausted. At the end of the
parameter entity's replacement text, this function would then call
xmlPopInput which invalidated the input buffer.
There should be no need to invoke GROW in this situation because the
buffer is grown periodically every XML_PARSER_CHUNK_SIZE characters and,
at least for UTF-8, in xmlCurrentChar. This also matches the code path
executed when XML_PARSE_OLD10 is not set.
This fixes bugs 781205 (CVE-2017-9049) and 781361 (CVE-2017-9050).
Thanks to Marcel Böhme and Thuan Pham for the report.
Additional hardening
====================
A separate check was added in xmlParseNameComplex to validate the
buffer size.
For https://bugzilla.gnome.org/show_bug.cgi?id=737840
the fix for 724903 introduced a regression on external entities carrying
IDs, revert that patch in part and add a specific test to avoid readding it
* valid.c: fix a bug when valdating mixed content lists and some
name use namespaces prefixes.
* result/valid/notes.xml* test/valid/dtds/notes.dtd * test/valid/notes.xml:
add the test case to the regression suite
* parser.c: fix a problem reported by Ashwin for system parameter
entities referenced from entities in external subset, add a
specific loading routine.
* test/valid/dtds/external.ent test/valid/dtds/external2.ent
test/valid/t11.xml result/valid/t11.xml*: added the test to
the regression suite
Daniel
svn path=/trunk/; revision=3713
* encoding.c: poblem with encoding detection for UTF-16 reported by
Ashwin and found by Bill
* test/valid/dtds/utf16b.ent test/valid/dtds/utf16l.ent
test/valid/UTF16Entity.xml result/valid/UTF16Entity.xml*: added
the example to the regression tests
Daniel
svn path=/trunk/; revision=3700
* parser.c: fixed bug #170489 reported by Jirka Kosek
* test/valid/objednavka.xml test/valid/dtds/objednavka.dtd
result/valid/objednavka*: added the test to the regression suite.
Daniel
* configure.in: making DSO support an option
* xmlmodule.c xmlreader.c include/libxml/xmlmodule.h: code
and documentation cleanups
* elfgcchack.h testapi.c doc/*: regenerated the docs and
checks for new module
* test/valid/REC-xml-19980210.xml: fix a small change introduced
previously
Daniel
* valid.c: fixed ID deallocation problem based on patch from
Steve Shepard fixes bug #160893
* xmlmemory.c: improving comment.
* testapi.c: new test for xmlDictExists() is generated.
Daniel
* valid.c: implemented bugfix from Massimo Morara for DTD
dumping problem.
* test/valid/t10.xml, result/valid/t10.*: added regression
for above
* configure.in: small change for my profile settings
* parser.c: swapped the attribute defaulting and attribute checking
parts of parsing a new element start, fixes bug #127772
* result/valid/127772.* test/valid/127772.xml
test/valid/dtds/127772.dtd: added the example in the regression tests
Daniel
* valid.c: fixed bug #118712 about mixed content, and namespaced
element names.
* test/valid/mixed_ns.xml result/valid/mixed_ns*: added a check
in the regression tests
Daniel
* SAX.c test/valid/ns* test/result/ns*: applied the patch
provided by Brent Hendricks fixing #105992 and integrated the
examples in the testsuite.
Daniel
* parser.c: fixing bug #108976 get the ID/REFs to reference
the ID in the document content and not in the entity copy
* SAX.c include/libxml/parser.h: more checking of the ID/REF
stuff, better solution for #107208
* xmlregexp.c: removed a direct printf, dohhh
* xmlreader.c: fixed a bug on streaming validation of empty
elements in entities
* result/VC/ElementValid8 test/VCM/v20.xml result/valid/xhtml1.xhtml:
cleanup of the validation tests
* test/valid/id* test/valid/dtds/destfoo.ent result/valid/id*:
added more ID/IDREF tests to the suite
Daniel
* SAX.c valid.c include/libxml/valid.h: fixed bug #92518 validation
error were not covering namespace declarations.
* result/valid/dia.xml test/valid/dia.xml: the test wasn't valid,
it was missing the attribute declaration for the namespace
* result/VC/NS3: the fix now report breakages in that test
Daniel
* parser.c: applied a couple of patches from Peter Jacobi to start
to get rid of ctxt->token, with a possible significant speed
improvement to be gained once done. Better compliance with PE
references constructs in DTDs too.
* test/valid/t[0-9]* result/valid/t[0-9]*: added a set of tests
from Peter too
Daniel
* valid.c include/libxml/tree.h: trying to fix namespaces +
validation problems for good, closing #63619 in the process
* result/valid/dia.xml test/valid/dia.xml: the Dia test was
wrong in this respect, fixed it.
Daniel
* parser.c: fixed an erroneous validation bug when PE refs
occurs in external parsed entities referenced from the
internals subset
* test/valid/index.xml test/valid/dtds/nitf-2-5.dtd
test/valid/dtds/NewsMLv1.0.dtd result/valid/index.xml*:
added the associated testcase, it's a nice one.
* HTMLparser.c: generate the DTD node as HTML still ...
* HTMLtree.c: fixed errors in Set/GetMetaEncoding
Daniel
- TODO: updated
- xmlmemory.[ch] : added xmlMemSetup() and xmlMemGet() to override
libxml default allocation function with another set (like gmalloc/gfree).
- Makefile.am, uri.c, uri.h: added a set of functions to do exact (litteraly
copied from the RFC 2396 productions) parsing and handling of URI.
Daniel
- changed the way Windows socket stuff get included
- removed an indetermination xmLDecl/PI(xml...)
- xmlNewNs wasn't checking for double definition
- fixed a problem with dist-hook duplicates
- fixed the loading of external entities APIs, now xmlLoadExternalEntity()
is used everywhere
- now the xhtml spec validates with the xhtml DTD.
- error.c: fixed crashes in case of no input stream
- added the xhtml spec and dtds to the validation tests and results
Daniel
