Search parent inputs of internal parameter entities for base URI.
Fixes a long-standing bug, which manifested in a different way after
commit 955c177f. Reproduce with
xmllint --noent xmlconf/eduni/errata-2e/E18.xml
After the failed experiment with a static XML namespace, introduce
versions of xmlSearchNs that report malloc failures.
Optimize the no-document case by only adding the XML namespace
declaration if it wasn't found in an ancestor.
Replace xmlStringGetNodeList and xmlStringLenGetNodeList with
xmlNodeParseContentInternal which also updates an optional parent
node.
Don't look up entities a second time via xmlNewReference.
Don't use a separate function to handle "complex" attributes. Validate
UTF-8 byte sequences without decoding. This should improve performance
considerably when parsing multi-byte UTF-8 sequences.
Use a string buffer to avoid unnecessary allocations and copying when
expanding entities.
Normalize attribute values in a single pass while expanding entities.
Be more lenient in recovery mode.
If no entity substitution was requested, validate entities without
expanding. Fixes#596.
Also fixes#655.
Use in ctxt->input->entity instead of ctxt->inputNr to determine whether
we are inside a parameter entity.
Stop using ctxt->external to check whether we're in an external DTD.
This is signaled by ctxt->inSubset == 2.
Today I learned that the TSCII character encoding [1] can blow up the
size of text 12 times when converted to UTF-8:
$ printf '\x82' |iconv -f TSCII -t UTF-8 |hexdump -C
00000000 e0 ae b8 e0 af 8d e0 ae b0 e0 af 80
0000000c
[1] https://en.wikipedia.org/wiki/Tamil_Script_Code_for_Information_Interchange
Introduce xmlCtxtSetErrorHandler allowing to set a structured error for
a parser context. There already was the "serror" SAX handler but this
always receives the parser context as argument.
Start to use xmlRaiseMemoryError.
Remove useless arguments from memory error functions. Rename
xmlErrMemory to xmlCtxtErrMemory.
Remove a few calls to xmlGenericError.
Remove support for runtime entity debugging.
Set the dictionary for newDoc in xmlParseBalancedChunkMemoryRecover.
This is a long-standing bug which was masked by
- xmlParseBalancedChunkMemoryRecover changing the document of the root
node. This is a really bad idea, resulting in a mismatch between
ctxt->myDoc and ctxt->node->doc.
- SAX2.c preferring ctxt->node->doc over ctxt->myDoc until commit
a31e1b06.
Fixes#641.
Use a hash table to lookup namespaces by prefix. The hash table stores
an index into the namespace table. Auxiliary data for namespaces is
stored in a separate array along the main namespace table.
Use a hash table to verify attribute uniqueness. The hash table stores
an index into the attribute table.
Reuse hash value from the dictionary to avoid computing them twice.
See #346.
When parsing with HTML_PARSE_NOIMPLIED, the result document can contain
multiple top-level elements. Rework xmlSAX2StartElement to simply add
the element as a child of ctxt->node or ctxt->myDoc.
Don't invoke xmlAddSibling for non-element parents. The context node
should always be an element node.
Fixes#584.
This should make the line number of text nodes deterministic. Before,
it depended on the callback sequence which depends on the size of chunks
fed to the parser.
Introduce XML_INPUT_HAS_ENCODING flag for xmlParserInput which is set
when xmlSwitchEncoding is called. The parser can use the flag to
reliably detect whether an encoding was already set via user override,
BOM or other auto-detection. In this case, the encoding declaration
won't be used to switch the encoding.
Before, an inscrutable mix of ctxt->charset, ctxt->input->encoding
and ctxt->input->buf->encoder was used.
Introduce private helper functions to switch encodings used by both the
XML and HTML parser:
- xmlDetectEncoding which skips over the BOM, allowing to remove the
BOM checks from other encoding functions.
- xmlSetDeclaredEncoding, replacing htmlCheckEncodingDirect, which warns
about encoding mismatches.
If users override the encoding, store the declared instead of the actual
encoding in xmlDoc. In this case, the actual encoding is known and the
raw value from the doc is more useful.
Also use the input flags to store the ISO-8859-1 fallback state.
Restrict the fallback to cases where no encoding was specified. (The
fallback is only useful in recovery mode and these days broken UTF-8 is
probably more likely than ISO-8859-1, so it might eventually be removed
completely.)
The 'charset' member of xmlParserCtxt is now unused. The 'encoding'
member of xmlParserInput is now unused.
The 'standalone' member of xmlParserInput is renamed to 'flags'.
A new parser state XML_PARSER_XML_DECL is added for the push parser.
It seems that this code path could only be triggered after an encoding
error in recovery mode. Creating char-ref nodes is unnecessary and
typically unexpected.
Follow-up to commit d0c3f01e. A parser context will be initialized to
SAX version 2, but this can be overridden with XML_PARSE_SAX1 later,
so we must initialize the SAX1 element handlers as well.
Change the check in xmlDetectSAX2 to only look for XML_SAX2_MAGIC, so
we don't switch to SAX1 if the SAX2 element handlers are NULL.
In commit 21ca8829, we started to ignore namespaces in HTML element
names but we still called xmlSplitQName, effectively stripping the
namespace prefix. This would cause elements like <o:p> being parsed
as <p>. Now we leave the name untouched.
Fixes#508.
This commit implements robust detection of entity amplification attacks,
better known as the "billion laughs" attack.
We now limit the size of the document after substitution of entities to
10 times the size before expansion. This guarantees linear behavior by
definition. There already was a similar check before, but the accounting
of "sizeentities" (size of external entities) and "sizeentcopy" (size of
all copies created by entity references) wasn't accurate.
We also need saturation arithmetic since we're historically limited to
"unsigned long" which is 32-bit on many platforms.
A maximum of 10 MB of substitutions is always allowed. This should make
use cases like DITA work which have caused problems in the past.
The old checks based on the number of entities were removed. This is
accounted for by adding a fixed cost to each entity reference.
Entity amplification checks are now enabled even if XML_PARSE_HUGE is
set. This option is mainly used to allow larger text nodes. Most users
were unaware that it also disabled entity expansion checks.
Some of the limits might be adjusted later. If this change turns out to
affect legitimate use cases, we can add a separate parser option to
disable the checks.
Fixes#294.
Fixes#345.
