Libxml2 XmlTextReader Interface tutorial

Linux 4gvps.4gvps.com 3.10.0-1127.18.2.vz7.163.46 #1 SMP Fri Nov 20 21:47:55 MSK 2020 x86_64
SOFT : Apache PHP : 7.4.33
/usr/share/doc/libxml2-devel-2.9.1/

38.135.39.45

[ NAME ]	[ SIZE ]	[ PERM ]	[ DATE ]	[ ACT ]
+FILE +DIR
examples	dir	drwxr-xr-x	2025-06-12 01:21	R D
html	dir	drwxr-xr-x	2025-06-12 01:21	R D
tutorial	dir	drwxr-xr-x	2025-06-12 01:21	R D
APIchunk0.html	29.384 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk1.html	36.614 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk10.html	62.818 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk11.html	32.428 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk12.html	85.95 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk13.html	62.372 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk14.html	44.632 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk15.html	43.199 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk16.html	35.448 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk17.html	52.79 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk18.html	41.746 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk19.html	35.798 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk2.html	39.792 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk20.html	31.978 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk21.html	36.93 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk22.html	56.599 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk23.html	62.125 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk24.html	92.134 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk25.html	40.752 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk26.html	31.368 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk27.html	33.186 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk28.html	56.492 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk29.html	13.099 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk3.html	35.345 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk4.html	35.34 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk5.html	29.043 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk6.html	28.34 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk7.html	32.077 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk8.html	29.517 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIchunk9.html	28.025 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIconstructors.html	58.105 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIfiles.html	320.544 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIfunctions.html	211.936 KB	-rw-r--r--	2013-04-19 07:28	R E G D
APIsymbols.html	317.849 KB	-rw-r--r--	2013-04-19 07:28	R E G D
AUTHORS	0.203 KB	-rw-r--r--	2009-07-30 15:24	R E G D
Copyright	1.259 KB	-rw-r--r--	2013-02-27 05:08	R E G D
DOM.gif	3.092 KB	-rw-r--r--	2009-07-30 15:24	R E G D
DOM.html	6.462 KB	-rw-r--r--	2013-03-27 08:08	R E G D
FAQ.html	20.571 KB	-rw-r--r--	2013-03-27 08:08	R E G D
Libxml2-Logo-180x168.gif	8.003 KB	-rw-r--r--	2009-07-30 15:24	R E G D
Libxml2-Logo-90x34.gif	2.998 KB	-rw-r--r--	2009-07-30 15:24	R E G D
NEWS	93.596 KB	-rw-r--r--	2013-02-27 05:08	R E G D
README	1.161 KB	-rw-r--r--	2013-02-27 05:08	R E G D
XMLinfo.html	6.638 KB	-rw-r--r--	2013-03-27 08:08	R E G D
XSLT.html	5.64 KB	-rw-r--r--	2013-03-27 08:08	R E G D
architecture.html	6.699 KB	-rw-r--r--	2013-03-27 08:08	R E G D
bugs.html	10 KB	-rw-r--r--	2013-03-27 08:08	R E G D
catalog.gif	5.962 KB	-rw-r--r--	2009-07-30 15:24	R E G D
catalog.html	23.077 KB	-rw-r--r--	2013-03-27 08:08	R E G D
contribs.html	7.495 KB	-rw-r--r--	2013-03-27 08:08	R E G D
docs.html	7.439 KB	-rw-r--r--	2013-03-27 08:08	R E G D
downloads.html	8.066 KB	-rw-r--r--	2013-03-27 08:08	R E G D
encoding.html	18.926 KB	-rw-r--r--	2013-03-27 08:08	R E G D
entities.html	9.222 KB	-rw-r--r--	2013-03-27 08:08	R E G D
example.html	12.795 KB	-rw-r--r--	2013-03-27 08:08	R E G D
guidelines.html	17.248 KB	-rw-r--r--	2009-07-30 15:24	R E G D
help.html	6.17 KB	-rw-r--r--	2013-03-27 08:08	R E G D
index.html	16.46 KB	-rw-r--r--	2013-03-27 08:08	R E G D
interface.html	8.015 KB	-rw-r--r--	2013-03-27 08:08	R E G D
intro.html	7.035 KB	-rw-r--r--	2013-03-27 08:08	R E G D
library.html	14.605 KB	-rw-r--r--	2013-03-27 08:08	R E G D
libxml.gif	7.512 KB	-rw-r--r--	2009-07-30 15:24	R E G D
libxml2-api.xml.gz	157.778 KB	-rw-r--r--	2025-05-08 14:26	R E G D
namespaces.html	8.142 KB	-rw-r--r--	2013-03-27 08:08	R E G D
news.html	125.522 KB	-rw-r--r--	2013-03-27 08:08	R E G D
python.html	19.46 KB	-rw-r--r--	2013-03-27 08:08	R E G D
redhat.gif	0.681 KB	-rw-r--r--	2009-07-30 15:24	R E G D
searches.html	7.382 KB	-rw-r--r--	2009-07-30 15:24	R E G D
smallfootonly.gif	2.707 KB	-rw-r--r--	2009-07-30 15:24	R E G D
structure.gif	5.429 KB	-rw-r--r--	2009-07-30 15:24	R E G D
threads.html	6.94 KB	-rw-r--r--	2013-03-27 08:08	R E G D
tree.html	7.726 KB	-rw-r--r--	2013-03-27 08:08	R E G D
upgrade.html	12.344 KB	-rw-r--r--	2013-03-27 08:08	R E G D
w3c.png	1.98 KB	-rw-r--r--	2009-07-30 15:24	R E G D
xml.html	264.822 KB	-rw-r--r--	2013-03-23 02:31	R E G D
xmlcatalog_man.html	8.893 KB	-rw-r--r--	2013-02-27 05:08	R E G D
xmldtd.html	13.301 KB	-rw-r--r--	2013-03-27 08:08	R E G D
xmlio.html	12.483 KB	-rw-r--r--	2013-03-27 08:08	R E G D
xmllint.html	15.124 KB	-rw-r--r--	2009-07-30 15:24	R E G D
xmlmem.html	14.082 KB	-rw-r--r--	2013-03-27 08:08	R E G D
xmlreader.html	19.667 KB	-rw-r--r--	2009-07-30 15:24	R E G D
REQUEST EXIT

This document describes the use of the XmlTextReader streaming API added to libxml2 in version 2.5.0 . This API is closely modeled after the XmlTextReader and XmlReader classes of the C# language.

This tutorial will present the key points of this API, and working examples using both C and the Python bindings:

Introduction: why a new API

Libxml2 main API is tree based, where the parsing operation results in a document loaded completely in memory, and expose it as a tree of nodes all availble at the same time. This is very simple and quite powerful, but has the major limitation that the size of the document that can be hamdled is limited by the size of the memory available. Libxml2 also provide a SAX based API, but that version was designed upon one of the early expat version of SAX, SAX is also not formally defined for C. SAX basically work by registering callbacks which are called directly by the parser as it progresses through the document streams. The problem is that this programming model is relatively complex, not well standardized, cannot provide validation directly, makes entity, namespace and base processing relatively hard.

The XmlTextReader API from C# provides a far simpler programming model. The API acts as a cursor going forward on the document stream and stopping at each node in the way. The user's code keeps control of the progress and simply calls a Read() function repeatedly to progress to each node in sequence in document order. There is direct support for namespaces, xml:base, entity handling and adding DTD validation on top of it was relatively simple. This API is really close to the DOM Core specification This provides a far more standard, easy to use and powerful API than the existing SAX. Moreover integrating extension features based on the tree seems relatively easy.

In a nutshell the XmlTextReader API provides a simpler, more standard and more extensible interface to handle large documents than the existing SAX version.

Walking a simple tree

Basically the XmlTextReader API is a forward only tree walking interface. The basic steps are:

The only things worth adding are that the xmlTextReader is abstracted as a class like in C# with the same method names (but the properties are currently accessed with methods) and that one doesn't need to free the reader at the end of the processing. It will get garbage collected once all references have disapeared.

Extracting information for the current node

So far the example code did not indicate how information was extracted from the reader. It was abstrated as a call to the processNode() routine, with the reader as the argument. At each invocation, the parser is stopped on a given node and the reader can be used to query those node properties. Each Property is available at the C level as a function taking a single xmlTextReaderPtr argument whose name is xmlTextReaderProperty , if the return type is an xmlChar * string then it must be deallocated with xmlFree() to avoid leaks. For the Python interface, there is a Property method to the reader class that can be called on the instance. The list of the properties is based on the C# XmlTextReader class set of properties and methods:

Let's look first at a small example to get this in practice by redefining the processNode() function in the Python example:

and look at the result of calling streamFile("tst.xml") for various content of the XML test file.

Only one node is found, its depth is 0, type 1 indicate an element start, of name "doc" and it is empty. Trying now with "<doc></doc>" instead leads to:

The document root node is not flagged as empty anymore and both a start and an end of element are detected. The following document shows how character data are reported:

The equivalent routine for processNode() as used by xmllint --stream --debug is the following and can be found in the xmllint.c module in the source distribution:

Extracting information for the attributes

The previous examples don't indicate how attributes are processed. The simple test "<doc a="b"/>" provides the following result:

This proves that attribute nodes are not traversed by default. The HasAttributes property allow to detect their presence. To check their content the API has special instructions. Basically two kinds of operations are possible:

In both case the attribute can be designed either by its position in the list of attribute (MoveToAttributeNo or GetAttributeNo) or by their name (and namespace):

Validating a document

Libxml2 implementation adds some extra features on top of the XmlTextReader API. The main one is the ability to DTD validate the parsed document progressively. This is simply the activation of the associated feature of the parser used by the reader structure. There are a few options available defined as the enum xmlParserProperties in the libxml/xmlreader.h header file:

The GetParserProp() and SetParserProp() methods can then be used to get and set the values of those parser properties of the reader. For example

This routine will parse and validate the file. Error messages can be captured by registering an error handler. See python/tests/reader2.py for more complete Python examples. At the C level the equivalent call to cativate the validation feature is just:

Entities substitution

By default the xmlReader will report entities as such and not replace them with their content. This default behaviour can however be overriden using:

Relax-NG Validation

Libxml2 can now validate the document being read using the xmlReader using Relax-NG schemas. While the Relax NG validator can't always work in a streamable mode, only subsets which cannot be reduced to regular expressions need to have their subtree expanded for validation. In practice it means that, unless the schemas for the top level element content is not expressable as a regexp, only chunk of the document needs to be parsed while validating.

Example, assuming the reader has already being created and that the schema string contains the Relax-NG schemas:

Mixing the reader and tree or XPath operations

While the reader is a streaming interface, its underlying implementation is based on the DOM builder of libxml2. As a result it is relatively simple to mix operations based on both models under some constraints. To do so the reader has an Expand() operation allowing to grow the subtree under the current node. It returns a pointer to a standard node which can be manipulated in the usual ways. The node will get all its ancestors and the full subtree available. Usual operations like XPath queries can be used on that reduced view of the document. Here is an example extracted from reader5.py in the sources which extract and prints the bibliography for the "Dragon" compiler book from the XML 1.0 recommendation:

Note, however that the node instance returned by the Expand() call is only valid until the next Read() operation. The Expand() operation does not affects the Read() ones, however usually once processed the full subtree is not useful anymore, and the Next() operation allows to skip it completely and process to the successor or return 0 if the document end is reached.