Processing large XML documents using TransacXML

Introduction

This document describes issues to take into account when processing large XML documents using the TransacXML patterns.

DOM and memory usage

The Document Object Model (DOM) is an API for accessing and manipulating XML documents. DOM specifies how an XML document is represented as a logical tree and DOM contains methods for building documents, navigating their structure, and adding, deleting, and updating nodes (elements, attributes, etc).

Websydian TransacXML uses a DOM implementation to process and maintain XML documents. In order for the DOM implementation to process the XML document, the entire XML document is loaded into memory as a tree structure. This sets an upper limit to the size of the XML documents the DOM implementation can process.

TransacXML and memory usage

The implementation of the DOM API used in the TransacXML patterns results in creation of object references to XML nodes. All object references are deleted from memory when the function DomServices.DestroyDocument is called, but if a large XML document is processed it may result in a lot of object references being created potentially resulting in memory exhaustion.

To avoid this the function DomServices.DeleteObjectReference can be used to delete object references that are no longer used.

...

Call DomInterfaces.Document.createComment

I InputParameters<ObjectStoreReference> Local<ObjectStoreReference>

I InputParameters<ObjectReference> Local<ObjectDocument>
I InputParameters<data> <MyData.Comment>

Call DomInterfaces.Node.appendChild

I InputParameters<ObjectStoreReference> Local<ObjectStoreReference>
I InputParameters<ObjectReference> Local<ObjectDocument>
I InputParameters<newChild> DomInterfaces.Document.createComment/Output<ObjectComment>

Call DomServices.DeleteObjectReference

I InputParameters<ObjectStoreReference> Local<ObjectStoreReference>
I InputParameters<ObjectReference> DomInterfaces.Document.createComment/Output<ObjectComment>

...

The call to DomInterfaces.Document.createComment creates a comment node. In addition to this an object reference to the comment node is created so the comment node can be referenced across Websydian TransacXML functions.
When the comment node has been added to the XML document using DomInterfaces.Node.appendChild the object reference can safely be deleted.

Please note that deleting an object reference does not delete the object! In the above example only the reference to the comment is deleted. The comment node still exists in the XML document and can be retrieved using the DOM navigation functions.

To delete nodes from the XML document use the appropriate DOM functions to do this, e.g. DomInterfaces.Node.removeChild, DomInterfaces.Element.removeAttributeNode, ...