(**
===============================================================================================
Name : LibXmlParser
===============================================================================================
Project : All Projects
===============================================================================================
Subject : Progressive XML Parser for all types of XML Files
===============================================================================================
Author : Stefan Heymann
Eschenweg 3
72076 Tübingen
GERMANY
E-Mail: stefan@destructor.de
URL: www.destructor.de
===============================================================================================
Source, Legals ("Licence")
--------------------------
The official site to get this parser is http://www.destructor.de/
Usage and Distribution of this Source Code is ruled by the
"Destructor.de Source code Licence" (DSL) which comes with this file or
can be downloaded at http://www.destructor.de/
IN SHORT: Usage and distribution of this source code is free.
You use it completely on your own risk.
Postcardware
------------
If you like this code, please send a postcard of your city to my above address.
===============================================================================================
!!! All parts of this code which are not finished or not conforming exactly to
the XmlSpec are marked with three exclamation marks
-!- Parts where the parser may be able to detect errors in the document's syntax are
marked with the dash-exlamation mark-dash sequence.
===============================================================================================
Terminology:
------------
- Start: Start of a buffer part
- Final: End (last character) of a buffer part
- DTD: Document Type Definition
- DTDc: Document Type Declaration
- XMLSpec: The current W3C XML Recommendation (version 1.0 as of 1998-02-10), Chapter No.
- Cur*: Fields concerning the "Current" part passed back by the "Scan" method
===============================================================================================
Scanning the XML document
-------------------------
- Create TXmlParser Instance MyXml := TXmlParser.Create;
- Load XML Document MyXml.LoadFromFile (Filename);
- Start Scanning MyXml.StartScan;
- Scan Loop WHILE MyXml.Scan DO
- Test for Part Type CASE MyXml.CurPartType OF
- Handle Parts ... : ;;;
- Handle Parts ... : ;;;
- Handle Parts ... : ;;;
END;
- Destroy MyXml.Free;
===============================================================================================
Loading the XML document
------------------------
You can load the XML document from a file with the "LoadFromFile" method.
It is beyond the scope of this parser to perform HTTP or FTP accesses. If you want your
application to handle such requests (URLs), you can load the XML via HTTP or FTP or whatever
protocol and hand over the data buffer using the "LoadFromBuffer" or "SetBuffer" method.
"LoadFromBuffer" loads the internal buffer of TXmlParser with the given null-terminated
string, thereby creating a copy of that buffer.
"SetBuffer" just takes the pointer to another buffer, which means that the given
buffer pointer must be valid while the document is accessed via TXmlParser.
===============================================================================================
Encodings:
----------
This XML parser kind of "understands" the following encodings:
- UTF-8
- ISO-8859-1
- Windows-1252
Any flavor of multi-byte characters (and this includes UTF-16) is not supported. Sorry.
Every string which has to be passed to the application passes the virtual method
"TranslateEncoding" which translates the string from the current encoding (stored in
"CurEncoding") into the encoding the application wishes to receive.
The "TranslateEncoding" method that is built into TXmlParser assumes that the application
wants to receive Windows ANSI (Windows-1252, about the same as ISO-8859-1) and is able
to convert UTF-8 and ISO-8859-1 encodings.
For other source and target encodings, you will have to override "TranslateEncoding".
===============================================================================================
Buffer Handling
---------------
- The document must be loaded completely into a piece of RAM
- All character positions are referenced by PChar pointers
- The TXmlParser instance can either "own" the buffer itself (then, FBufferSize is > 0)
or reference the buffer of another instance or object (then, FBuffersize is 0 and
FBuffer is not NIL)
- The Property DocBuffer passes back a pointer to the first byte of the document. If there
is no document stored (FBuffer is NIL), the DocBuffer returns a pointer to a NULL character.
===============================================================================================
Whitespace Handling
-------------------
The TXmlParser property "PackSpaces" determines how Whitespace is returned in Text Content:
While PackSpaces is true, all leading and trailing whitespace characters are trimmed of, all
Whitespace is converted to Space #x20 characters and contiguous Whitespace characters are
compressed to one.
If the "Scan" method reports a ptContent part, the application can get the original text
with all whitespace characters by extracting the characters from "CurStart" to "CurFinal".
If the application detects an xml:space attribute, it can set "PackSpaces" accordingly or
use CurStart/CurFinal.
Please note that TXmlParser does _not_ normalize Line Breaks to single LineFeed characters
as the XmlSpec requires (XmlSpec 2.11).
The xml:space attribute is not handled by TXmlParser. This is on behalf of the application.
===============================================================================================
Non-XML-Conforming
------------------
TXmlParser does not conform 100 % exactly to the XmlSpec:
- UTF-16 is not supported (XmlSpec 2.2)
(Workaround: Convert UTF-16 to UTF-8 and hand the buffer over to TXmlParser)
- As the parser only works with single byte strings, all Unicode characters > 255
can currently not be handled correctly.
- Line breaks are not normalized to single Linefeed #x0A characters (XmlSpec 2.11)
(Workaround: The Application can access the text contents on its own [CurStart, CurFinal],
thereby applying every normalization it wishes to)
- The attribute value normalization does not work exactly as defined in the
Second Edition of the XML 1.0 specification.
- See also the code parts marked with three consecutive exclamation marks. These are
parts which are not finished in the current code release.
This list may be incomplete, so it may grow if I get to know any other points.
As work on the parser proceeds, this list may also shrink.
===============================================================================================
Things Todo
-----------
- Introduce a new event/callback which is called when there is an unresolvable
entity or character reference
- Support Unicode
- Use Streams instead of reading the whole XML into memory
===============================================================================================
Change History, Version numbers
-------------------------------
The Date is given in ISO Year-Month-Day (YYYY-MM-DD) order.
Versions are counted from 1.0.0 beginning with the version from 2000-03-16.
Unreleased versions don't get a version number.
Date Author Version Changes
-----------------------------------------------------------------------------------------------
2000-03-16 HeySt 1.0.0 Start
2000-03-28 HeySt 1.0.1 Initial Publishing of TXmlParser on the destructor.de Web Site
2000-03-30 HeySt 1.0.2 TXmlParser.AnalyzeCData: Call "TranslateEncoding" for CurContent
2000-03-31 HeySt 1.0.3 Deleted the StrPosE function (was not needed anyway)
2000-04-04 HeySt 1.0.4 TDtdElementRec modified: Start/Final for all Elements;
Should be backwards compatible.
AnalyzeDtdc: Set CurPartType to ptDtdc
2000-04-23 HeySt 1.0.5 New class TObjectList. Eliminated reference to the Delphi 5
"Contnrs" unit so LibXmlParser is Delphi 4 compatible.
2000-07-03 HeySt 1.0.6 TNvpNode: Added Constructor
2000-07-11 HeySt 1.0.7 Removed "Windows" from USES clause
Added three-exclamation-mark comments for Utf8ToAnsi/AnsiToUtf8
Added three-exclamation-mark comments for CHR function calls
2000-07-23 HeySt 1.0.8 TXmlParser.Clear: CurAttr.Clear; EntityStack.Clear;
(This was not a bug; just defensive programming)
2000-07-29 HeySt 1.0.9 TNvpList: Added methods: Node(Index), Value(Index), Name(Index);
2000-10-07 HeySt Introduced Conditional Defines
Uses Contnrs unit and its TObjectList class again for
Delphi 5 and newer versions
2001-01-30 HeySt Introduced Version Numbering
Made LoadFromFile and LoadFromBuffer BOOLEAN functions
Introduced FileMode parameter for LoadFromFile
BugFix: TAttrList.Analyze: Must add CWhitespace to ExtractName call
Comments worked over
2001-02-28 HeySt 1.0.10 Completely worked over and tested the UTF-8 functions
Fixed a bug in TXmlParser.Scan which caused it to start over when it
was called after the end of scanning, resulting in an endless loop
TEntityStack is now a TObjectList instead of TList
2001-07-03 HeySt 1.0.11 Updated Compiler Version IFDEFs for Kylix
2001-07-11 HeySt 1.0.12 New TCustomXmlScanner component (taken over from LibXmlComps.pas)
2001-07-14 HeySt 1.0.13 Bugfix TCustomXmlScanner.FOnTranslateEncoding
2001-10-22 HeySt Don't clear CurName anymore when the parser finds a CDATA section.
2001-12-03 HeySt 1.0.14 TObjectList.Clear: Make call to INHERITED method (fixes a memory leak)
2001-12-05 HeySt 1.0.15 TObjectList.Clear: removed call to INHERITED method
TObjectList.Destroy: Inserted SetCapacity call.
Reduces need for frequent re-allocation of pointer buffer
Dedicated to my father, Theodor Heymann
2002-06-26 HeySt 1.0.16 TXmlParser.Scan: Fixed a bug with PIs whose name is beginning
with 'xml'. Thanks to Uwe Kamm for submitting this bug.
The CurEncoding property is now always in uppercase letters (the XML
spec wants it to be treated case independently so when it's uppercase
comparisons are faster)
2002-03-04 HeySt 1.0.17 Included an IFDEF for Delphi 7 (VER150) and Kylix
There is a new symbol HAS_CONTNRS_UNIT which is used now to
distinguish between IDEs which come with the Contnrs unit and
those that don't.
*)
UNIT libxmlparser;
{$I jedi-sdl.inc}
INTERFACE
USES
SysUtils, Classes,
(*$IFDEF HAS_CONTNRS_UNIT *) // The Contnrs Unit was introduced in Delphi 5
Contnrs,
(*$ENDIF*)
Math;
CONST
CVersion = '1.0.17'; // This variable will be updated for every release
// (I hope, I won't forget to do it everytime ...)
TYPE
TPartType = // --- Document Part Types
(ptNone, // Nothing
ptXmlProlog, // XML Prolog XmlSpec 2.8 / 4.3.1
ptComment, // Comment XmlSpec 2.5
ptPI, // Processing Instruction XmlSpec 2.6
ptDtdc, // Document Type Declaration XmlSpec 2.8
ptStartTag, // Start Tag XmlSpec 3.1
ptEmptyTag, // Empty-Element Tag XmlSpec 3.1
ptEndTag, // End Tag XmlSpec 3.1
ptContent, // Text Content between Tags
ptCData); // CDATA Section XmlSpec 2.7
TDtdElemType = // --- DTD Elements
(deElement, // !ELEMENT declaration
deAttList, // !ATTLIST declaration
deEntity, // !ENTITY declaration
deNotation, // !NOTATION declaration
dePI, // PI in DTD
deComment, // Comment in DTD
deError); // Error found in the DTD
TYPE
TAttrList = CLASS;
TEntityStack = CLASS;
TNvpList = CLASS;
TElemDef = CLASS;
TElemList = CLASS;
TEntityDef = CLASS;
TNotationDef = CLASS;
TDtdElementRec = RECORD // --- This Record is returned by the DTD parser callback function
Start, Final : PChar; // Start/End of the Element's Declaration
CASE ElementType : TDtdElemType OF // Type of the Element
deElement, //
deAttList : (ElemDef : TElemDef); //
deEntity : (EntityDef : TEntityDef); //
deNotation : (NotationDef : TNotationDef); //
dePI : (Target : PChar; //
Content : PChar;
AttrList : TAttrList);
deError : (Pos : PChar); // Error
// deComment : ((No additional fields here)); //
END;
TXmlParser = CLASS // --- Internal Properties and Methods
PROTECTED
FBuffer : PChar; // NIL if there is no buffer available
FBufferSize : INTEGER; // 0 if the buffer is not owned by the Document instance
FSource : STRING; // Name of Source of document. Filename for Documents loaded with LoadFromFile
FXmlVersion : STRING; // XML version from Document header. Default is '1.0'
FEncoding : STRING; // Encoding from Document header. Default is 'UTF-8'
FStandalone : BOOLEAN; // Standalone declaration from Document header. Default is 'yes'
FRootName : STRING; // Name of the Root Element (= DTD name)
FDtdcFinal : PChar; // Pointer to the '>' character terminating the DTD declaration
FNormalize : BOOLEAN; // If true: Pack Whitespace and don't return empty contents
EntityStack : TEntityStack; // Entity Stack for Parameter and General Entities
FCurEncoding : STRING; // Current Encoding during parsing (always uppercase)
PROCEDURE AnalyzeProlog; // Analyze XML Prolog or Text Declaration
PROCEDURE AnalyzeComment (Start : PChar; VAR Final : PChar); // Analyze Comments
PROCEDURE AnalyzePI (Start : PChar; VAR Final : PChar); // Analyze Processing Instructions (PI)
PROCEDURE AnalyzeDtdc; // Analyze Document Type Declaration
PROCEDURE AnalyzeDtdElements (Start : PChar; VAR Final : PChar); // Analyze DTD declarations
PROCEDURE AnalyzeTag; // Analyze Start/End/Empty-Element Tags
PROCEDURE AnalyzeCData; // Analyze CDATA Sections
PROCEDURE AnalyzeText (VAR IsDone : BOOLEAN); // Analyze Text Content between Tags
PROCEDURE AnalyzeElementDecl (Start : PChar; VAR Final : PChar);
PROCEDURE AnalyzeAttListDecl (Start : PChar; VAR Final : PChar);
PROCEDURE AnalyzeEntityDecl (Start : PChar; VAR Final : PChar);
PROCEDURE AnalyzeNotationDecl (Start : PChar; VAR Final : PChar);
PROCEDURE PushPE (VAR Start : PChar);
PROCEDURE ReplaceCharacterEntities (VAR Str : STRING);
PROCEDURE ReplaceParameterEntities (VAR Str : STRING);
PROCEDURE ReplaceGeneralEntities (VAR Str : STRING);
FUNCTION GetDocBuffer : PChar; // Returns FBuffer or a pointer to a NUL char if Buffer is empty
PUBLIC // --- Document Properties
PROPERTY XmlVersion : STRING READ FXmlVersion; // XML version from the Document Prolog
PROPERTY Encoding : STRING READ FEncoding; // Document Encoding from Prolog
PROPERTY Standalone : BOOLEAN READ FStandalone; // Standalone Declaration from Prolog
PROPERTY RootName : STRING READ FRootName; // Name of the Root Element
PROPERTY Normalize : BOOLEAN READ FNormalize WRITE FNormalize; // True if Content is to be normalized
PROPERTY Source : STRING READ FSource; // Name of Document Source (Filename)
PROPERTY DocBuffer : PChar READ GetDocBuffer; // Returns document buffer
PUBLIC // --- DTD Objects
Elements : TElemList; // Elements: List of TElemDef (contains Attribute Definitions)
Entities : TNvpList; // General Entities: List of TEntityDef
ParEntities : TNvpList; // Parameter Entities: List of TEntityDef
Notations : TNvpList; // Notations: List of TNotationDef
PUBLIC
CONSTRUCTOR Create;
DESTRUCTOR Destroy; OVERRIDE;
// --- Document Handling
FUNCTION LoadFromFile (Filename : STRING;
FileMode : INTEGER = fmOpenRead OR fmShareDenyNone) : BOOLEAN;
// Loads Document from given file
FUNCTION LoadFromBuffer (Buffer : PChar) : BOOLEAN; // Loads Document from another buffer
PROCEDURE SetBuffer (Buffer : PChar); // References another buffer
PROCEDURE Clear; // Clear Document
PUBLIC
// --- Scanning through the document
CurPartType : TPartType; // Current Type
CurName : STRING; // Current Name
CurContent : STRING; // Current Normalized Content
CurStart : PChar; // Current First character
CurFinal : PChar; // Current Last character
CurAttr : TAttrList; // Current Attribute List
PROPERTY CurEncoding : STRING READ FCurEncoding; // Current Encoding
PROCEDURE StartScan;
FUNCTION Scan : BOOLEAN;
// --- Events / Callbacks
FUNCTION LoadExternalEntity (SystemId, PublicId,
Notation : STRING) : TXmlParser; VIRTUAL;
FUNCTION TranslateEncoding (CONST Source : STRING) : STRING; VIRTUAL;
PROCEDURE DtdElementFound (DtdElementRec : TDtdElementRec); VIRTUAL;
END;
TValueType = // --- Attribute Value Type
(vtNormal, // Normal specified Attribute
vtImplied, // #IMPLIED attribute value
vtFixed, // #FIXED attribute value
vtDefault); // Attribute value from default value in !ATTLIST declaration
TAttrDefault = // --- Attribute Default Type
(adDefault, // Normal default value
adRequired, // #REQUIRED attribute
adImplied, // #IMPLIED attribute
adFixed); // #FIXED attribute
TAttrType = // --- Type of attribute
(atUnknown, // Unknown type
atCData, // Character data only
atID, // ID
atIdRef, // ID Reference
atIdRefs, // Several ID References, separated by Whitespace
atEntity, // Name of an unparsed Entity
atEntities, // Several unparsed Entity names, separated by Whitespace
atNmToken, // Name Token
atNmTokens, // Several Name Tokens, separated by Whitespace
atNotation, // A selection of Notation names (Unparsed Entity)
atEnumeration); // Enumeration
TElemType = // --- Element content type
(etEmpty, // Element is always empty
etAny, // Element can have any mixture of PCDATA and any elements
etChildren, // Element must contain only elements
etMixed); // Mixed PCDATA and elements
(*$IFDEF HAS_CONTNRS_UNIT *)
TObjectList = Contnrs.TObjectList; // Re-Export this identifier
(*$ELSE *)
TObjectList = CLASS (TList)
DESTRUCTOR Destroy; OVERRIDE;
PROCEDURE Delete (Index : INTEGER);
PROCEDURE Clear; OVERRIDE;
END;
(*$ENDIF *)
TNvpNode = CLASS // Name-Value Pair Node
Name : STRING;
Value : STRING;
CONSTRUCTOR Create (TheName : STRING = ''; TheValue : STRING = '');
END;
TNvpList = CLASS (TObjectList) // Name-Value Pair List
PROCEDURE Add (Node : TNvpNode);
FUNCTION Node (Name : STRING) : TNvpNode; OVERLOAD;
FUNCTION Node (Index : INTEGER) : TNvpNode; OVERLOAD;
FUNCTION Value (Name : STRING) : STRING; OVERLOAD;
FUNCTION Value (Index : INTEGER) : STRING; OVERLOAD;
FUNCTION Name (Index : INTEGER) : STRING;
END;
TAttr = CLASS (TNvpNode) // Attribute of a Start-Tag or Empty-Element-Tag
ValueType : TValueType;
AttrType : TAttrType;
END;
TAttrList = CLASS (TNvpList) // List of Attributes
PROCEDURE Analyze (Start : PChar; VAR Final : PChar);
END;
TEntityStack = CLASS (TObjectList) // Stack where current position is stored before parsing entities
PROTECTED
Owner : TXmlParser;
PUBLIC
CONSTRUCTOR Create (TheOwner : TXmlParser);
PROCEDURE Push (LastPos : PChar); OVERLOAD;
PROCEDURE Push (Instance : TObject; LastPos : PChar); OVERLOAD;
FUNCTION Pop : PChar; // Returns next char or NIL if EOF is reached. Frees Instance.
END;
TAttrDef = CLASS (TNvpNode) // Represents a ';
// --- Name Constants for the above enumeration types
CPartType_Name : ARRAY [TPartType] OF STRING =
('', 'XML Prolog', 'Comment', 'PI',
'DTD Declaration', 'Start Tag', 'Empty Tag', 'End Tag',
'Text', 'CDATA');
CValueType_Name : ARRAY [TValueType] OF STRING = ('Normal', 'Implied', 'Fixed', 'Default');
CAttrDefault_Name : ARRAY [TAttrDefault] OF STRING = ('Default', 'Required', 'Implied', 'Fixed');
CElemType_Name : ARRAY [TElemType] OF STRING = ('Empty', 'Any', 'Childs only', 'Mixed');
CAttrType_Name : ARRAY [TAttrType] OF STRING = ('Unknown', 'CDATA',
'ID', 'IDREF', 'IDREFS',
'ENTITY', 'ENTITIES',
'NMTOKEN', 'NMTOKENS',
'Notation', 'Enumeration');
FUNCTION ConvertWs (Source: STRING; PackWs: BOOLEAN) : STRING; // Convert WS to spaces #x20
PROCEDURE SetStringSF (VAR S : STRING; BufferStart, BufferFinal : PChar); // SetString by Start/Final of buffer
FUNCTION StrSFPas (Start, Finish : PChar) : STRING; // Convert buffer part to Pascal string
FUNCTION TrimWs (Source : STRING) : STRING; // Trim Whitespace
FUNCTION AnsiToUtf8 (Source : ANSISTRING) : STRING; // Convert Win-1252 to UTF-8
FUNCTION Utf8ToAnsi (Source : STRING; UnknownChar : CHAR = '¿') : ANSISTRING; // Convert UTF-8 to Win-1252
(*
===============================================================================================
TCustomXmlScanner event based component wrapper for TXmlParser
===============================================================================================
*)
TYPE
TCustomXmlScanner = CLASS;
TXmlPrologEvent = PROCEDURE (Sender : TObject; XmlVersion, Encoding: STRING; Standalone : BOOLEAN) OF OBJECT;
TCommentEvent = PROCEDURE (Sender : TObject; Comment : STRING) OF OBJECT;
TPIEvent = PROCEDURE (Sender : TObject; Target, Content: STRING; Attributes : TAttrList) OF OBJECT;
TDtdEvent = PROCEDURE (Sender : TObject; RootElementName : STRING) OF OBJECT;
TStartTagEvent = PROCEDURE (Sender : TObject; TagName : STRING; Attributes : TAttrList) OF OBJECT;
TEndTagEvent = PROCEDURE (Sender : TObject; TagName : STRING) OF OBJECT;
TContentEvent = PROCEDURE (Sender : TObject; Content : STRING) OF OBJECT;
TElementEvent = PROCEDURE (Sender : TObject; ElemDef : TElemDef) OF OBJECT;
TEntityEvent = PROCEDURE (Sender : TObject; EntityDef : TEntityDef) OF OBJECT;
TNotationEvent = PROCEDURE (Sender : TObject; NotationDef : TNotationDef) OF OBJECT;
TErrorEvent = PROCEDURE (Sender : TObject; ErrorPos : PChar) OF OBJECT;
TExternalEvent = PROCEDURE (Sender : TObject; SystemId, PublicId, NotationId : STRING;
VAR Result : TXmlParser) OF OBJECT;
TEncodingEvent = FUNCTION (Sender : TObject; CurrentEncoding, Source : STRING) : STRING OF OBJECT;
TCustomXmlScanner = CLASS (TComponent)
PROTECTED
FXmlParser : TXmlParser;
FOnXmlProlog : TXmlPrologEvent;
FOnComment : TCommentEvent;
FOnPI : TPIEvent;
FOnDtdRead : TDtdEvent;
FOnStartTag : TStartTagEvent;
FOnEmptyTag : TStartTagEvent;
FOnEndTag : TEndTagEvent;
FOnContent : TContentEvent;
FOnCData : TContentEvent;
FOnElement : TElementEvent;
FOnAttList : TElementEvent;
FOnEntity : TEntityEvent;
FOnNotation : TNotationEvent;
FOnDtdError : TErrorEvent;
FOnLoadExternal : TExternalEvent;
FOnTranslateEncoding : TEncodingEvent;
FStopParser : BOOLEAN;
FUNCTION GetNormalize : BOOLEAN;
PROCEDURE SetNormalize (Value : BOOLEAN);
PROCEDURE WhenXmlProlog(XmlVersion, Encoding: STRING; Standalone : BOOLEAN); VIRTUAL;
PROCEDURE WhenComment (Comment : STRING); VIRTUAL;
PROCEDURE WhenPI (Target, Content: STRING; Attributes : TAttrList); VIRTUAL;
PROCEDURE WhenDtdRead (RootElementName : STRING); VIRTUAL;
PROCEDURE WhenStartTag (TagName : STRING; Attributes : TAttrList); VIRTUAL;
PROCEDURE WhenEmptyTag (TagName : STRING; Attributes : TAttrList); VIRTUAL;
PROCEDURE WhenEndTag (TagName : STRING); VIRTUAL;
PROCEDURE WhenContent (Content : STRING); VIRTUAL;
PROCEDURE WhenCData (Content : STRING); VIRTUAL;
PROCEDURE WhenElement (ElemDef : TElemDef); VIRTUAL;
PROCEDURE WhenAttList (ElemDef : TElemDef); VIRTUAL;
PROCEDURE WhenEntity (EntityDef : TEntityDef); VIRTUAL;
PROCEDURE WhenNotation (NotationDef : TNotationDef); VIRTUAL;
PROCEDURE WhenDtdError (ErrorPos : PChar); VIRTUAL;
PUBLIC
CONSTRUCTOR Create (AOwner: TComponent); OVERRIDE;
DESTRUCTOR Destroy; OVERRIDE;
PROCEDURE LoadFromFile (Filename : TFilename); // Load XML Document from file
PROCEDURE LoadFromBuffer (Buffer : PChar); // Load XML Document from buffer
PROCEDURE SetBuffer (Buffer : PChar); // Refer to Buffer
FUNCTION GetFilename : TFilename;
PROCEDURE Execute; // Perform scanning
PROTECTED
PROPERTY XmlParser : TXmlParser READ FXmlParser;
PROPERTY StopParser : BOOLEAN READ FStopParser WRITE FStopParser;
PROPERTY Filename : TFilename READ GetFilename WRITE LoadFromFile;
PROPERTY Normalize : BOOLEAN READ GetNormalize WRITE SetNormalize;
PROPERTY OnXmlProlog : TXmlPrologEvent READ FOnXmlProlog WRITE FOnXmlProlog;
PROPERTY OnComment : TCommentEvent READ FOnComment WRITE FOnComment;
PROPERTY OnPI : TPIEvent READ FOnPI WRITE FOnPI;
PROPERTY OnDtdRead : TDtdEvent READ FOnDtdRead WRITE FOnDtdRead;
PROPERTY OnStartTag : TStartTagEvent READ FOnStartTag WRITE FOnStartTag;
PROPERTY OnEmptyTag : TStartTagEvent READ FOnEmptyTag WRITE FOnEmptyTag;
PROPERTY OnEndTag : TEndTagEvent READ FOnEndTag WRITE FOnEndTag;
PROPERTY OnContent : TContentEvent READ FOnContent WRITE FOnContent;
PROPERTY OnCData : TContentEvent READ FOnCData WRITE FOnCData;
PROPERTY OnElement : TElementEvent READ FOnElement WRITE FOnElement;
PROPERTY OnAttList : TElementEvent READ FOnAttList WRITE FOnAttList;
PROPERTY OnEntity : TEntityEvent READ FOnEntity WRITE FOnEntity;
PROPERTY OnNotation : TNotationEvent READ FOnNotation WRITE FOnNotation;
PROPERTY OnDtdError : TErrorEvent READ FOnDtdError WRITE FOnDtdError;
PROPERTY OnLoadExternal : TExternalEvent READ FOnLoadExternal WRITE FOnLoadExternal;
PROPERTY OnTranslateEncoding : TEncodingEvent READ FOnTranslateEncoding WRITE FOnTranslateEncoding;
END;
(*
===============================================================================================
IMPLEMENTATION
===============================================================================================
*)
IMPLEMENTATION
(*
===============================================================================================
Unicode and UTF-8 stuff
===============================================================================================
*)
CONST
// --- Character Translation Table for Unicode <-> Win-1252
WIN1252_UNICODE : ARRAY [$00..$FF] OF WORD = (
$0000, $0001, $0002, $0003, $0004, $0005, $0006, $0007, $0008, $0009,
$000A, $000B, $000C, $000D, $000E, $000F, $0010, $0011, $0012, $0013,
$0014, $0015, $0016, $0017, $0018, $0019, $001A, $001B, $001C, $001D,
$001E, $001F, $0020, $0021, $0022, $0023, $0024, $0025, $0026, $0027,
$0028, $0029, $002A, $002B, $002C, $002D, $002E, $002F, $0030, $0031,
$0032, $0033, $0034, $0035, $0036, $0037, $0038, $0039, $003A, $003B,
$003C, $003D, $003E, $003F, $0040, $0041, $0042, $0043, $0044, $0045,
$0046, $0047, $0048, $0049, $004A, $004B, $004C, $004D, $004E, $004F,
$0050, $0051, $0052, $0053, $0054, $0055, $0056, $0057, $0058, $0059,
$005A, $005B, $005C, $005D, $005E, $005F, $0060, $0061, $0062, $0063,
$0064, $0065, $0066, $0067, $0068, $0069, $006A, $006B, $006C, $006D,
$006E, $006F, $0070, $0071, $0072, $0073, $0074, $0075, $0076, $0077,
$0078, $0079, $007A, $007B, $007C, $007D, $007E, $007F,
$20AC, $0081, $201A, $0192, $201E, $2026, $2020, $2021, $02C6, $2030,
$0160, $2039, $0152, $008D, $017D, $008F, $0090, $2018, $2019, $201C,
$201D, $2022, $2013, $2014, $02DC, $2122, $0161, $203A, $0153, $009D,
$017E, $0178, $00A0, $00A1, $00A2, $00A3, $00A4, $00A5, $00A6, $00A7,
$00A8, $00A9, $00AA, $00AB, $00AC, $00AD, $00AE, $00AF, $00B0, $00B1,
$00B2, $00B3, $00B4, $00B5, $00B6, $00B7, $00B8, $00B9, $00BA, $00BB,
$00BC, $00BD, $00BE, $00BF, $00C0, $00C1, $00C2, $00C3, $00C4, $00C5,
$00C6, $00C7, $00C8, $00C9, $00CA, $00CB, $00CC, $00CD, $00CE, $00CF,
$00D0, $00D1, $00D2, $00D3, $00D4, $00D5, $00D6, $00D7, $00D8, $00D9,
$00DA, $00DB, $00DC, $00DD, $00DE, $00DF, $00E0, $00E1, $00E2, $00E3,
$00E4, $00E5, $00E6, $00E7, $00E8, $00E9, $00EA, $00EB, $00EC, $00ED,
$00EE, $00EF, $00F0, $00F1, $00F2, $00F3, $00F4, $00F5, $00F6, $00F7,
$00F8, $00F9, $00FA, $00FB, $00FC, $00FD, $00FE, $00FF);
(* UTF-8 (somewhat simplified)
-----
Character Range Byte sequence
--------------- -------------------------- (x=Bits from original character)
$0000..$007F 0xxxxxxx
$0080..$07FF 110xxxxx 10xxxxxx
$8000..$FFFF 1110xxxx 10xxxxxx 10xxxxxx
Example
--------
Transforming the Unicode character U+00E4 LATIN SMALL LETTER A WITH DIAERESIS ("ä"):
ISO-8859-1, Decimal 228
Win1252, Hex $E4
ANSI Bin 1110 0100
abcd efgh
UTF-8 Binary 1100xxab 10cdefgh
Binary 11000011 10100100
Hex $C3 $A4
Decimal 195 164
ANSI Ã ¤ *)
FUNCTION AnsiToUtf8 (Source : ANSISTRING) : STRING;
(* Converts the given Windows ANSI (Win1252) String to UTF-8. *)
VAR
I : INTEGER; // Loop counter
U : WORD; // Current Unicode value
Len : INTEGER; // Current real length of "Result" string
BEGIN
SetLength (Result, Length (Source) * 3); // Worst case
Len := 0;
FOR I := 1 TO Length (Source) DO BEGIN
U := WIN1252_UNICODE [ORD (Source [I])];
CASE U OF
$0000..$007F : BEGIN
INC (Len);
Result [Len] := CHR (U);
END;
$0080..$07FF : BEGIN
INC (Len);
Result [Len] := CHR ($C0 OR (U SHR 6));
INC (Len);
Result [Len] := CHR ($80 OR (U AND $3F));
END;
$0800..$FFFF : BEGIN
INC (Len);
Result [Len] := CHR ($E0 OR (U SHR 12));
INC (Len);
Result [Len] := CHR ($80 OR ((U SHR 6) AND $3F));
INC (Len);
Result [Len] := CHR ($80 OR (U AND $3F));
END;
END;
END;
SetLength (Result, Len);
END;
FUNCTION Utf8ToAnsi (Source : STRING; UnknownChar : CHAR = '¿') : ANSISTRING;
(* Converts the given UTF-8 String to Windows ANSI (Win-1252).
If a character can not be converted, the "UnknownChar" is inserted. *)
VAR
SourceLen : INTEGER; // Length of Source string
I, K : INTEGER;
A : BYTE; // Current ANSI character value
U : WORD;
Ch : CHAR; // Dest char
Len : INTEGER; // Current real length of "Result" string
BEGIN
SourceLen := Length (Source);
SetLength (Result, SourceLen); // Enough room to live
Len := 0;
I := 1;
WHILE I <= SourceLen DO BEGIN
A := ORD (Source [I]);
IF A < $80 THEN BEGIN // Range $0000..$007F
INC (Len);
Result [Len] := Source [I];
INC (I);
END
ELSE BEGIN // Determine U, Inc I
IF (A AND $E0 = $C0) AND (I < SourceLen) THEN BEGIN // Range $0080..$07FF
U := (WORD (A AND $1F) SHL 6) OR (ORD (Source [I+1]) AND $3F);
INC (I, 2);
END
ELSE IF (A AND $F0 = $E0) AND (I < SourceLen-1) THEN BEGIN // Range $0800..$FFFF
U := (WORD (A AND $0F) SHL 12) OR
(WORD (ORD (Source [I+1]) AND $3F) SHL 6) OR
( ORD (Source [I+2]) AND $3F);
INC (I, 3);
END
ELSE BEGIN // Unknown/unsupported
INC (I);
FOR K := 7 DOWNTO 0 DO
IF A AND (1 SHL K) = 0 THEN BEGIN
INC (I, (A SHR (K+1))-1);
BREAK;
END;
U := WIN1252_UNICODE [ORD (UnknownChar)];
END;
Ch := UnknownChar; // Retrieve ANSI char
FOR A := $00 TO $FF DO
IF WIN1252_UNICODE [A] = U THEN BEGIN
Ch := CHR (A);
BREAK;
END;
INC (Len);
Result [Len] := Ch;
END;
END;
SetLength (Result, Len);
END;
(*
===============================================================================================
"Special" Helper Functions
Don't ask me why. But including these functions makes the parser *DRAMATICALLY* faster
on my K6-233 machine. You can test it yourself just by commenting them out.
They do exactly the same as the Assembler routines defined in SysUtils.
(This is where you can see how great the Delphi compiler really is. The compiled code is
faster than hand-coded assembler!)
===============================================================================================
--> Just move this line below the StrScan function --> *)
FUNCTION StrPos (CONST Str, SearchStr : PChar) : PChar;
// Same functionality as SysUtils.StrPos
VAR
First : CHAR;
Len : INTEGER;
BEGIN
First := SearchStr^;
Len := StrLen (SearchStr);
Result := Str;
REPEAT
IF Result^ = First THEN
IF StrLComp (Result, SearchStr, Len) = 0 THEN BREAK;
IF Result^ = #0 THEN BEGIN
Result := NIL;
BREAK;
END;
INC (Result);
UNTIL FALSE;
END;
FUNCTION StrScan (CONST Start : PChar; CONST Ch : CHAR) : PChar;
// Same functionality as SysUtils.StrScan
BEGIN
Result := Start;
WHILE Result^ <> Ch DO BEGIN
IF Result^ = #0 THEN BEGIN
Result := NIL;
EXIT;
END;
INC (Result);
END;
END;
(*
===============================================================================================
Helper Functions
===============================================================================================
*)
FUNCTION DelChars (Source : STRING; CharsToDelete : TCharset) : STRING;
// Delete all "CharsToDelete" from the string
VAR
I : INTEGER;
BEGIN
Result := Source;
FOR I := Length (Result) DOWNTO 1 DO
IF Result [I] IN CharsToDelete THEN
Delete (Result, I, 1);
END;
FUNCTION TrimWs (Source : STRING) : STRING;
// Trimms off Whitespace characters from both ends of the string
VAR
I : INTEGER;
BEGIN
// --- Trim Left
I := 1;
WHILE (I <= Length (Source)) AND (Source [I] IN CWhitespace) DO
INC (I);
Result := Copy (Source, I, MaxInt);
// --- Trim Right
I := Length (Result);
WHILE (I > 1) AND (Result [I] IN CWhitespace) DO
DEC (I);
Delete (Result, I+1, Length (Result)-I);
END;
FUNCTION ConvertWs (Source: STRING; PackWs: BOOLEAN) : STRING;
// Converts all Whitespace characters to the Space #x20 character
// If "PackWs" is true, contiguous Whitespace characters are packed to one
VAR
I : INTEGER;
BEGIN
Result := Source;
FOR I := Length (Result) DOWNTO 1 DO
IF (Result [I] IN CWhitespace) THEN
IF PackWs AND (I > 1) AND (Result [I-1] IN CWhitespace)
THEN Delete (Result, I, 1)
ELSE Result [I] := #32;
END;
PROCEDURE SetStringSF (VAR S : STRING; BufferStart, BufferFinal : PChar);
BEGIN
SetString (S, BufferStart, BufferFinal-BufferStart+1);
END;
FUNCTION StrLPas (Start : PChar; Len : INTEGER) : STRING;
BEGIN
SetString (Result, Start, Len);
END;
FUNCTION StrSFPas (Start, Finish : PChar) : STRING;
BEGIN
SetString (Result, Start, Finish-Start+1);
END;
FUNCTION StrScanE (CONST Source : PChar; CONST CharToScanFor : CHAR) : PChar;
// If "CharToScanFor" is not found, StrScanE returns the last char of the
// buffer instead of NIL
BEGIN
Result := StrScan (Source, CharToScanFor);
IF Result = NIL THEN
Result := StrEnd (Source)-1;
END;
PROCEDURE ExtractName (Start : PChar; Terminators : TCharset; VAR Final : PChar);
(* Extracts the complete Name beginning at "Start".
It is assumed that the name is contained in Markup, so the '>' character is
always a Termination.
Start: IN Pointer to first char of name. Is always considered to be valid
Terminators: IN Characters which terminate the name
Final: OUT Pointer to last char of name *)
BEGIN
Final := Start+1;
Include (Terminators, #0);
Include (Terminators, '>');
WHILE NOT (Final^ IN Terminators) DO
INC (Final);
DEC (Final);
END;
PROCEDURE ExtractQuote (Start : PChar; VAR Content : STRING; VAR Final : PChar);
(* Extract a string which is contained in single or double Quotes.
Start: IN Pointer to opening quote
Content: OUT The quoted string
Final: OUT Pointer to closing quote *)
BEGIN
Final := StrScan (Start+1, Start^);
IF Final = NIL THEN BEGIN
Final := StrEnd (Start+1)-1;
SetString (Content, Start+1, Final-Start);
END
ELSE
SetString (Content, Start+1, Final-1-Start);
END;
(*
===============================================================================================
TEntityStackNode
This Node is pushed to the "Entity Stack" whenever the parser parses entity replacement text.
The "Instance" field holds the Instance pointer of an External Entity buffer. When it is
popped, the Instance is freed.
The "Encoding" field holds the name of the Encoding. External Parsed Entities may have
another encoding as the document entity (XmlSpec 4.3.3). So when there is an " 0 THEN BEGIN
ESN := TEntityStackNode (Items [Count-1]);
Result := ESN.LastPos;
IF ESN.Instance <> NIL THEN
ESN.Instance.Free;
IF ESN.Encoding <> '' THEN
Owner.FCurEncoding := ESN.Encoding; // Restore current Encoding
Delete (Count-1);
END
ELSE
Result := NIL;
END;
(*
===============================================================================================
TExternalID
-----------
XmlSpec 4.2.2: ExternalID ::= 'SYSTEM' S SystemLiteral |
'PUBLIC' S PubidLiteral S SystemLiteral
XmlSpec 4.7: PublicID ::= 'PUBLIC' S PubidLiteral
SystemLiteral and PubidLiteral are quoted
===============================================================================================
*)
TYPE
TExternalID = CLASS
PublicId : STRING;
SystemId : STRING;
Final : PChar;
CONSTRUCTOR Create (Start : PChar);
END;
CONSTRUCTOR TExternalID.Create (Start : PChar);
BEGIN
INHERITED Create;
Final := Start;
IF StrLComp (Start, 'SYSTEM', 6) = 0 THEN BEGIN
WHILE NOT (Final^ IN (CQuoteChar + [#0, '>', '['])) DO INC (Final);
IF NOT (Final^ IN CQuoteChar) THEN EXIT;
ExtractQuote (Final, SystemID, Final);
END
ELSE IF StrLComp (Start, 'PUBLIC', 6) = 0 THEN BEGIN
WHILE NOT (Final^ IN (CQuoteChar + [#0, '>', '['])) DO INC (Final);
IF NOT (Final^ IN CQuoteChar) THEN EXIT;
ExtractQuote (Final, PublicID, Final);
INC (Final);
WHILE NOT (Final^ IN (CQuoteChar + [#0, '>', '['])) DO INC (Final);
IF NOT (Final^ IN CQuoteChar) THEN EXIT;
ExtractQuote (Final, SystemID, Final);
END;
END;
(*
===============================================================================================
TXmlParser
===============================================================================================
*)
CONSTRUCTOR TXmlParser.Create;
BEGIN
INHERITED Create;
FBuffer := NIL;
FBufferSize := 0;
Elements := TElemList.Create;
Entities := TNvpList.Create;
ParEntities := TNvpList.Create;
Notations := TNvpList.Create;
CurAttr := TAttrList.Create;
EntityStack := TEntityStack.Create (Self);
Clear;
END;
DESTRUCTOR TXmlParser.Destroy;
BEGIN
Clear;
Elements.Free;
Entities.Free;
ParEntities.Free;
Notations.Free;
CurAttr.Free;
EntityStack.Free;
INHERITED Destroy;
END;
PROCEDURE TXmlParser.Clear;
// Free Buffer and clear all object attributes
BEGIN
IF (FBufferSize > 0) AND (FBuffer <> NIL) THEN
FreeMem (FBuffer);
FBuffer := NIL;
FBufferSize := 0;
FSource := '';
FXmlVersion := '';
FEncoding := '';
FStandalone := FALSE;
FRootName := '';
FDtdcFinal := NIL;
FNormalize := TRUE;
Elements.Clear;
Entities.Clear;
ParEntities.Clear;
Notations.Clear;
CurAttr.Clear;
EntityStack.Clear;
END;
FUNCTION TXmlParser.LoadFromFile (Filename : STRING; FileMode : INTEGER = fmOpenRead OR fmShareDenyNone) : BOOLEAN;
// Loads Document from given file
// Returns TRUE if successful
VAR
f : FILE;
ReadIn : INTEGER;
OldFileMode : INTEGER;
BEGIN
Result := FALSE;
Clear;
// --- Open File
OldFileMode := SYSTEM.FileMode;
TRY
SYSTEM.FileMode := FileMode;
TRY
AssignFile (f, Filename);
Reset (f, 1);
EXCEPT
EXIT;
END;
TRY
// --- Allocate Memory
TRY
FBufferSize := Filesize (f) + 1;
GetMem (FBuffer, FBufferSize);
EXCEPT
Clear;
EXIT;
END;
// --- Read File
TRY
BlockRead (f, FBuffer^, FBufferSize, ReadIn);
(FBuffer+ReadIn)^ := #0; // NULL termination
EXCEPT
Clear;
EXIT;
END;
FINALLY
CloseFile (f);
END;
FSource := Filename;
Result := TRUE;
FINALLY
SYSTEM.FileMode := OldFileMode;
END;
END;
FUNCTION TXmlParser.LoadFromBuffer (Buffer : PChar) : BOOLEAN;
// Loads Document from another buffer
// Returns TRUE if successful
// The "Source" property becomes '
BEGIN
IF Assigned (FOnStartTag) THEN FOnStartTag (Self, TagName, Attributes);
END;
PROCEDURE TCustomXmlScanner.WhenEmptyTag (TagName : STRING; Attributes : TAttrList);
// Is called when the parser has parsed an Empty Element Tag like
BEGIN
IF Assigned (FOnEmptyTag) THEN FOnEmptyTag (Self, TagName, Attributes);
END;
PROCEDURE TCustomXmlScanner.WhenEndTag (TagName : STRING);
// Is called when the parser has parsed an End Tag like